Testing Palo Alto's NGFW capabilities with the Cisco ENCS Platform

Summary

In this specific scenario, they asked us to lab up an environment in the WWT Advanced Technology Center (or ATC) that would help them evaluate Palo Alto's capabilities around virtual firewalls.  They specifically asked for the Palo Alto VM-300 and it needed to live directly on Cisco's Enterprise Network Compute System (or ENCS) which was serving as the Universal Customer Premises Equipment (or uCPE).  

We tested several scenarios and use cases specified by our customer around Palo Alto's virtual firewall capabilities.

ATC Insight

We tested several scenarios as I mentioned in the Overview Section, and we will go into depth on three different testing scenarios that were important to our customer to obtain relevant results.  The higher-level testing scenarios were:

• Bandwidth Traffic Testing
• Firewall Efficacy Testing
• Redundancy and Disaster Recovery Testing
   

Bandwidth Traffic Testing

The maximum throughput of the IXIA Traffic Profile AppMix through the Palo Alto VM-300 HA pair was 217.9 Mbps.  The traffic flow used to achieve that rate was the following three IXIA Flow Indexes that were defined in the initial scope of the Proof of Concept (or POC).   Additionally, we ran the IXIA Flow Index for Internet traffic alone and increased the flow rate. Due to the larger transaction sizes associated with this Flow Index comprised of HTTP, HTTPS, and DNS, we observed a maximum throughput of 383.5 Mbps. SR-IOV was enabled on the ISRv Gi2 interface. Data Plane Development Kit (or DPDK) was enabled globally on both Cisco ENCS chassis. The Palo Alto VM-300s also had DPDK enabled. 

Testing in this POC for the Palo Alto Networks VM-300 firewall ran on 4 CPU cores with the default allocation of 2 + 2 (data plane + management plane).  Palo Alto today also has a 5-core solution that we did not test in this POC in the ATC.  

Firewall Efficacy Testing

While sending strikes (only) from our IXIA toolset, our team observed that the Palo Alto VM-300 firewall was able to block a majority of the strikes that we threw at it via our IXIA BreakingPoint Malware Strike Lists.  Additionally, when we sent strikes alongside the IXIA Traffic Profile AppMix (traffic that mimics customer traffic) the firewall was able to block the same majority of strikes which indicated that the Palo Alto VM-300 would work properly in a production customer scenario. All strikes were sent in the clear from IXIA without an SSL/TLS evasion profile enabled. SR-IOV was enabled on the ISRv Gi2 interface. DPDK was enabled globally on both ENCS chassis. The VM-300s also had DPDK enabled. 

The Palo Alto solution can also provide additional analysis and adjust action based on real-world traffic to avoid false positives to minimize unnecessary customer traffic disruption.

Redundancy and Disaster Recovery Testing

There were a plethora of tests that were involved to cover the redundancy and disaster recovery scenarios in the mimicked customer environment.  

Here is a high-level list of the test cases that were necessary:

• WAN failure on ENCS-A and how traffic is effected to ENCS-B
• WAN restoration from ENCS-B back to ENCS-A and how traffic is effected
• ISRv-A failure or out of service and how traffic is effected to ISRv-B
• Restoration from ISRv-B back to ISRv-A and how traffic is effected
• ISRv-A LAN virtual interface shutdown on ENCS-A
• Restoration of ISRv-A LAN virtual interface and how traffic is effected
• Palo VM-300-A failure or taken out of service and how traffic is effected to Palo VM-300-B
• Restoration from Palo VM-300-B back to Palo VM-300-A and how traffic is effected
• Physical shutdown or disconnect of cables to ENCS-A LAN interfaces
• Restoration or connect cables to ENCS-A LAN interfaces
• Shutdown or disconnect the Interconnect Link between ENCS-A and ENCS-B
• Restoration of Interconnect Link between ENCS-A and ENCS-B
• Power off ENCS-A
• Restore Power to ENCS-A

The good news is that all of these tests were successful in terms of redundancy and disaster recovery.  The ATC Lab Services team leveraged a specialized app mix of 1000 UDP packets per second for these tests in order to show any granular loss, if any, with what we consider sensitive traffic flow (UDP packets).  In most of the test cases above normal UDP packet loss was encountered and expected during failover and restoration.  

Additional observations in traffic behavior were around "Split Brain" scenarios.  For example, when the Interconnect Link between ENCS-A and ENCS-B was disconnected, the team observed an HSRP "Split Brain" condition in which both ISRv hosts went active for the associated HSRP groups. No loss was observed as a result of the "Split Brain" condition.  When restored, no loss was observed and ISRv-A became the primary for all groups again. 

Final Thoughts On The Solution

Our customer gained quite a bit of knowledge around the Palo Alto VM-300 Firewall and how it can be used on Cisco's ENCS Platform.  They were able to explore security upgrade options within their current NFV platforms by testing Palo Alto VNF capabilities. 

• The redundancy and disaster recovery testing helped our customer understand exactly how chained NFV components will behave upon the loss of various resources. 
   
• The bandwidth traffic testing exposed several throughput considerations based on the type of traffic being utilized.  For example, we achieved line rate bandwidth scale with TCP packet blasting, but with true negotiations and conversations being mimicked at scale with our BreakingPoint tool, we yielded significantly less throughput through the Palo Alto VM-300 (which is to be expected).  For this testing, our customer gained a valuable understanding of how to right-size the solution based upon their needs.
   
• Finally, through the firewall efficacy testing our customer became knowledgeable with how the Palo Alto VM-300 was able to block malicious traffic and even more importantly how tuning would occur in Day 2 operations.

Expectations

Our customer at a high level wanted to gain visibility into the overall functionality of the Cisco ENCS appliance that will host both and ISRv router VNF and a firewall VNF.  At a deeper level, they specifically wanted to understand how Palo Alto's Firewall VNF would function and perform under production like circumstances.  The tests that we ran in the Advanced Technology Center (or ATC) around Palo Alto would help our customer save time and money by accelerating the tests and observations with this Proof of Concept (or POC).

Technology Under Test

uCPE technology
Cisco Enterprise Network Compute System (or ENCS platform)

Palo Alto Next-Generation Firewalls (Virtual) technology
Palo Alto Virtual Network Function (or VNF) PA-VM-300

Testing Components

Devices Under Test

VNF Software Versions

We used the current Cisco NFVIS release version 3.12.2-FC. Release notes can be found here. The customer selected the OS versions for the VNFs per direction from Cisco and Palo Alto in regards to the solution. See the above table for OEM-recommended software versions at the time of this writing (July 2020).

VNF Features Enabled

• Cisco ISRv - Flexible Neflow, NBAR, NAT
• Palo Alto VM-300 - Threat Module, Anti-virus, Anti-spyware, vulnerability protection, URL Filtering, logging to Panorama, DNS inspection

VNF Resource Requirements

Below are the resource allocations to run each VNF.

• Cisco ISRv - datasheet here
  • vCPU: 4, RAM: 4GB, Disk: 8GB
• Palo Alto VM-300 – datasheet here
  • vCPU: 4, RAM: 9GB, Disk: 60GB

Supporting Infrastructure and Tools

IXIA Application Profiles Used In Testing

The sample target distribution below that was used to mimic customer traffic production flows and model the IXIA application mix.  

Lab Diagram (Some physical and logical depiction)

This is a physical and logical depiction of some of the components and environment that was built specifically for this Proof of Concept (or POC) in the ATC for this effort.  You can see the positioning of the Cisco ENCS platform as well as some of the logical breakouts of the VNFs needed in this solution.

Test Tools

Test Tools

For this Proof of Concept (or POC) we utilized the BreakingPoint tool from IXIA because of its capabilities around simulating real-world traffic up to layer 7, malware, and exploits.  Using this tool gives the ATC Lab Services team a very valid and industry-standard way to help our customers test out security solutions. If you would like to learn more about this product please visit IXIA's BreakingPoint page.

(Figure A) A quick depiction of just a slice of reporting and metrics that we gather when executing POCs with our customers in the Advanced Technology Center (ATC).  We used the reporting and metrics out of the BreakingPoint tool quite extensively so our customer would have recorded metrics to evaluate the Palo Alto NGFW solution running on Cisco's Enterprise Network Compute System Platform (or ENCS Platform).