2050 only handles 1/4 of its advertised throughput

[jambulo]

Has anyone here tried to benchmark there Palo Alto Firewalls? We are using Breaking Point(same company that Palo Alto uses)to test our Lab 2050's.  We have come to the conclusion that the PA 2050 starts dropping packets at about 250Mbps(with about 5-600 new sessions per second).  This is with Threat Prevention disabled.  The 2050 is spec'd out to be able to handle 1Gbps of Firewall traffic with Threat Prevention disabled.  The Breaking Point is acting as the Client and the Server.  We put the rules that allow the Breaking Point traffic at the very top.  The Breaking Point Client is doing a simple HTTP GET request, and the Breaking Point Server responds with a 44k text file.

Running this command...

"show counter global filter severity warn delta yes"

...these counters normally have the highest hits...

-Software packet buffer allocation error

-packets dropped because of failure in tcp reassembly

-out-of-window packets dropped

[ericgearhart]

This test was conducted using some pretty idealized conditions as well (as idealized as we can come up with)... HTTP GETs between a client and server, with only the PA 2050 acting as the router/firewall between the two subnets. Also we made sure App-ID caching was enabled, just in case turning that off had a performance impact.

[bnelson]

Hi,

There is also a resource limitation on the packets per second (~20000 - 25000) that the PA-2000 series hardware can handle, in my experience. 44k is a small packet, resulting in a fair amount of packets per second. What is the total throughput is you increase the packet size to 1500k?

Ben

[mikand]

44k is 44000 bytes (or 45056 bytes depending on if you count k as 1000 or 1024)... meaning contains several 1500 bytes packets already if thats what you mean - of course unless the test performed forced to use smaller packets?

Jambulo: Could you paste the config used so the community forum can take a peak in case there is something in there which (for performande reasons) isnt properly setup?

Also verify so you dont have bad interfaces or bad cabling which could end up with resends which of course will get you lower values.

The 2050 is speced for:

(1) Gbps firewall throughput (App-ID enabled1)

500 Mbps threat prevention throughput

300 Mbps IPSec VPN throughput

250,000 max sessions

15,000 new sessions per second

With the note of:

(1) All performance and capacities are measured under ideal testing conditions using PAN-OS 5.0.

where it previously was written that the test (to get the performancenumbers) was using a specific size of http-transmissions (if I recall it correctly).

Looking at other performance tests you could try a payload of 1megabyte to see if you can reach topspeed, however number of transactions per second will drop compared to lets say when using 64k payload (which will have far more transactions per second but only slighly lower throughput counted in Mbit/s).

To max out transactions per second you can go down to 4k payload which will drop the Mbit/s but show you peak in transactions per second (or even down to 512 bytes I think).

Another thing to test is to enable jumboframes (both on the PA device and on the loadgenerator) and see how that will affect transactions per second and throughput for a given payloadsize.

Yet another thing to test is to use a single download but lets say 1gigabyte of payloadsize and see which throughput you get with that.

So to max out throughput numbers a test something like this could be performed:

1) Use a single security rule that is setup like:

srczone: any

dstzone: any

srcip: any

dstip: any

appid: any

service: any

threat: none selected (like no IPS, AV, FILE, URL etc).

options: none select (like no logging etc).

2) Verify that both ends are up with 1Gbit/s full duplex (or 10Gbit/s if you use such interfaces) - connect directly to the PA device (like no switch or such in between).

3) Enable jumboframes on both the PA device and the loadgenerator.

4) Perform the test with a single transaction but with 1Gbyte (or more) as payloadsize.

If the above doesnt get you close or above the numbers presented in the datasheet I would contact support to verify that there is nothing wrong with the particular PA unit (like RMA or such).

If I recall it correctly NSS Labs got optimal performance numbers of 115% above the one stated in the datasheet - these numbers will of course vary or for that matter go down if you have smaller payloadsize etc.

[ericgearhart]

Thanks for the reply mikand... yes we were definitely trying to recreate something near the NSS Labs results. We went into this hoping that we'd be pleasantly surprised at the 2050's performance numbers given NSS Labs' numbers.

[ericgearhart]

mikand wrote:

So to max out throughput numbers a test something like this could be performed:

1) Use a single security rule that is setup like:
srczone: any
dstzone: any
srcip: any
dstip: any
appid: any
service: any
threat: none selected (like no IPS, AV, FILE, URL etc).
options: none select (like no logging etc).

We created something similar to this... I built an "outside" zone and an "inside" zone, with them bound to one interface each. I had one rule at the top of the rulebase for the test, testing from outside -> inside for application web-browsing. Also the Service column was set to "service-http."

We tried with logging on and logging off as I recall... we still weren't able to get decent numbers out of the PA2050.

Also we checked the switch interfaces throughout this test (show int gig1/1, show int gig1/2, etc) and they were clean throughout the testing.

[nisse]

I did some simple benchmarking when doing VPN testing and got better speeds than that over IPSEC.

What release are you running? I have a few 2050s in our lab so I can have a go at it some time this week to see if I can reproduce your problem.

As I understand you have two zones and an allow all rule on service http with one interface assigned per zone?

What tool did you use for testing?

[ericgearhart]

nisse wrote:
What release are you running? I have a few 2050s in our lab so I can have a go at it some time this week to see if I can reproduce your problem.

We have two lab PA2050s... we tested with both 4.1.11 and 5.0.2 as I recall

nisse wrote:

As I understand you have two zones and an allow all rule on service http with one interface assigned per zone?

That is correct... we used the "web-browsing" App-ID too.

nisse wrote:

What tool did you use for testing?

We used IXIA's BreakingPoint to test. The BreakingPoint device was acting as the client and the server (two separate interfaces). We actually had a BreakingPoint engineer here during the tests as well.

FYI, I would honestly love to be proven wrong! I like our PA devices, and I'm trying to push for using them more in our network designs.

[ericgearhart]

An update on this thread... PA is sending us an eval 5000 series box and an eval 500 series box, and they promise they're going to hook up us with their BreakingPoint test profile so we can import their settings and hopefully recreate their own findings.

I'll add to this thread when we get the boxes, get them racked, apply the BreakingPoint test case, etc.

[ericgearhart]

As a followup to this, we're still waiting on PA to provide us with their in-house Breaking Point test profile.

[mikand]

Did you refresh your contact with PA so it isnt that somebody is on vacation, sick or even left PA all together or such?

Regarding 2000 series, if I would about to buy some new gear I would look at the 3000 series instead of the 2000 series. Much better commit times and better SSL decryption performance (in terms of concurrent ssl decryptions) aswell.

And in case 3000 isnt enough I would go for 5000 series.

[ericgearhart]

Just to add to this, PA shipped us a PA-500 and I am successfully meeting or beating their spec sheet numbers using the exact same testing methodology we used for this thread.

See http://media.paloaltonetworks.com/documents/PA500_Specsheet.pdf for the spec sheet I'm referring to.

We're getting around 300 to 350 megs with no threats turned on with the PA500, and around 200-250 megs with threat profiles turned on.