topic Re: 2050 running high dataplane CPU in General Topics

2050 running high dataplane CPU

Lopes — Wed, 07 Mar 2012 15:29:46 GMT

Our PA-2050 is consistently running at 70-85% on the dataplane CPU despite running at 1/5 of the advertised maximum specs (40-40k sessions and 100-110mbps). I understand that the specs listed are best case scenario and don't expect to get close but I do expect better performance than what I am getting. It could be that our device is just wonky so I wanted to see if anyone else has seen this. We only have 10-12 security policies. No QoS, SSL Decrypt, etc.

Re: 2050 running high dataplane CPU

mikand — Wed, 07 Mar 2012 22:50:23 GMT

According to tests performed by NSS Labs the PAN devices datasheets are underrated rather than overrated. One of the tests showed 115% performance of the stated throughput for a particular model (I think it was a box from the 4000 series).

First of all having a high "cpu usage" for the dataplane isnt a problem as long as the usage isnt at 100% which you will then notice will affect the latency and then throughput and stuff like that.

Which PANOS do you use and when was it you last rebooted the device?

Of course rebooting the device should only be necessary doing firmware updates but if there is something stalled which gets fixed by a reboot (but then returns after some time) you should contact your sales rep. to file a support case so some support engineer can bring you a list of commands to run to identify what goes wrong (in case there is a hardware malfunction or something else).

I have seen reports in this forum that older versions of 4.1 might have had some issues but it seems that 4.1.3 and 4.1.4 solved those.

Re: 2050 running high dataplane CPU

Lopes — Thu, 08 Mar 2012 14:48:02 GMT

Thanks for the response.

My concern is that I am running high CPU on a device which should be capable of 5x's the performance I am seeing currently. There are several additional services I would like to utilize (QoS for example) and additional capacity that I would like to add however, I don't dare risk it seeing as the device is already running close to its limit.

Regardless of how it tested out, I would never presume I could run a device at or close to 100%. This is asking for serious issues. We have already throttled back the capacity to maintain the current (high) utilization.

It has been rebooted at least twice in the last couple weeks. It is running 4.1.2 currently however I saw the same utilization while on 4.0.7. We have 4.1.3 loaded but are waiting for a maintenance window to reboot the device following the patch.

Re: 2050 running high dataplane CPU

mikand — Thu, 08 Mar 2012 23:03:21 GMT

I guess you have already seen a similar case in this thread?

https://live.paloaltonetworks.com/message/11918#11918

Re: 2050 running high dataplane CPU

Lopes — Thu, 08 Mar 2012 23:21:13 GMT

I had not. Thanks for pointing it out. There are some interesting correlations. There really wasn't a mention of a fix. It sounded more like they just cranked it up and it has been able to handle the additional load. However it would be interesting to see how it faired with 1M+ sessions. That would be about half the advertised max (session wise). They were getting ~7700 session/CPU% at 600k but that would max the device out at 770k sessions unless greater efficiency was realized.

Another interesting correlation is that we have been seeing sporadic high management plane CPU as well (hitting 100% several times) as they have. I am going to lower some of our logging levels and see what happens.

Do you know if the dataplane CPU $ that is displayed on the dashboard is the same as is displayed in the CLI when you do "show running resource-monitor"? I have started to monitor that as I have been told it is a true indicator of the dataplane CPU usage.

I will be patching to 4.1.3 tonight so maybe that will help.