Packet Descriptors spike

[abraun]

Hi All,

I've been collecting and plotting CPU, Session, Packets Descriptors, Packets Per Second, as well as some other metrics. Once in a while I see a spike on the Packet Descriptor graph.

According to KB article How to Interpret: show running resource-monitor :

The cores specified in the CPU usage output have dedicated functionalities:

• Core 0: used for Management Plane (MP) and Data Plane (DP) communication
• Core 1: used for session and flow management
• Core 2 and above: used to process network traffic and log processing

Same article talks about: If the output of packet descriptors or buffers stays > 80%, it indicates the device is over loaded and may lead to packet loss and abnormal behavior of the device.

Question: Is there a way to pinpoint WHAT exactly contributes into Packet Descriptor spike? None of the other graphs show anything for the particular period of time. CPU is normal, Session util is fine, Packets per Second are fine for each zone. Is there anything in the logs that I should search for?

Thanks!

[mikand]

Whoch model and which PANOS version?

[Phoenix]

Hello Abraun,

When there is a scenario where the packet descriptors are High as in more than 80% for some time and staying there, it indicates lot of dataplane traffic is being processed.

Now there are many ways to look into it,

1> See if there are any attacks or threats, this can be seen in Threat logs and its frequency.

2> Implement DOS protection or Zone protection as needed and see if there are any Syn attacks or so on.

3> Wonderful idea is to look for ACC tab and see what kind of applications were running at the time when descriptors were high. Find the time frame and select the time range in ACC and filter the results. Ideally now that should tell us which applications were seen which were consuming DP resources. Also from the Bytes consumed per APP we can know narrow down the most used app.

> Look for the Dataplane counters available in dp-monitor.log at the time of issue. This will be the pointer towards any issues. They may not be very clear to understand but some of them are clear to point out the cause. If you compare the counters from some time ago when packet descriptors were good, we get good lot of stats as to which counters increased, giving us greater granularity of knowledge.

Hope all this helps !

[rvandegrift]

Sorry to follow up to an old post, but I'm interested in understanding more about this issue.  You say high packet descriptors indicates lots of dataplane traffic is being processed - can you elaborate?  Is this any traffic handled in the dataplane, is it only slowpath dataplane traffic, traffic subject to DoS policy, etc?

I've seen degraded performance on a PA-5060 running 6.0.3, with high packet descriptors, and I'm trying to understand the cause.  There was a large spike in DNS traffic at the time, but almost all of it was dropped by a DoS policy.  The permitted DNS traffic was much lower in volume and sessions than other normal traffic - approximately 4000-6000 sessions of it were active, compared to a total of 1.2 million sessions.

According to the specs, a 5060 can setup 120k sessions per second.  I assumed that means full session setups, and excludes packets denied by DoS protection policies.  Is this understanding correct?

[Rameshwar]

 Look for the Dataplane counters available in dp-monitor.log at the time of issue ... how can i check this ?

[snekkanti]

Hi,

We see latency issues on Firewalls running on 8.1.6. Rule for the traffic is service based rule with port 80, 8081 & other ports.  User seems to be using "jfrog-artifactory' app for some file sharing and experienced latency. When we look into the resource monitor, packet buffers & sessions looks good but packet descriptors (on chip) (maximum) reaching 100.  An App-over ride rule for application "jfrog-artifactory' on port 80, 8081 is applied for ingress and egress traffic and values on packet descriptors value looks idle but we see 100 spike intermetivley. Any kind of answer is helpful to reslove this issue, 

Thanks 

[tmhorne]

We're seeing the same on 8.1.8 largely for git traffic. I believe it's a bug. Did you put in a ticket? 

[MP18]

We see this on PA 5220 running 8.1.9.

Packet  descriptor average is below 80 but spike goes to 100 very often