cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Who rated this post

L1 Bithead

Hello,

We had similar case for 3 devices, problem with rcv_fifo_overrun counters increasing on external interface directly connected to the ISP router. 

Devices were on 8.1.5 and 8.0.12 PANOS, model PA820.

First we thought problem might be due to high packet burst where device can't handle it properly and simply drops the packets, also we suspected PAN-104116 leak or another thought was that the device could be simply to busy and drops the packets that can't handle.

Leak was fixed in 8.1.5 and we try the upgrade path, which didn’t help in the end.

Then when we noticed that 2 more devices (same model PA820) had the same problem, we have logged the case with PA TAC.

Upon troubleshooting engineer provided some more insight based on the past experience:

Data receive buffer overrun - is only indication of ingress traffic bursts on the marvell switch port(Firewall DP Ports) .rcv_fifo_overrun errors are suggesting that Marvell(PA's switch backplane) was experiencing buffer shortage/switch port on Marvell fabric is receiving more frames than can be managed and this can be related to bursty traffic pattern.

 

Errors can only be found on Marvell external ports (phy ports, RJ45) and not between internal

chipsets(From Marvell ---> Dataplane processor). Buffer size cannot be changed/adjusted.

> debug dataplane internal pdt marvell stats

 Capture.JPG

 

What we are interested in is the UL1 and UL2 (Uplinks to Octeon Processor) and

they appear to be good, in all of them indicating marvell is not sending pause frames or

any recv errors caused by hardware.

 

 Also the traffic rate in my case and for that particular device was above 20K pps Rcv_fifo_overrun is expected/normal during bursty traffic pattern and it's not an issue on the firewall.

 

Problem is only when the traffic rate is low and if this counter increases, even with reasonable amount of traffic, then it needs investigation.

 

Also this doesn't have relevance to packet-descriptor depletion or leak as I have thought originally with reference to PAN-104116.

Because PD is a DP component and "rcv_fifo_overrun"  is drop stage in marvell even before it makes it to DP.

 

In my case from TAC perspective there is not much we can do.

 

Kind Regards,

Eli

 

Who rated this post