topic Re: A weird high latency ping issue with PaloAlto running version 8.1.15 hotfix 3 in General Topics

A weird high latency ping issue with PaloAlto running version 8.1.15 hotfix 3

dtran — Wed, 25 Nov 2020 17:04:29 GMT

I have this really weird issue and I don't know how to solve it. I noticed it when I first migrating from Checkpoint over to PaloAlto while doing verification in my Development environment.

- A PAN 5050 running 8.1.15 hotfix 3. I have a CentOS host (LinuxB) behind the PAN and another CentOS host (LinuxA) outside the PAN. FW rule is any any allow between these two host and the rule is at the top of the rulebase and nothing else. The PAN firewall is managed by Panorama, The interface on the PAN is VLAN tagging on a single physical interface,

- I do a 100 ping from LinuxA to LinuxB, the average RTT is 1.4ms. The Linux machines and PAN are connected to the same L2 Cisco 3750 switch. My background prior to PAN is Cisco so I am very familiar with Cisco products.

- I took out the PAN firewall and put in either a Cisco ASA or IOS router, using the same cable and switchport that I use on the PAN firewall. When I do 100 ping from LinuxA to LinuxB, the average RTT is 0.9ms, almost a 50% improvement in latency.

I go back and check the latency in my production environment regarding ping (we use open source smokeping tool to monitor our network/firewall devices) before the cutover from Checkpoint to PAN and I also notice the same thing.

I want to upgrade the PAN 5050 to 8.1.17 and test again; unfortunately, we ran out of support on the device so I could no longer download 8.1.17 for the 5050 but I don't think it would have made any differences.

Thoughts?

Re: A weird high latency ping issue with PaloAlto running version 8.1.15 hotfix 3

MP18 — Wed, 25 Nov 2020 21:29:31 GMT

@dtran

Does your ping rule has security profiles attached to it?

Other thing i can say is for application ping do a test without any security profile to see if it make any difference.

Also try ping from PA CLI by using source IP and see any difference.

Regards

Re: A weird high latency ping issue with PaloAlto running version 8.1.15 hotfix 3

dtran — Thu, 26 Nov 2020 01:56:58 GMT

Does your ping rule has security profiles attached to it?

Other thing i can say is for application ping do a test without any security profile to see if it make any difference."

No ping rule does NOT have any security profiles attached to it.

Re: A weird high latency ping issue with PaloAlto running version 8.1.15 hotfix 3

Broderick2352 — Thu, 26 Nov 2020 10:22:08 GMT

How did you check the connectivity between the mgmt interface and the internet?

Re: A weird high latency ping issue with PaloAlto running version 8.1.15 hotfix 3

dtran — Fri, 27 Nov 2020 12:46:13 GMT

Re: A weird high latency ping issue with PaloAlto running version 8.1.15 hotfix 3

dtran — Fri, 27 Nov 2020 12:48:06 GMT

as you can see in the graph, the high latency is ping across PAN firewall while low latency is across non PAN firewall. The endpoints are the same, L2 switchport is the same. The only difference is the firewall.

Re: A weird high latency ping issue with PaloAlto running version 8.1.15 hotfix 3

BPry — Fri, 27 Nov 2020 17:03:26 GMT

@dtran,

ICMP traffic is never fast tracked, and while a 50% increase sounds bad your talking about roughly a .5ms difference. What happens if you actually try testing with legitimate traffic (websites, DB calls, ect)?

Re: A weird high latency ping issue with PaloAlto running version 8.1.15 hotfix 3

dtran — Mon, 08 Feb 2021 18:09:28 GMT

50% difference does sound bad.

Re: A weird high latency ping issue with PaloAlto running version 8.1.15 hotfix 3

Daniel_Garry — Mon, 28 Nov 2022 10:59:48 GMT

Hi ,

Did you ever get an answer for this? We have a similar issue, I swapped out SW FW for Palo and the pings went from >1ms to 2/3 ms even the odd 16ms.

All traffic is one the LAN same intrazone and has no profiles associated. We are seeing people having issues now with applications etc

Re: A weird high latency ping issue with PaloAlto running version 8.1.15 hotfix 3

dtran — Mon, 28 Nov 2022 11:20:59 GMT

@Daniel_Garry: The response I received back from PAN support, after wasting so many hours with them, was that "it is expected". To me, that is NOT an acceptable response, at the same time, I didn't want to waste anymore of my time to fight PAN support any further. The support from TAC was not good in this situation.

Re: A weird high latency ping issue with PaloAlto running version 8.1.15 hotfix 3

Daniel_Garry — Mon, 28 Nov 2022 11:47:28 GMT

Great thanks. I have a case opened with them now also, but feel like they won't give a good answer

Re: A weird high latency ping issue with PaloAlto running version 8.1.15 hotfix 3

dtran — Mon, 28 Nov 2022 13:36:34 GMT

No they will not. If anything else, they will waste your time and resource.

Re: A weird high latency ping issue with PaloAlto running version 8.1.15 hotfix 3

OtakarKlier — Mon, 28 Nov 2022 17:34:38 GMT

Hello All,

Hopefully you are running a newer version of PANOS as t8.x is very old. Please check the following for newer version recommendations:

https://live.paloaltonetworks.com/t5/customer-resources/support-pan-os-software-release-guidance/ta-p/258304

Regards,

Re: A weird high latency ping issue with PaloAlto running version 8.1.15 hotfix 3

dtran — Mon, 28 Nov 2022 17:36:32 GMT

FYI: I am running version 10.1.6-h6 and I am still seeing the ping latency issues.

Re: A weird high latency ping issue with PaloAlto running version 8.1.15 hotfix 3

OtakarKlier — Tue, 29 Nov 2022 22:40:10 GMT

Hello,

Check out this article and see if it helps:

https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA14u000000HBjNCAW

Regards,