topic Re: HA session sync too slow? in General Topics

HA session sync too slow?

dmgeurts — Mon, 12 May 2025 13:48:09 GMT

I've recently migrated to a pair of active/active HA firewalls and am seeing some DNS return traffic dropped. Checking the logs, I can see that traffic is returned via another firewall as the DNS request was received. No problem, as normally the HA session sync is fast enough for the other firewall to have the session.

However, the DNS servers reply so quickly that the session state hasn't synced yet and the return traffic is dropped. Has anyone seen this, and any suggestions other than delaying DNS replies on the servers?

Forgot to mention that the pair of firewalls are PA-3410 with direct HCSI link in adjacent racks. So, latency should be as small as it gets.

Re: HA session sync too slow?

Brandon_Wertz — Tue, 13 May 2025 13:34:23 GMT

@dmgeurts wrote:

I've recently migrated to a pair of active/active HA firewalls and am seeing some DNS return traffic dropped. Checking the logs, I can see that traffic is returned via another firewall as the DNS request was received. No problem, as normally the HA session sync is fast enough for the other firewall to have the session.

However, the DNS servers reply so quickly that the session state hasn't synced yet and the return traffic is dropped. Has anyone seen this, and any suggestions other than delaying DNS replies on the servers?

Forgot to mention that the pair of firewalls are PA-3410 with direct HCSI link in adjacent racks. So, latency should be as small as it gets.

That really doesn't make much sense. 2 networked (routed) endpoints shouldn't be talking faster than firewalls directly connected to each other. I've never ran a A/A deployment so unfortunately I don't have much insight to share. I'm assuming you have the HA1, HA2, and HA3 interfaces defined? The HA3 link as I understand it is required for an A/A deployment.

Are your management and data CPUs all running at normal percentages?

Re: HA session sync too slow?

dmgeurts — Tue, 13 May 2025 14:03:42 GMT

Yes, all HA1/2/3 and their backups are connected. CPU on data and management planes is low. The DNS server caches entries and it's a very small percentage of traffic affected.

Agree with you completely. I did not expect this, so apart from CPU load anything I should be checking?

PanOS version is 11.1.6h something (off the top of my head)

Re: HA session sync too slow?

Brandon_Wertz — Tue, 13 May 2025 15:08:43 GMT

Unfortunately I don't have any other ideas, other than opening a support case getting TAC to take a look.