HIP Check reports fail to send to internal gateway following internal gateway certificate change or patching of firewall

[timspenc]

Hi all, we've recently had a couple of issues which have caused us to investigate client Global Protect connectivity issues.

The first was off the back of the recent security patching of our firewall HA pair.

The second was after replacing the internal gateway certificate on the firewall HA pair.

We run an Active/Passive pair on 9.2.6-h3

Global Protect client is on 6.2.2.

The security patching was done via manual failover to maintain connectivity. This seemed to be OK however we then noted our clients that were connected before we patched and failed over, had to restart their GP vpns around 3 hours later. Upon further investigation it would appear the client was unable to send the HIPreportcheck successfully every hour following the patching. Consequently they failed this process 3 times and their user agent was no longer sent to the firewalls. All looked fine on the sessions on the firewall which appeared to still be renewing automatically around the halfway point of the 10800 session countdown. show session user id mapping showed they were counting down from 10800 and renewing round the 5400 second mark so clearly some sort of communication was still working.

We didn't get to the bottom of this at the time.

We then recently had to renew the internal gateway certificate on the firewalls. Again we noted clients had to restart their GP vpns around 3 hours later. This time the hipchecks were failing to verify the internal gateway certificate so failed 3 checkins and cut the customer off.

We've raised a ticket with our support company for both of these but having come up with a sensible resolution yet.

Has anyone else seen these sort of issues before? Its like the client token doesn't get carried over in an upgrade failover and consequently times out rather than reconnecting automatically if necessary. Same with the certificate replacement. Failed to verify the gateway certificate and the firewall logs show Decrypt failed each time the client device attempted to send the hipcheck report until the customers restarted the vpn which then obviously did a full gateway certificate check and then started working again with the new one.

Suggestions from the support company have been to increase the idle timeout for the GP client however this will only prolong the time before the client gets disconnected as the hip check connections will still be failing in the background, just for a longer period of time.

Surely we can't be the only ones experiencing these issues?

Many thanks all!