Global Protect Client disconnect Issues after upgrading to Pan-OS 11.1.0

[UtkarshKumar]

Hi Team

We upgraded Palo Alto FW to 11.1 and started having VPN Global Protect Client Issues where it would disconnect/reconnect multiple times.

It would get connected and there wouldn't be any internet access, wouldn't allow any traffic so something isn't working. Flipped back over to the passive firewall which hadn't been upgraded yet and everything stabilized.

Seems to be some issue with the 11.1.0 and GP Client versions:6.21. 6.22 and 6.20 clients. We tried connecting with all of them

In PanGPS.log

We see multiple disconnects:

(P4084-T2576)Debug(6677): 11/30/23 13:08:42:870 NetworkDiscoverThread: PortalStatus is 2, HasLoggedOnGateway is 1
(P4084-T2576)Debug(6810): 11/30/23 13:08:42:870 Reset NetworkDiscovery waitTime to 5 seconds.
(P4084-T2576)Debug(6151): 11/30/23 13:08:42:870 NetworkDiscoverThread: wait for network discover event.
(P4084-T15224)Debug(5374): 11/30/23 13:08:42:871 Found virtual IP route entry
(P4084-T15224)Debug(6137): 11/30/23 13:08:42:871 No change for gateway route
(P4084-T16620)Debug(12635): 11/30/23 13:08:42:871 m_preUsername cdougall
(P4084-T16620)Debug(1449): 11/30/23 13:08:42:871 m_msp->IsVPNConnected() is 1, CControlManager::GetInstance()->IsInRetry() is 0
(P4084-T16620)Debug(7692): 11/30/23 13:08:42:872 --Set state to Disconnecting...
(P4084-T16620)Debug(1499): 11/30/23 13:08:42:872 AddAttribute for proxy agent
(P4084-T16620)Info (2744): 11/30/23 13:08:42:873 Disconnect(VPN recv failed) called
(P4084-T1664)Debug(5002): 11/30/23 13:08:42:873 LifeTimeThread receives m_hExitLifeTimeThreadEvent
(P4084-T1664)Debug(5030): 11/30/23 13:08:42:873 LifeTimeThread quits
(P4084-T16620)Debug(1132): 11/30/23 13:08:42:873 vpn disconnect

We see this repeating every 30-40 seconds:

(P4084-T17104)Debug(6677): 11/30/23 13:13:05:869 NetworkDiscoverThread: PortalStatus is 1, HasLoggedOnGateway is 1
(P4084-T17104)Debug(6810): 11/30/23 13:13:05:869 Reset NetworkDiscovery waitTime to 5 seconds.
(P4084-T17104)Debug(6151): 11/30/23 13:13:05:869 NetworkDiscoverThread: wait for network discover event.
(P4084-T17312)Debug(5374): 11/30/23 13:13:05:870 Found virtual IP route entry
(P4084-T17312)Debug(6137): 11/30/23 13:13:05:870 No change for gateway route
(P4084-T17280)Debug(12635): 11/30/23 13:13:05:870 m_preUsername cdougall
(P4084-T17280)Debug(1449): 11/30/23 13:13:05:870 m_msp->IsVPNConnected() is 1, CControlManager::GetInstance()->IsInRetry() is 0
(P4084-T17280)Debug(7692): 11/30/23 13:13:05:871 --Set state to Disconnecting...
(P4084-T17280)Debug(1499): 11/30/23 13:13:05:871 AddAttribute for proxy agent
(P4084-T17280)Info (2744): 11/30/23 13:13:05:871 Disconnect(VPN recv failed) called
(P4084-T8188)Debug(5002): 11/30/23 13:13:05:871 LifeTimeThread receives m_hExitLifeTimeThreadEvent
(P4084-T8188)Debug(5030): 11/30/23 13:13:05:871 LifeTimeThread quits
(P4084-T17280)Debug(1132): 11/30/23 13:13:05:871 vpn disconnect
(P4084-T17280)Debug(1133): 11/30/23 13:13:05:871 Delete m_vpn in CControlManager::DisconnectVPN()

We have checked and there is no known issue reported or at least documented in the Palo Publick link. Anyone faced this similar issue?

Thanks and Regards

GlobalProtect 

[TTNhan]

Thanks Bro.

[TTNhan]

thanks bro for confirmation

[Zorgnet]

We had the same issue upgrading from 10.0.11-h3 to 10.1.11-h4

we had to rollback

[joshuahill9]

"10.2.7 -> 10.2.7-h3. After downgrade back to 10.2.7 GlobalProtect VPN is stable."

This is our exact situation. We only installed h3 to be ready for "2024 Certificate Expiration"

I cannot leave our systems in a broken state anymore, so I do hope they are fixing it before any sort of cert-apocalypse.

[emr_1]

10.2.8 was just released. Even I'm not testing it yet, share this info with you all.

[Bdubs324]

I'll add to the sentiment here, we upgraded to 11.0.3-h3 from 11.0.2-h2 to cover the BGP CVE and experienced the same behavior. GP version didn't seem to change the behavior, we tried multiple GP versions. Currently sitting on GP 6.0.7 and PAN OS 11.0.3-H3. We disabled IPV6 on the PANGP adapter and haven't had issues since.

[holbem]

I have tested 10.2.8 this weekend and despite what is in the release notes it's not fixed for me. I had to rollback to 10.2.7 (without HF). GP Client disconnected and no ip traffic was passing on GW with 10.2.8

[emr_1]

Really? Hmm... it looks to be fixed on my platform (PA-3220) with 10.2.8.

[holbem]

I did some more tests today and those are the results:

1. Disabled IPv6 on my Wi-Fi Adapter (on MAC with sudo networksetup -setv6off Wi-Fi)

I am still getting a Tunnel IPv4 and IPv6 address assigned (as expected) and getting disconnects and no IPv4 and IPv6 communication possible

1. Enabled IPv6 on my Wi-Fi again and de-configured IPv6 Tunnel IP’s on the GlobalProtect Gateway (simply removed the IPv6 IP Pool from the list)

I am only getting IPv4 Address as expected on the Tunnel and all works as expected (except IPv6 communication inside the tunnel)

So the error seems not to be related on the local Ethernet Adapter getting v6 Adresses or connecting to v6 Gateway. The problem seems to be with v6 inside the tunnel.

Same configuration is working on PanOS 10.2.7 without any problems.