Problems connecting to Globalprotect after users install latest windows Cumulative updates

[jclements]

There seems to be a bit of an issue connecting to Globalprotect after our windows machines have the latest microsoft cumulative updates, KB5018410 (windows 10) and KB5018418 (windows 11).

Looking in reddit it looks like other users are seeing the same problem as well, anyone got any ideas on how to fix this going forward? The only way we've been able to get users to connect is by uninstalling the latest update.

I've raised a call with our partner support but havent got anything back yet.

thanks

[mlinsemier]

Interesting and thanks for clarifying as our certificate is 366 days.  I will see about getting this regenerated and see if it makes a difference.

[LFura]

The only other thing I had done prior to installing the updated certificate was to go in to the Device->Certificate Management->SSL/TLS Service Profile and changed my service profile from Min Version TLSv1.0 to TLSv1.2 and had left the max version at Max and commit it but that did not fix the issue.  So then I generated a new go daddy cert and installed it and committed it and it started working.  So then I went back and changed the min version back to TLSv1.0 and committed that and it was still working.  Also I just want to clarify that I updated the certificate that is installed on the Palo Alto that is used in my SSL/TLS Service Profile which that profile is then used by my Global Protect Portals and also for the public facing Global Protect portal.  I don't require the global protect clients to have a certificate installed which I think is an additional security option.  This is the certificate that if you went to https://yourportaldnsname in a browser would show up in the developer tools.  My original one was still valid until 10/24/2022 but it was created on 9/22/2021 so while it was still valid it was issued more than 365 days ago.  But maybe messing with the min version setting in the tls profile did something even though I set it back to the way it was?

[marcelmikusik]

After 5 days , we really don't have official statement about issue?

[ekneebone]

Within my support case the engineer acknowledged they are looking into it, that is as far as they were willing to go it seems.

I've most of the recommended changes without luck, TLS1.2/Max, new cert, most versions of GPVPN released in the past year, etc and not much progress.

[ChrisCon2355]

We're supporting a few customers.
I can connect from same Global Protect client to Palo Alto customer1, but not to customer2. Since Windows 10 update.
Both have official *.domain certificates from the same issuer.
When connecting to customer1 a part of the certicate is shown in PanGPA.log and a file "C:\Users\xxx\AppData\Local\Palo Alto Networks\GlobalProtect\ServerCert.pan" is written.
But with customer2 we see this "Server cert query failed with error 12019" in PanGPA.log

[BSwientoniowski]

Great info to know, thanks for sharing!   Can you share any details on the PAN-OS versions and the SSL/TLS profiles used at each customer?

[LFura]

If you browse to the portal address of your clients portal that is not working with google chrome does the portal site come up properly giving you the login option?  if so if you go to the developer tools  by hitting F12 and then select the Security option in developer tools and view the certificate what is the issue date and expiration date?  Are they more than 365 days apart?  If so get a new updated cert that is only good for one year.  That was my issue that fixed it.  For additional info my PAN OS version 10.2.1 and I had the issue with global protect clients 4.1.6-12 and also with the latest version of the global protect client 6.1.0 after I got a new cert everything started working for both versions of the global protect client and my SSL/TLS Service Profile under Device->Certificate Management is set to Min Version :TLSv1.0 and Max Version: Max

You might be able to figure something out by installing Wireshark on a computer with the Windows Update installed an then remove the update.  Just load wireshark and then try to connect to the VPN portal that is not working and then right click on some of of the traffic to that portals IP address and filter it to that address.  Then screen shot it and then remove the windows update and have it try and connect again and compare.  Attached is a side by side comparison of the wireshark outputs that I did (with my portal IP redacted) that got me thinking about the certs.

you will see that the machine with the update installed tries to connect to the portal with TLSv1.2 so that's good but it is not successful so then it tries to drop to TLSv1 which would definitely not work.  where on the right hand side I had a machine where I had removed the update and you see it tries to connect TLSv1.2 and then it exchanges client keys which tipped me off that something had to be wrong with the certificate. And after changing the cert I was totally fine.

You could also export the debug logs out of your global protect client under settings and troubleshooting and open the zip file it exports and look in the PanGPS.log file at the bottom which would show you the last connection.  I am also attaching a side by side comparison of those logs.  You will see in the left panel where the windows update is installed it does not actually get to the prelogin and instead gives a WINHTTP_CALLBACK_STATUS_FLAG_SECURITY_CHANNEL error.

Hopefully this helps you.

[LSUgEEk]

I reissued my cert to only have a 364-day life span, but no luck.  Any patched machine will show a Connection Failed.  I really wish I had some update so I knew if I was going to have to replace SAML, or anything that will let me patch my laptops.  My user base has been great so far, but that will only last so long.

[LFura]

See my reply to ChrisCon2355 above and check out the screen shots.  What do you see if you install wireshark on one of the patched machines?  Do you see it trying using TLS 1.2 and then dropping down?  Also what about your logs from the global protect client? Do you see a WINHTTP_CALLBACK_STATUS_FLAG_SECURITY_CHANNEL error?  wondering what else might be causing your issue. 

What about your SSL/TLS Service Profile on the firewall under Device->Certificate Management is set to Min Version :TLSv1.0 and Max Version: Max? or do you have a lower max version forcing the clients to use TLS 1.0 or TLS 1.1 that Microsoft is now disabling in that update?

[pontim]

It our case it appears to be SAML 2.0 through Okta, Global Protect and the latest Microsoft update (KB5018410) that breaks things consistently for our windows clients. Mac and Linux have 0 issues. At this point only fix is to rollback, nothing regarding cert expiration seems to help.

[LSUgEEk]

I have my Min TLS at 1.2 and Max is Max.  I do not see "WINHTTP_CALLBACK_STATUS_FLAG_SECURITY_CHANNEL" error.  We are using SAML to Azure for Auth

This is just before failure:

receive pan_msg_ping, 3

receive pan_msg_ping, 3

Successful connection:

receive pan_msg_ping, 1

Portal config digest matched

[mlinsemier]

@pontim Our configuration is almost identical to yours other than we are using SAML 2.0 against Azure Enterprise application in combination with certificates for pre-login/always-on.  Interestingly enough, our issue doesn't seem to be cropping up when connecting externally, but internally when a laptop is connected to the corporate LAN and tries to authenticate against the portal trying to pull the latest configuration. 

[pontim]

Seems to impact internal LAN and outside users in my case. Forcing TLS 2.0, Okta SAML for auth, latest client in GP 5 update train. Only fix this far is to remove the problem KB. All messing with certs and registry settings has done nothing to change situation.

[pontim]

As of a few minutes ago from Microsoft, installed on my machine and it fixed the GP problems. https://www.bleepingcomputer.com/news/microsoft/microsoft-fixes-windows-tls-handshake-failures-in-ou...

[mdua]

https://www.bleepingcomputer.com/news/microsoft/microsoft-fixes-windows-tls-handshake-failures-in-ou...