GlobalProtect stops to connect

[Udineverisch]

Hi all,
GlobalProtect stopped to connect to server. 
So it works before ( I did not install any new software, firewals, proxies, .... etc) It contiue work under VirtualBox machine, so it is not a problem of my internet provider, but it stops to connect from my machine:
I can reach portal throgh browser (chrome), or curl on 443 port without problems.
I tried without success reboot, reinstall GlobalProtect, etc..
So any help very appreciated I don't want to reinstall whole system without understanding what problem is.

My logs (just XXXXed real portal url): 

(T11260) 07/30/17 12:25:58:494 Debug(6677): Saved password is empty.
(T8556) 07/30/17 12:25:58:494 Info ( 734): HipMonitorThread starts
(T11260) 07/30/17 12:25:58:494 Debug(1675): Pre-logon-then-on-demand value is no
(T11260) 07/30/17 12:25:58:494 Debug(1336): SSO starts.
(T11260) 07/30/17 12:25:58:494 Debug(1365): SSO ----- PanCredGet failed with error=0x2
(T11260) 07/30/17 12:25:58:494 Debug(6693): SSO password is empty
(T11260) 07/30/17 12:25:58:494 Debug(1760): empty domain name.
(T11260) 07/30/17 12:25:58:495 Debug(4326): Set state to Retrieving configuration...
(T8556) 07/30/17 12:25:58:495 Debug( 387): Wscapi.dll is loaded.
(T8556) 07/30/17 12:25:58:495 Debug( 401): Register -- WscRegisterForChanges
(T11260) 07/30/17 12:25:58:495 Debug(1289): unknown network type.
(T11260) 07/30/17 12:25:58:495 Debug(4614): ServerThread: ProcessServerPortal -- GetConfigFromPortal
(T7100) 07/30/17 12:25:58:495 Debug( 351): Active session id is 1
(T8556) 07/30/17 12:25:58:495 Info ( 403): HipMonitorThread wait for exit event.
(T8556) 07/30/17 12:25:58:495 Debug( 405): before WaitForMultipleObjects
(T11260) 07/30/17 12:25:58:495 Debug(5021): entering.
(T11260) 07/30/17 12:25:58:495 Debug(5057): SSO enable status is 0, user name is slogvine, domain name is .
(T11260) 07/30/17 12:25:58:495 Debug(5060): reset user authentication status to true.
(T11260) 07/30/17 12:25:58:495 Debug(1795): open http session.
(T11260) 07/30/17 12:25:58:496 Debug( 370): set WINHTTP_OPTION_SECURE_PROTOCOLS
(T11260) 07/30/17 12:25:58:496 Debug(1398): Auto detect proxy for host XXXXXXXX
(T11260) 07/30/17 12:25:58:496 Debug(1411): CPanMSServiceWin::SetProxyForHost: fAutoDetect: 0 url: proxy: bypass:
url:https://XXXXXXXX/ returned proxystr:
(T11260) 07/30/17 12:25:58:496 Debug(1436): m_proxyInfo.dwAccessType is 0, m_proxyInfo.lpszProxy is (null)
(T11260) 07/30/17 12:25:58:496 Debug(7855): Scep clean
(T11260) 07/30/17 12:25:58:496 Debug(7857): Clean m_pScepCert
(T11260) 07/30/17 12:25:58:496 Debug(3194): Clean m_szScepCertPanName
(T11260) 07/30/17 12:25:58:496 Debug(2997): TriggerCaptivePortalDetection() end
(T8568) 07/30/17 12:25:58:496 Debug(3094): CaptivePortalDetectionThread: delay 2 seconds before captive portal detection. m_bIsDetectingCaptivePortal=1, m_bPreLoginIsDone = 0
(T8568) 07/30/17 12:25:58:496 Debug(3072): CaptivePortalDetectionThread: wait (2000 ms) for captive portal detection event.
(T11260) 07/30/17 12:25:58:497 Debug(4381): Pre-login...,verifyportalcert=yes
(T11260) 07/30/17 12:25:58:497 Debug( 76): pan_get_full_path(): full path in multibyte char is C:\Program Files\Palo Alto Networks\GlobalProtect\tca.cer
(T11260) 07/30/17 12:25:58:497 Info (1259): File C:\Program Files\Palo Alto Networks\GlobalProtect\tca.cer does not exist.
(T11260) 07/30/17 12:25:58:497 Debug(7000): File C:\Program Files\Palo Alto Networks\GlobalProtect\tca.cer does not exist.
(T11260) 07/30/17 12:25:58:497 Debug(7037): CheckServerCert() returns FALSE
(T11260) 07/30/17 12:25:58:497 Debug(2328): portal proxyparam is empty
(T11260) 07/30/17 12:25:58:497 Debug(2350): OID, oid=
(T11260) 07/30/17 12:25:58:497 Debug(2394): IPADDR=XXXXXXXX,PORT=443,URL=/global-protect/prelogin.esp,POST=1,PROXY_AUTO=0,PROXY_CFGURL=NULL,PROXY=NULL,PROXY_BYPASS=NULL,PROXY_USER=NULL,PROXY_PASS=****,VERIFY_CERT=1,ADDITIONAL_CHECK=1,SCEP_CERT=,oid=
(T11260) 07/30/17 12:25:58:497 Debug( 910): Send response to client for request https_request
(T11260) 07/30/17 12:25:58:497 Debug(2424): gpapintimeout not set, set it to 600 seconds
(T7100) 07/30/17 12:25:58:497 Debug( 274): Found PanGPA pid 6596
(T7100) 07/30/17 12:25:58:497 Debug( 278): Found active PanGPA pid is 6596
(T7100) 07/30/17 12:25:58:497 Debug( 55): Session id is 1 for pid 6596
(T7100) 07/30/17 12:25:58:497 Debug( 95): User profile directory is C:\Users\Lenovo-PC
(T7100) 07/30/17 12:25:58:497 Debug( 110): Found session 1
(T7100) 07/30/17 12:25:58:497 Debug( 140): Skip calling NetUserGetInfo for non-roaming profile.
(T7100) 07/30/17 12:25:58:497 Debug( 153): info4_buf is NULL
(T7100) 07/30/17 12:25:58:497 Debug( 155): profileInfo username Lenovo-PC, profile path (null), server (null)
(T7100) 07/30/17 12:25:58:587 Debug( 169): User profile loaded.
(T7100) 07/30/17 12:25:58:587 Debug( 185): Impersonated logged on user.
(T7100) 07/30/17 12:25:58:587 Debug( 187): Profile type is 0
(T7100) 07/30/17 12:25:58:588 Debug( 239): User profile unloaded
(T7100) 07/30/17 12:25:58:588 Debug( 76): pan_get_full_path(): full path in multibyte char is C:\WINDOWS\system32\config\systemprofile\AppData\Local\Palo Alto Networks\GlobalProtect\PanGpMPR.dat
(T11260) 07/30/17 12:25:58:613 Debug(2494): receive pan_msg_ping, 3
(T11260) 07/30/17 12:26:00:300 Debug(2494): receive pan_msg_ping, 3
(T8568) 07/30/17 12:26:00:501 Debug( 56): pan_captive_portal_detection: remote server address= 0xE12D9AC
(T8568) 07/30/17 12:26:00:501 Debug( 47): WSAGetLastError() returns 10035
(T11260) 07/30/17 12:26:00:522 Debug(2659): HTTP_RPC, len=0, result is
(NULL)...
(T11260) 07/30/17 12:26:00:522 Debug(4418): prelogin to portal result is
(null)
(T11260) 07/30/17 12:26:00:522 Debug(4574): Failed to pre-login to the portal XXXXXXXX. Error 0
(T11260) 07/30/17 12:26:00:522 Debug(1820): close WinHttp close handle.
(T11260) 07/30/17 12:26:00:522 Info (6175): Portal config does not exist, try registry/plist
(T11260) 07/30/17 12:26:00:522 Debug(6185): Failed to get version from config, try local
(T11260) 07/30/17 12:26:00:522 Info (5188): failed to retrieve value of the tag version.
(T11260) 07/30/17 12:26:00:522 Info (5228): Skip reading cached portal config.
(T11260) 07/30/17 12:26:00:522 Debug(7509): No scep profile
(T11260) 07/30/17 12:26:00:522 Debug(5241): portal status is Invalid portal.
(T11260) 07/30/17 12:26:00:522 Debug(5242): returns 0.
(T11260) 07/30/17 12:26:00:522 Debug(4326): Set state to Disconnected
(T11260) 07/30/17 12:26:00:522 Debug(1289): unknown network type.
(T11260) 07/30/17 12:26:00:522 Debug( 910): Send response to client for request portal
(T11260) 07/30/17 12:26:00:522 Debug(7269): Set m_bPreviousSwitchOffMsg to 0
(T8568) 07/30/17 12:26:00:638 Debug( 152): pan_http_captive_portal_detection: status is 204
(T8568) 07/30/17 12:26:00:638 Debug(3014): DetectCaptivePortal: captive portal is not detected for CP server index = 0. iStatus = 204
(T8568) 07/30/17 12:26:00:701 Debug( 56): pan_captive_portal_detection: remote server address= 0xC937FD11
(T8568) 07/30/17 12:26:00:701 Debug( 47): WSAGetLastError() returns 10035
(T8568) 07/30/17 12:26:00:885 Debug( 152): pan_http_captive_portal_detection: status is 200
(T8568) 07/30/17 12:26:00:885 Debug( 185): pan_http_captive_portal_detection(): head start=657, end=685.
(T8568) 07/30/17 12:26:00:885 Debug( 197): pan_http_captive_portal_detection() - captive portal isn't detected against server.
(T8568) 07/30/17 12:26:00:885 Debug(3014): DetectCaptivePortal: captive portal is not detected for CP server index = 1. iStatus = 200
(T8568) 07/30/17 12:26:00:885 Debug(3165): CaptivePortalDetectionThread: Didn't detect captive portal currently, and bCaptivePortalDetectedOnce=(0).
(T8568) 07/30/17 12:26:00:885 Debug(3072): CaptivePortalDetectionThread: wait (-1 ms) for captive portal detection event.

[Remo]

Hi @Alex_Samad

Is your pa3-vpn-gateway somehow private/hidden so that only you (maybe with a local host entry) can connect?

The reason I am asking is because from my point of view, with the knowledge I have from your posts so far, there is a problem on pa3-vpn-gateway. I am able to resolve your gateway 1, 2 and the portal by DNS but not gateway 3. In addition, a TLS check shows the correct information for gateway 1, 2 and the portal and all 3 certs show the earlier mentionned self signes root cert. But on gateway 3 or better on the IP where I assume gateway 3 should be, the TLS check fails completely.

In an earlier post you wrote that other clients are working: what rules do you have in plave to decide which gateway will be chosen by the clients? All with the same priority or based on usergroups ... So rhe working clients, do they also try to connect to gateway 3?

[Alex_Samad]

pa3 is a internal gateway, dns and connectivity is only accessible whilst your on the internal network

same rules.... thats the annoying thing

[Remo]

Of course ... would have been too easy, if that was the solution 😛

At the times of these failing connections: is there something useful or at least regarding that client in the system log of your portal and/or gateway (s)?

[Remo]

So a short recap: @Alex_Samad

• Global protect infrastructure with one portal, 2 external gateways and one internal --> is this a fully distributed setup? Do you use authentication cookies?
• You have 6 absolutely identical clients but only 4 of them are able to connect --> do all of them connect to the same external gateway?
• It was all working until day x where 2 clients started having problems with the connection

In addition to this discussion you probably also want to open a TAC case...

[Alex_Samad]

Hi

I have, buts its been nearly 4 weeks not getting much progress, there seemed to be some life to it here so ..

A

[Remo]

I just wanted to mention this ... with a TAC case you have the possibility to escalate it by your SE ... if the community stops helping, then ...

[Remo]

@Alex_Samad

As there are no specific configurations for specific clients, this sounds more like a client issue. So did you already search for differences on these clients (working/not working)? (Different OS versions, different installed updates, diffetent local software,...)

Or did you compare the global protect client logs from a working withva not working client to find out if there are big differences or to find out more about the actual problem?

[Alex_Samad]

tried to. the feedback from support is that the client doesn't have access to the root ca.

Strange because if I point IE to the portal it works and certifies it.

I might try and run reg mon or file mon on the GP process and see what is happening

[Udineverisch]

Yes I have root cert same as for portal.

I tried to remove all certificates and resttarting. My cert automatically added as root cert.

It is "Go Daddy Class 2 Certificate Authority" SHA1RSA algorithm expired in 34th year 

Tried the same in my virtual (VirtualBox from Oracle) host, and again virtual host works, my host don't work

Any more ideas?

Here pcap screen short for my portal firstly I tried to connect using globalprotect and second time with browser

 

Thanks

[Remo]

@Udineverisch

Without more details about the logs and the configuration of the portal and the gateway it slowly gets difficult to suggest ways to solve the problem.

If the portal/gateway has the same configuration for every client and does not manually block your computer, it has to be an issue on your computer, specially because your virtual box still works.

Is there any difference on these 2? Special software, windows updates, ...?

[Udineverisch]

---

@Remo wrote:

@Udineverisch

Without more details about the logs and the configuration of the portal and the gateway it slowly gets difficult to suggest ways to solve the problem.

If the portal/gateway has the same configuration for every client and does not manually block your computer, it has to be an issue on your computer, specially because your virtual box still works.

Is there any difference on these 2? Special software, windows updates, ...?

---

No any special. Just usual actions.

I don't know what problem was. 
I reinstalled Win10 and it solve problem. 
Anyway thank you for your accistance.

[Alex_Samad]

Thought I would add by findings. so after lots of wireshark captures and talking with support.

we isolated my problem down to me wanting a client cert, signed by a specific CA.

on my laptop that wasn't working, deleting and re installing the cert seemed to make it work.

My cert is being stored in the machine store, not a user store.

and the cert was working before for openvpn and mmc said it was okay.

So I hope that helps somebody. off to test on my other machines

[Ameshlal]

Got any resolution for this issue later?  Me also facing the same issue. Please share the updates. Thanks

[HGilmore_72]

The fix in this video worked for us. Palo Alto Networks GlobalProtect VPN using Microsoft Azure AD & SAML - YouTube