cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements
Please sign in to see details of an important advisory in our Customer Advisories area.

Who rated this post

Knowledge sharing: Globalprotect troubleshooting/investgation. Split tunnel,Globalprotect app/agent configuration options and etc. to solve issues

L3 Networker

Hello to All,

 

 

Just as a note I have issues with my old community account, to this is why I am using new one for now (Edit: the issue is fixed but I will keep this article under this profile).

 

 

I had to use several options in the split tunel options to solve VPN issues and I decided to share it.

 

 

 

1.The first issue we had is that some applications may not work when doing split tunnel for users with a particular operational system, so we used the options to do split tunnel configurtion based on operational system (for this you don't have to enable HIP data collection under the portal). Split tunnel is great if the CPU on the firewalls is jumping and as of the Covid many of us work from home the firewalls need this:

 

 

https://docs.paloaltonetworks.com/globalprotect/8-1/globalprotect-admin/globalprotect-gateways/confi...

 

Also when doing a change on the globalprotect agent config on the portal you can enable the new config just for test AD users and groups just in case:

 

https://docs.paloaltonetworks.com/pan-os/8-1/pan-os-web-interface-help/globalprotect/network-globalp...

 

Edit:

 

 

 

&&&&&&&&&&&&

 

 

 

Also exluding Teams and other micrsoft etc, could really help:

 

 

https://live.paloaltonetworks.com/t5/globalprotect-articles/globalprotect-optimizing-office-365-traf...

 

 

Also to exlude it for the enforce globalprotect connection for network access if you use such option as if the there is an issue with the VPN the IT team may use Teams to help the user, so better exclude it:

 

 

https://docs.paloaltonetworks.com/globalprotect/5-1/globalprotect-app-new-features/new-features-rele...

 

 

&&&&&&&&&

 

 

 

 

2. Another issue was with the zoom as it has too many ip addresses, so doing an optimized split tunnel based on domain and app is really nice feature that not many other vendors have and you may even split dns resolution for the excluded domains:

 

 

https://docs.paloaltonetworks.com/pan-os/8-1/pan-os-new-features/globalprotect-features/split-tunnel...

 

https://docs.paloaltonetworks.com/globalprotect/5-2/globalprotect-app-new-features/new-features-rele...

 

https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA14u0000008UkECAU&lang=en_US%E2%80%A...

 

 

 

We used this to try solving the issue with the Invalid Address Errors as mentioned in https://live.paloaltonetworks.com/t5/general-topics/globalprotect-split-tunnel-some-clients-get-inva... . I think the issue is that for some split tunnel configurations only on address and domain some of the traffic for the excluded addresses is send over tunnel. Other vendors like F5 Big-IP suggest blocking the traffic on a firewall, so that any traffic that shouldn't be in the vpn tunnel does not pass the firewall and this sometimes solves the issue but I suggest first doing split tunnel from the article for zoom and split DNS and this is final option to also block the traffic from the VPN zone to the outside.

 

 

 

 

3. I also had issue Wi-Fi confirmation captive portals in the hotels with the free wi-fi but the community helped me with this with the options. Also "Enforce GlobalProtect for Network Access" was enabled and this causes the Palo Alto to work differently with the captive portals :

 

https://live.paloaltonetworks.com/t5/general-topics/palo-alto-globalprotect-app-to-gateway-communica...

 

 

  • Captive Portal Exception Timeout
  • Automatically lauch Webpage in default Browser upon captive portal detection
  • Display captive portal detection message
  • Captive portal detection message

 

I suggest reading:

 

https://docs.paloaltonetworks.com/globalprotect/9-1/globalprotect-admin/globalprotect-quick-configs/...

 

 

 

Edit!:

 

%%%%%%%%%%%%%%%%%%

 

 

 

Many times when there is captive portal with ssl self-signed certificate if it is not confirmed the Globalprotect Agent will also show an error that the gateway/portal certificate is wrong/self signed. Also if "Enforce GlobalProtect for Network Access" is used you may need to exclude the domains (msftncsi.com) used by the operational system to detect the Wi-Fi portal. Please read the article below and don't forget that many web browsers block self signed ssl certificates and it is a real mess with captive portals that have such as captive portals without ssl certs are better handled https://docs.microsoft.com/en-us/windows-hardware/drivers/mobilebroadband/captive-portals

 

For Apple Mac it should be http://captive.apple.com/hotspot-detect.html :

 

https://discussions.apple.com/thread/7491051

 

For allowing Mozilla to not block self signed SSL cets stop the OCSP strapping and  for in the newer versions of Chrome and Edge there seems to be a bug when the captive portal is with self signed cert and I have not found a way to allow this:

 

https://blog.mozilla.org/security/2013/07/29/ocsp-stapling-in-firefox/

 

 

Don't forget just in case to check globalprotect agent the PanGPS and PanGPA logs as they will show if captive portal detected and to as the customer to refresh the connection or reboot the pc as the max captive portal "Captive Portal Exception Timeout" is 1 hour.

 

 

 

Also for issues with Edge and Chrome and the Palo Alto Captive portal see the link below as for me this issue is seen with Cisco Wi-Fi Access Point Captive portals etc. but for each vendor the resolution will be different and if you don't own the Wi-Fi (like coffee shop quest wi-fi) then good luck with solving the issues as only using IE or Mozilla is the solution!

 

https://live.paloaltonetworks.com/t5/general-topics/captive-portal-not-working-in-chrome-and-edge-af...

 

 

 

%%%%%%%%%%%%%%

 

 

 

 

4. For Proxy issues with the VPN you can check:

 

 

https://docs.paloaltonetworks.com/globalprotect/4-1/globalprotect-app-new-features/new-features-rele...

 

 

 

 

 

 

 

5. Another issue was PanGPS error 'network type is unknown network' and we are still testing using fake FQDN that will fail and make the agent t realize that the network is external as we have only external gateways and this seems like a bug to me and we also upgraded to latest 9.1.x version and also the globalprotect agent itself, just in case. Always check the agent PangGPS and PanGPA logs for info.

 

 

https://live.paloaltonetworks.com/t5/general-topics/palo-alto-globalptotect-intermitant-pangps-error...

 

For more about the palo alto logs (PanGPS and PanGPA are the most important but if you use HIP reports and OPSWAT checks there is also log for that):

 

https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000ClaLCAS

 

 

 

 

 

 

6. In 9.1.x it is great that the Globalprotect log is in seperate tab in the GUI and you can also see latency reports from it, so this helps with investigating bad network connections and that the issue is not with the VPN, also if the hourly HIP reports failed for some reason, it will be in those logs. For HIP issues other than failed HIP reports like failed checks there is a log from  a long time in GUI called "HIP Match Logs":

 

https://docs.paloaltonetworks.com/globalprotect/5-1/globalprotect-app-new-features/new-features-rele...

 

https://docs.paloaltonetworks.com/pan-os/9-1/pan-os-admin/monitoring/view-and-manage-logs/log-types-...

 

Also for HIP checks failing to be send every hour check:

 

https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000PLXiCAO

 

 

 

 

7. Don't forget also for the great community article about Globalprotect MTU:

 

https://live.paloaltonetworks.com/t5/globalprotect-articles/troubleshooting-globalprotect-mtu-issues...

 

https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000PPbZCAW

 

 

An issue that we had that is not related to globalprotect but I will menton it is when a client was using site to site GRE tunnel and the GRE tunnel does not automatically adjust the MTU size with MTU discovery as the IPSEC tunnel, so the traffic was getting dropped by the firewall as also the do not fragment bit was configured, so by checking global counters we saw the issue. I recommend for such issues checking:

 

https://live.paloaltonetworks.com/t5/general-topics/knowledge-sharing-palo-alto-checking-for-drops-r...

 

 

 

 

 

8. Also if you need to test some Globalrotect changes and the option to have sepertate configs based on AD user/group or operational system then don't forget that you can add another public ip address on the same interface on the firewall where the portal or gateway is located and just make second test portal or gateway. For example testing something like different port for the portal/gatway communication https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000ClKPCA0 . Better first do such tests on a second test portal/gateway then on production.

 

 

 

 

9. A final thing is to always check that the traffic is that if ipsec is enabled and failovering to SSL as SSL has bad performance when compared to ipsec. On other vendors like F5 Big-IP the workaround is to use DTLS for VPN but I found out that it can't do automatic switchover between TLS and DTLS like palo alto does for SSL and IPsec:

 

 

https://docs.paloaltonetworks.com/pan-os/8-1/pan-os-web-interface-help/globalprotect/network-globalp...

 

 

https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000PPY1CAO

 

https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000ClfoCAC

 

 

 

 

 

10. As point 1 I mentioned CPU being high because of the too much VPN traffic and Palo Alto has some nice recomendations for this for extra optimizations:

 

https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000PP3ICAW

 

 

 

 

11. Anothe note is if your windows admins want for example to run scripts at computer bootup even before the VPN is started and you have "Enforce GlobalProtect for Network Access" then you my try the ''pre-logon VPN'' or ''connect before logon'' options.

 

https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000ClEYCA0

 

https://docs.paloaltonetworks.com/globalprotect/9-1/globalprotect-admin/globalprotect-quick-configs/...

 

 

https://docs.paloaltonetworks.com/globalprotect/5-2/globalprotect-app-new-features/new-features-rele...

 

https://docs.paloaltonetworks.com/globalprotect/5-2/globalprotect-app-user-guide/globalprotect-app-f...

 

 

 

 

12. Always read the "Features Introduced" for the globalprotect app at helps very much and also the "Customize the GlobalProtect App" should be read by anyone working with Palo Alto VPN from time to time as for example the option "User Switch Tunnel Rename Timeout" can solve RDP issues with the VPN (there is an issue with the ip to user-id mapping even outside the Globalprotect VPN when using RDP and you can read about it here https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000CleBCAS ). The option Windows SSO is also great even if you use RSA tolken MFA on the gateways as  atleast the signing will be automatic after 24 hours when you again to sign the portal with your AD credentials as with 'pre-logon'' or "start before logon" or using different authentications on the portals and gateways the Cookie Authentication that allows when you log to the portal to not log to the gateway does not work.

 

 

https://docs.paloaltonetworks.com/globalprotect/5-2/globalprotect-app-release-notes/gp-app-release-i...

 

https://docs.paloaltonetworks.com/globalprotect/8-1/globalprotect-admin/globalprotect-portals/define...

 

 

https://docs.paloaltonetworks.com/globalprotect/8-1/globalprotect-admin/authentication/about-globalp...

 

 

 

 

13. Another usefull option is to "Install in Local Root Certificate Store" expecially if you replace globalprotect portal/gateway certficate with another SSL vendor provider certificate:

 

 

https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000ClFbCAK

 

https://docs.paloaltonetworks.com/pan-os/8-1/pan-os-web-interface-help/globalprotect/network-globalp...

 

 

 

14. For Clienless VPN there are great Palo Alto articles hos to discover issues:

 

https://docs.paloaltonetworks.com/globalprotect/8-1/globalprotect-admin/globalprotect-clientless-vpn...

 

https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000PPizCAG

 

 

 

 

 

Edit:

 

 

 

%%%%%%%%%%%%%%%%

 

 

15. If you want to stop the palo alto globalprotect agent with a script even when it is password protected and you are not an admin of the computer then you can use powershell script that needs to be run with elevated admin priviliges:

 

https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000PNPRCA4

 

 

The powershel lcommand is (you can change it a little as "automatic" means that the PanGPS will start after reboot). After stoping the PanGPS then the PanGPA will be stopped as if you first stop the PanGPA then the working PanGPS will start it again in some cases. The normal kill command "Stop-Process" don't work, so use the ones below https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.management/stop-process?view....

 

 

Set-Service-Name "PanGPS" -Status stopped -StartupType automatic

 

Set-Service-Name "PanGPA" -Status stopped -StartupType automatic

 

 

 

 

%%%%%%%%%%%%%

 

 

 

 

 

 

I also suggest checking the offcial palo alto article for "Troubleshooting GlobalProtect".

 

 

https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000ClkBCAS

 

 

Who rated this post