06-27-2021 02:34 AM - edited 10-17-2021 03:39 AM
Hello to All,
Just as a note I have issues with my old community account, to this is why I am using new one for now (Edit: the issue is fixed but I will keep this article under this profile).
I had to use several options in the split tunel options to solve VPN issues and I decided to share it.
1.The first issue we had is that some applications may not work when doing split tunnel for users with a particular operational system, so we used the options to do split tunnel configurtion based on operational system (for this you don't have to enable HIP data collection under the portal). Split tunnel is great if the CPU on the firewalls is jumping and as of the Covid many of us work from home the firewalls need this:
Also when doing a change on the globalprotect agent config on the portal you can enable the new config just for test AD users and groups just in case:
Edit:
&&&&&&&&&&&&
Also exluding Teams and other micrsoft etc, could really help:
Also to exlude it for the enforce globalprotect connection for network access if you use such option as if the there is an issue with the VPN the IT team may use Teams to help the user, so better exclude it:
&&&&&&&&&
2. Another issue was with the zoom as it has too many ip addresses, so doing an optimized split tunnel based on domain and app is really nice feature that not many other vendors have and you may even split dns resolution for the excluded domains:
We used this to try solving the issue with the Invalid Address Errors as mentioned in https://live.paloaltonetworks.com/t5/general-topics/globalprotect-split-tunnel-some-clients-get-inva... . I think the issue is that for some split tunnel configurations only on address and domain some of the traffic for the excluded addresses is send over tunnel. Other vendors like F5 Big-IP suggest blocking the traffic on a firewall, so that any traffic that shouldn't be in the vpn tunnel does not pass the firewall and this sometimes solves the issue but I suggest first doing split tunnel from the article for zoom and split DNS and this is final option to also block the traffic from the VPN zone to the outside.
3. I also had issue Wi-Fi confirmation captive portals in the hotels with the free wi-fi but the community helped me with this with the options. Also "Enforce GlobalProtect for Network Access" was enabled and this causes the Palo Alto to work differently with the captive portals :
I suggest reading:
Edit!:
%%%%%%%%%%%%%%%%%%
Many times when there is captive portal with ssl self-signed certificate if it is not confirmed the Globalprotect Agent will also show an error that the gateway/portal certificate is wrong/self signed. Also if "Enforce GlobalProtect for Network Access" is used you may need to exclude the domains (msftncsi.com) used by the operational system to detect the Wi-Fi portal. Please read the article below and don't forget that many web browsers block self signed ssl certificates and it is a real mess with captive portals that have such as captive portals without ssl certs are better handled https://docs.microsoft.com/en-us/windows-hardware/drivers/mobilebroadband/captive-portals
For Apple Mac it should be http://captive.apple.com/hotspot-detect.html :
https://discussions.apple.com/thread/7491051
For allowing Mozilla to not block self signed SSL cets stop the OCSP strapping and for in the newer versions of Chrome and Edge there seems to be a bug when the captive portal is with self signed cert and I have not found a way to allow this:
https://blog.mozilla.org/security/2013/07/29/ocsp-stapling-in-firefox/
Don't forget just in case to check globalprotect agent the PanGPS and PanGPA logs as they will show if captive portal detected and to as the customer to refresh the connection or reboot the pc as the max captive portal "Captive Portal Exception Timeout" is 1 hour.
Also for issues with Edge and Chrome and the Palo Alto Captive portal see the link below as for me this issue is seen with Cisco Wi-Fi Access Point Captive portals etc. but for each vendor the resolution will be different and if you don't own the Wi-Fi (like coffee shop quest wi-fi) then good luck with solving the issues as only using IE or Mozilla is the solution!
%%%%%%%%%%%%%%
4. For Proxy issues with the VPN you can check:
5. Another issue was PanGPS error 'network type is unknown network' and we are still testing using fake FQDN that will fail and make the agent t realize that the network is external as we have only external gateways and this seems like a bug to me and we also upgraded to latest 9.1.x version and also the globalprotect agent itself, just in case. Always check the agent PangGPS and PanGPA logs for info.
For more about the palo alto logs (PanGPS and PanGPA are the most important but if you use HIP reports and OPSWAT checks there is also log for that):
https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000ClaLCAS
6. In 9.1.x it is great that the Globalprotect log is in seperate tab in the GUI and you can also see latency reports from it, so this helps with investigating bad network connections and that the issue is not with the VPN, also if the hourly HIP reports failed for some reason, it will be in those logs. For HIP issues other than failed HIP reports like failed checks there is a log from a long time in GUI called "HIP Match Logs":
Also for HIP checks failing to be send every hour check:
https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000PLXiCAO
7. Don't forget also for the great community article about Globalprotect MTU:
https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000PPbZCAW
An issue that we had that is not related to globalprotect but I will menton it is when a client was using site to site GRE tunnel and the GRE tunnel does not automatically adjust the MTU size with MTU discovery as the IPSEC tunnel, so the traffic was getting dropped by the firewall as also the do not fragment bit was configured, so by checking global counters we saw the issue. I recommend for such issues checking:
8. Also if you need to test some Globalrotect changes and the option to have sepertate configs based on AD user/group or operational system then don't forget that you can add another public ip address on the same interface on the firewall where the portal or gateway is located and just make second test portal or gateway. For example testing something like different port for the portal/gatway communication https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000ClKPCA0 . Better first do such tests on a second test portal/gateway then on production.
9. A final thing is to always check that the traffic is that if ipsec is enabled and failovering to SSL as SSL has bad performance when compared to ipsec. On other vendors like F5 Big-IP the workaround is to use DTLS for VPN but I found out that it can't do automatic switchover between TLS and DTLS like palo alto does for SSL and IPsec:
https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000PPY1CAO
https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000ClfoCAC
10. As point 1 I mentioned CPU being high because of the too much VPN traffic and Palo Alto has some nice recomendations for this for extra optimizations:
https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000PP3ICAW
11. Anothe note is if your windows admins want for example to run scripts at computer bootup even before the VPN is started and you have "Enforce GlobalProtect for Network Access" then you my try the ''pre-logon VPN'' or ''connect before logon'' options.
https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000ClEYCA0
12. Always read the "Features Introduced" for the globalprotect app at helps very much and also the "Customize the GlobalProtect App" should be read by anyone working with Palo Alto VPN from time to time as for example the option "User Switch Tunnel Rename Timeout" can solve RDP issues with the VPN (there is an issue with the ip to user-id mapping even outside the Globalprotect VPN when using RDP and you can read about it here https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000CleBCAS ). The option Windows SSO is also great even if you use RSA tolken MFA on the gateways as atleast the signing will be automatic after 24 hours when you again to sign the portal with your AD credentials as with 'pre-logon'' or "start before logon" or using different authentications on the portals and gateways the Cookie Authentication that allows when you log to the portal to not log to the gateway does not work.
13. Another usefull option is to "Install in Local Root Certificate Store" expecially if you replace globalprotect portal/gateway certficate with another SSL vendor provider certificate:
https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000ClFbCAK
14. For Clienless VPN there are great Palo Alto articles hos to discover issues:
https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000PPizCAG
Edit:
%%%%%%%%%%%%%%%%
15. If you want to stop the palo alto globalprotect agent with a script even when it is password protected and you are not an admin of the computer then you can use powershell script that needs to be run with elevated admin priviliges:
https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000PNPRCA4
The powershel lcommand is (you can change it a little as "automatic" means that the PanGPS will start after reboot). After stoping the PanGPS then the PanGPA will be stopped as if you first stop the PanGPA then the working PanGPS will start it again in some cases. The normal kill command "Stop-Process" don't work, so use the ones below https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.management/stop-process?view....
Set-Service-Name "PanGPS" -Status stopped -StartupType automatic
Set-Service-Name "PanGPA" -Status stopped -StartupType automatic
%%%%%%%%%%%%%
I also suggest checking the offcial palo alto article for "Troubleshooting GlobalProtect".
https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000ClkBCAS
06-27-2021 12:16 PM
Wow..... to think i pay **bleep** loads of money to palo tech support (who usually post here for solutions) ....and you have all the answers.. you have been busy and thanks for sharing...
06-27-2021 12:52 PM - edited 06-27-2021 03:03 PM
Palo Alto is great but they lack some available documentation for example for which log file should be checked for which issue etc. When you have some work with the palo alto TAC, you start to see somethings that are not officially documented. I also use my articles for something like One Note as I will be using them myself as things can be forgotten over time, but better share if I am going to write it anyway.
Also from what I see most people like to open a case and wait 2 or 3 weeks for the issue to be resolved than reading for 2 or 3 hours or checking if there is a solution in the community, because this way they don't take responsibility and can get by without reading too much and because of this just write an email or call each day with "....ASAP ......URGENT.....", so the TAC will still have more cases than they can handle but this article is for the small amount of people that want to solve their issues if possible themselves. Yes, some cases need TAC support but many don't and if you are not paying for premium support, you can't expect a solution from the TAC in 1 hour. This is at least how I see it.
Edit:
Still the Palo Alto TAC needs some improvment as I have found more answers on my own in the community articles or even found out that some things are not like day say at all. The Example is:
01-25-2022 01:02 PM
Now Again I am using my old account. In addition to everything I mentioned the "GlobalProtect App Log Collection for Troubleshooting" paid feature that uses the Cortex Data Lake is something really great for slowness and performance issues and for network environments that are unstable and see such issues very often.
GlobalProtect App Log Collection for Troubleshooting (paloaltonetworks.com)
01-26-2022 12:04 AM
Thanks mister Dimitrov, you really contribute with us \o/
Click Accept as Solution to acknowledge that the answer to your question has been provided.
The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!
These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!
The LIVEcommunity thanks you for your participation!
