Troubleshooting random GlobalProtect disconnects...

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements
Please sign in to see details of an important advisory in our Customer Advisories area.

Troubleshooting random GlobalProtect disconnects...

L2 Linker

Hello. i have been experiencing random GlobalProtect disconnects on my home computer.  I'm running Windows 10 [1909] with GlobalProtect 5.0.8 64-bit connecting back to my office's Palo Alto firewall (not 100% sure of the version). A few times a day, GlobalProtect will just disconnect on its own. I am typically running a Remote Desktop Connection from my home PC to my work PC when this happens and the RD session will freeze and disconnect as well.  I started collecting logs when this happens but dont really know how to parse them.  I have noticed a few things in various logs.   

  • In the  pan_gp_event log, i see an Info message "Tunnel is down due to network change."
  • In the PanGPA log, i see the Info message "Window session changed with state 7", followed by several Debug log entries.
    • Debug(2825): user session locked.
    • Debug( 669): balltip, m_hipMsg=00000171893379A0
    • Debug( 672): balltip, getcount return 0
    • Debug( 766): CAC, type is 8004, data=0000006C7E4FF7C0
    • Debug( 810): CAC, DBT_DEVICEREMOVECOMPLETE, device type=00000005, cacUnplugLogout=0
    • Debug( 851): CAC, do nothing for device remove message
    • (NOTE: Those last three repeat a dozen or so times).
    • Debug( 93): Received data from Pan Service
    • Debug( 172): username field is not empty. not override the username.
    • Debug( 193): CPanBaseReceiver::HandleStatus - found discover-ready tag. value = y.
    • Then i get a message that says "Gateway <portal address>: Checking network availability and restoring VPN connection when network is available.
    • Then some messages about trying to restore the connection
  • In the PanGPS log i see a Debug message "Received session change, event type 7, session 1" followed by and Info message "lock off  session 1"

How would you recommend troubleshooting these disconnects.  I'm not familiar enough with the firewall side of things to know how to look at any logging on that side.  Is the issue with my home computer/OS/hardware/network?  or is it likely something in GlobalProtect and/or Palo Alto that needs attention?

 

Thank You!

21 REPLIES 21

Like Devavrotoka, our problems only started happening after we put in our Palo firewall (not Prisma, but an HA Pair of 3220's) a few weeks back.  Never had a single issue with RDP freezing from 10+ years on a SonicWall.  It's definitely an issue with either GP or PanOS or perhaps even both. 

L0 Member

Hi

 

Have the same issue, what I can see is that it´s when you start a new RDP session that make the GP stops working. But it´s not all sessions that get dropped out only a few and as time pass more and more sessions drops out.

 

/Karl

We are seeing at least a few people have more frequent disconnects with GP 5.0.9-15.  We haven't been able to find the issue yet.  The 4.1.X branch that we were previously on kept a more stable connection.

I am trying a possible solution from another user's post after being unsuccessful resolving this for a while

https://live.paloaltonetworks.com/t5/general-topics/rdp-freeze-fix-globalprotect/m-p/335816/highligh...

 

They said the following:

 

We struggled with the RDP freezing issue with GlobalProtect for a long time.  The initial "fix" was to disable UDP for RDP in the registry.  This fixed the issue for many users but also slowed down the RDP performance.  We thought the issue was with GlobalProtect but after troubleshooting with Palo Alto we were able to see that at some point the remote PC just simply stopped sending RDP packets.

We opened a case with Microsoft and they gave us a registry fix to try that fixed all our RDP freeze issues and allowed us to re-enable UDP for RDP.

 

Important: This regedit goes on the machine you are remoting into, not the machine you are remoting from.

 

HKLM\SOFTWARE\Microsoft\Terminal Server Client

UseURCP (Create this new DWORD with value of 0)

 

You can use this from a command prompt as long as you have admin privileges on the box:

REG ADD "HKLM\SOFTWARE\Microsoft\Terminal Server Client" /v UseURCP /t REG_DWORD /d 0 /f

 

If you previously disabled UDP for RDP you can re-enable it.

Thanks Fhewiufhwefhwe,

 

Your reg key seems to fix the issue. While my RDP client feels more sluggish I'm not experiencing the Freeze anymore.

 

To All, if you have a rule which only allows [ms-rdp {app-default}], you should consider adding an [any {udp-3389}] rule below it, this initially helped make my problem less frustrating but I would still experience freezes if I did the below:

 

If this helps anyone from Palo Alto Networks support if they come across this issue, I found in pcaps at the firewall that it was dropping RST packets from the remote terminal server (my windows 10 desktop in this case) when under heavy screen redraw load while trying to scroll wheel through either the start menu, a folder with large images and the view set to Extra Large Icons, or the way I could always force it to freeze would be to grab a full screen window such as my browser open to my pan firewall UI and flail it around the screen wildly. I would see a massive amount of TRANSMIT and FIREWALL stage (as expected with RDP Experience options set to everything on), then when it froze about three dozen of TCP,RST packets capped in the DROP stage.

 

In looking at the session through a second VDI connection to our Horizon View I noticed the UDP session showed this:

c2s flow:
source: <rd> [user_vpn-L3]
dst: <rd>
proto: 17
sport: 55450 dport: 3389
state: ACTIVE type: FLOW
src user: <rd>
dst user: <rd>

s2c flow:
source: <rd> [Inside-L3]
dst: <rd>
proto: 17
sport: 3389 dport: 55450
state: ACTIVE type: FLOW
src user: <rd>
dst user: <rd>

DP : 2
index(local): : 1203093
start time : Wed Jul 22 05:38:09 2020
timeout : 36000 sec
time to live : 35992 sec
total byte count(c2s) : 330115
total byte count(s2c) : 52126669
layer7 packet count(c2s) : 1926
layer7 packet count(s2c) : 42793
vsys : vsys1
application : ms-rdp
rule : davery-wfh-limited
service timeout override(index) : False
session to be logged at end : True
session in session ager : True
session updated by HA peer : False
layer7 processing : completed
URL filtering enabled : True
URL category : any
session via syn-cookies : False
session terminated on host : False
session traverses tunnel : True
captive portal session : False
ingress interface : tunnel.<rd>
egress interface : ae2
session QoS rule : N/A (class 4)
tracker stage l7proc : ctd err sw

end-reason : unknown

 

Sometimes I would see an unknown-udp session start on the same rule, as I suspect the client side was trying a new session start but most times I did not see that though.

 

Normally I would expect my second rule which follows this one 'davery-wfh-limited-1' which is not app-specific on port udp-3389 to pickup the traffic that was unknown-rdp and allow but it seems like whatever is happening in the packets that the appID engine is hitting a problem identifying and then the stream breaks. Since it's udp the client would just keep sending AND since the firewall was dropping the server side TCP RST packets, the client side was unaware the server side wants the session to disconnect and start over.

I've been dealing with this problem since probably October 2019 and my users have been very frustrated to say the least. It took me about 2 months to find that I needed the non-app specific udp-3389 rule, and most clients we ended up turning off UDP. Thanks again Fhewiufhwefhwe, and thanks to the solution OP Brianhill88


--Why so many drops! Firewall stop telling me I made the rule wrong and tell me how to fix it 😛

Same problem here, would love to see a solution  GP client 5.2.5

L0 Member

Has there been any real resolution to solve this issue? I still have users that cant stay connected with GP but our old VPN works fine. I have tried using SSL and IPSEC versions of Global Protect and they get the same result. The Palo does a keep alive that is failing and dropping their connection. I asked if that timeout can be extended and they gave me no answer.

  • 65330 Views
  • 21 replies
  • 3 Likes
Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!