Global Protect Users Experiencing Telnet Disconnects

[CCullhaj]

I wanted to see if I can get some help with some session termination problems that I am experiencing for Global Protect users. Our remote users connect to an on-prem ERP systems through telnet, tcp/23.  I recognize that this protocol has inherited performance and security problems, but unfortunately that's what we are given to work with.  The bottom line is that Global Protect users get kicked off from telnet sessions constantly.  Prisma Access is connected through a service connection to our on-prem Sonicwall firewall, where the ERP system lives.

To remediate these telnet issues, I have created an application override policy and attached a custom app-id with max TCP timeouts for tcp/23.  This is getting applied to all Global Protect users.  I have also increased the TCP timeouts in the Sonicwall firewall.  However, users are getting kicked off the session while they are idle.  The telnet server is designed to not kick off users for a long period of time, regardless if they're active or not.

I know that the app-id override with max TCP timeouts is working well because that same app override policy fixed the same session kick off issues that we were experiencing for some other users connecting from azure to the same on-prem ERP server.

I have collected some packet capture from the sonicwall firewall and from my computer, and every time I inspect the packets, I see that my Global Protect client is sending TCP Reset requests to the server.  I also show in Cortex logs that the reason for the disconnect was client side.  I am not sure as to why Global Protect would reset the telnet session all of a sudden.  

I wanted to check if anyone else in community has experienced the same issue.  Any thoughts would be appreciated!

[kiwi]

Hi @CCullhaj ,

Have you checked the GP client logs (on debug level) at the time of disconnects ?

Just thinking out loud here ... some timed event (for example HIP check occurs every hour by default) or some other event that correlates with the disconnects that could help us further with the investigation ?

Regards,

-Kiwi.

[CCullhaj]

Sorry for the late reply @kiwi ... the bottom line of this problem is that the sonicwall firewall needed constant attention because the policy rule that increased TCP timeouts, somehow every 2 weeks or so would become sort of inactive.  Every time I increased the TCP time out on that sonicwall rule, users would stop complaining for 2 weeks and then the same story would happen over and over again.

We replaced the sonicwall with a PA firewall and I followed this guide to increase the timeout for telnet TCP/23: https://docs.paloaltonetworks.com/pan-os/8-1/pan-os-admin/app-id/service-based-session-timeouts.html

It has been working like a charm up to now.

[nikoolayy1]

Have you checked if it is not just the global protect app timeout as the telnet is in the vpn tunnel but the VPN tunnel also has timeout?

https://docs.paloaltonetworks.com/pan-os/9-1/pan-os-web-interface-help/globalprotect/network-globalp...

Also I had issues because of a bad network as the global protect app checks by icmp if the gateway is reachable so check the PanGps.log logs on the client device as it was already mentioned:

https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000PPgZCAW

As you mentioned the session drop reason is tcp client rst but I always check the global counters for a source and destination just in case to confirm that it is not the firewall:

https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000PPgZCAW