Issue with user detection on terminal server

[WESupportIT]

Hello everyone,

We're having an issue with our terminal server agent software on multiple terminal servers running Windows Server 2019.

The problem lies in recognizing users who are logged on to these servers. The TS Agent sees the users on the Terminal Server Agent Monitor tab, but they do not appear in the traffic log in the Firewall Source User column.

Although in the past this scenario always worked for all applications that the logged-in user was using on the server (Microsoft 365, Adobe Acrobat) now users only appear in the traffic log when using the web browser on the server.

We have not changed our firewall configuration or the configuration of the terminal user agents on the terminal servers.

The only thing that has changed is the LACP connection to the trust zone between the firewall and a switch.

User detection via the User-ID agent on our domain controller is still working properly. All users on all clients are recognized and show up in the traffic log. The exception entries for the terminal server in the User-ID Agent are still there.

Anyone know where the problem could be?

Why did this always work before and now only via the web browser?

PA 450

PAN OS 10.2.4

[reaper]

has the server been upgraded (which could cause a change in behavior), and has the TS-agent been upgraded to the latest version?

[WESupportIT]

The server has not been upgraded since the issue. The TS-Agent is already running the newest version (11.0.102).

However after some more tests, we can narrow down the problem further. It seems as if logging into Office products is the only situation in which users are not recognized by the firewall. While the registration/licensing window opens, the Ts agent does not seem to be able to assign a user to the packages. After signing in, the user is recognized again while navigating through templates or add-ins in the Office software.

[LaForce]

We deployed some 2019 TS servers and are getting the same issue here. Were you able to find a fix yet?

Our 2016 TS servers work fine though. We even upgraded the PAN OS to 10.2.6. Same results.

[WESupportIT]

Unfortunately we don't. We contacted Palo Alto technical support through our distribution partner. We tested the solution suggestions from the Palo Alto knowledge base:

Terminal Server Agent (TSA) Advanced Configuration using Window... - Knowledge Base - Palo Alto Netw...

Also without success. The technical support said they need to investigate the problem further. Our AV software was also considered as a potential cause, as it rewrites the source ports. We've used the same AV software before (also on Server 2019 systems) and it ran without any problems. We were asked by PA Support to enter some commands in the CLI and send the returned information to them. We have not yet received an answer to this.

[SaurabhB]

We have also upgraded the PA terminal Server Agent software to version - 10.2.3 and started facing the same challenges.

[WESupportIT]

Sorry to hear that. Unfortunately, Palo Alto support couldn't help us. They suggested we continue working with the exception policy we built for our terminal servers that are trying to connect to O365. We still do that to this day.

[Darren_Ramsey]

This is a known issue with TSAgent and not PanOS since at least 2021.  Support in general doesn't know about it but development has known since at least 2021 and below is messaging we received in 2021.

--------------------------------

I've spoken with Engineering and reviewed Jira and FRs.

These O365 processes are using the 'backgroundtaskhost" which was introduced to Windows Server in Windows Server 2019. Processes that use backgroundtaskhost bypass the Windows Kernel TDI Network Framework. The Terminal Server Agent (TSA) intercepts requests from within the TDI Framework and since these processes bypass it, the TSA will not see them.

As it stands today, we cannot see these processes from the TSA. I've noted the limitation, and we are monitoring to see if more processes move away from the TDI Framework and/or more customers are adopting these applications.

There's no timeline on when the TSA will be able to detect these processes. Please let me know if I can be of further assistance.

Respectfully,

Warren

--------------------------------

We have been pushing Palo for 4 years to fix this bug with TSAgent on Windows 2019 and later with Office 365 based authentication. 

In the meantime, my suggestion would be to work with your account team and have them escalate this to the TSAgent product manager and developers and reference the above bug description.

Thanks!

[WESupportIT]

We will contact the dev team of Palo Alto.

The more people report the issue there, the sooner Palo Alto might take action 🙂

[LaForce]

We have opened a ticket couple of times with support and we never got it to work properly. Still an issue with 2019 and 2022 server.

[Darren_Ramsey]

Here is another conversation with product development in early 2024.  Have your account teams escalate to the Director of Product Engineering.

Thanks!

---------------------------------------------------

Good morning Darren,

I appreciate your patience and I remember the challenge that you are facing. This is specifically in regards to the Terminal Server agent and changes that Microsoft made in the way that specific applications (eg. Microsoft Office) operates on multi-user systems on Windows server 2019 and beyond.

I took the notes and information from our previous conversations, and conversations with your team, to my engineering team and we performed a deep dive on how the Terminal Server agent currently operates, reviewing back to the first release's product requirement document and functional specification to pinpoint the cause of your challenge. Based on this analysis we believe that Microsoft, having moved these applications to their TDI framework, causes the agent to not intercept the traffic to apply IP-Port user mappings for this application; which are required to apply user based policy on the firewall for users on multi-user systems. If this is correct, which we currently believe it to be, this is also why you are not seeing this challenge on Windows Server 2016 which does not use the TDI framework for these applications. To be entirely confident in this analysis we will need to perform some additional testing.

Best regards,

Jxxx Mcxxxxxxxx

Sr Product Manager - Identity & Cloud Identity Engine

---------------------------------------------------