Terminal Server Agent Registry Tuning for Better Port Allocation and Handling, Time Wait State

Issue

When going to certain sites using a web browser, users are unable to browse fully due to the terminal server.

Cause

The root cause is because some websites may trigger high usage of ports for a short duration before releasing them. An ongoing port allocation from a website may cause the premature allocation of a port from the Terminal Server Agent (TSA) before the Windows operating system officially releases it. Thus, connectivity errors are encountered.

Note: The issue can also be experienced with applications that use a high number of ports and release them very quickly.

Windows imposes a Timed Wait State on a port. This is a configurable parameter in Windows. The value range is 30-300 seconds (decimal) with a default of 240 seconds.

(Default)                REG_SZ          (value not set)

TcpTimedWaitDelay        REG_DWORD       0x0000001e (30)

Corresponding registry settings:

• System Key: [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters]
• Value Name: TcpTimedWaitDelay
• Data Type: REG_DWORD (DWORD Value)

Resolution

A setting was introduced in TSA version 5.0.1 to address this issue. This Time Wait State (TWS) setting enables the TS Agent to optionally track the TcpTimedWaitDelay of a Windows system, thus preventing the TS Agent from choosing ports that are still in a Timed Wait State. With this setting enabled, if the system reaches the usual low threshold of port blocks, a new port block is allocated. By default this new behavior is off and can only be modified by editing the registry.

Additionally, because ports are placed in a TWS, there is a possibility that a user may produce a lot of activity that results in ports entering a TWS state and allocating additional port blocks that are no longer needed later. Another user may come into the picture and be starved since another user(s) may be hoarding port blocks that are not needed but are not getting reclaimed back to the system due to the owners' lack of activity (log on/off). As a result we introduced a timer that polls the driver to free port blocks that are no longer needed. This timer only runs when TWS feature is enabled. The default timer value is 240 seconds and can be modified through the registry as well.

Time Wait State (TWS) Feature Behavior and Configuration

There is no UI to enable this feature.  The Windows registry must be edited to disable the feature.

Warning: Exercise caution when performing registry modifications. There is no error checking of values in the registry.

Enable/Disable TWS

• Registry Key: HKEY_LOCAL_MACHINE\SOFTWARE\Palo Alto Networks\TS Agent\Conf\EnableTws
• Default Value: 0    [0-disabled, 1-enabled]
• Restart the Terminal Server Agent services for the change to take effect

Periodic cleanup of Unused Port Blocks:

• Registry Key: HKEY_LOCAL_MACHINE\SOFTWARE\Palo Alto Networks\TS Agent\Conf\FreePortBlockDelay
• Default Value: 240  (seconds), a 0 value will disable this timer
• There is no need to restart the services, as changes for this value will automatically take effect
• FreePortBlockDelay should not be shortened too aggressively as this will cause a lot of driver cleanup activity.  It is recommended that this value match the system's default Timed Wait Delay (TcpTimedWaitDelay) of 240 seconds.

Resolution

If port exhaustion has occured due to large numbers of Time Wait connections, make the following registry changes:

• Key: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\TcpTimedWaitDelay
  • Value: 30
• Key: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\StrictTimeWaitSeqCheck
  • Value: 1
• Key: HKEY_LOCAL_MACHINE\SOFTWARE\Palo Alto Networks\TS Agent\Conf\EnableTws
  • Value: 1
• Key: HKEY_LOCAL_MACHINE\SOFTWARE\Palo Alto Networks\TS Agent\Conf\FreePortBlockDelay
  • Value: 30

You may need to add the second regedit value manually if it's not there. This value will tell Windows Kernel, instead of honor TcpTimedWaitDelay setting, Kernel will free those port strictly in 30 seconds. Otherwise Windows Kernel may delay freeing ports when its busy doing something else.

The first two values will direct Windows Kernel to free Time_Waited ports within 30 seconds (default Kernel value: 240 seconds). If you shorter such value in TS Agent, you will need to shorter the value for Windows Kernel as well to keep the consistency.

Note: The Terminal Server Agent service must be restarted after TS Agent value changes (the third & fourth value). Windows OS needs to be rebooted if Kernel Tcpip stack Parameters (the first & second value) are changed.

owner: sberti