Tracking Corrupt Cortex XDR Agents

[thenetworksfine]

I am looking for any input on how other customers are handling situations where:

1. The agent is installed on a host and says it is checking in, but it does not appear in the Cortex XDR Console

2. The agent is corrupt and has stopped reporting back (due to a failed upgrade or otherwise)

I didn't know if anyone has any unique solutions for these situations. From a corrupt agent standpoint, it would be nice to have a Tenable plugin to report back the current signature versions.

[Alexandre_Jodoin]

I can't say that I saw 1) but 2) sounds too familliar.

I estimate that we have, at any given time, about 3-5% of the agents in that situation.

Basically, I have a script that will re-create/re-enable the runtimes, as I found this is mostly the case with corrupted installation.

The cyserver services won't start as one of the dependencies is borked. So the script tries to fix them all.

This works in 20% of the corrupted agent cases.

For the others it is an issue with the file system filter. trying to reload it using fltmc fails with a "file not found" error and I haven't been able to find a solution for those, so it is xdrcleaner, reboot, xdrcleaner, re-install.

But 3-5% of 18k agents is a lot to manually fix month after month.

[btenberge]

I'm sorry I don't have an answer to your question either, but I think this is a good subject because I'm having the same "revelations" lately.

I have encountered some agents in our environment that appear to be working (they appear online and up-to-date), but half the files are missing in the installation folder, only cyserver.exe is running and upgrade attempts fail all the time.

These specific instances do make it seem like everything is fine and dandy, while it's not.

I'm currently trying to figure out a way to find these half-, or non-operational agents, myself, but I have nothing yet.

I'm sure we're not the only ones dealing with this problem, so I'm hoping that there are some people that have already found a working solution.

[thenetworksfine]

@Alexandre_Jodoin We are working through a wide range of hosts with support to get to the bottom of this situation. One of the commonalities ended up being deleted Installation Packages under Cortex XDR Administrative Console >> Endpoints >> Endpoint Management >> Agent Installations. I'm not saying this is the case for you but we did not know that someone was cleaning up these packages and wouldn't have though it would put the agent dead in the water. The agent is not smart enough to go out and get the latest version from the console if the previously sent version no longer exists.

A helpful command that we used is below. This prevented us from having to reinstall the agent in a number of situations.

1. From an administrative command prompt
   1. C:\Program Files\Palo Alto Networks\Traps\cytool.exe reconnect force <ID>
2. Where <ID> = the agent ID that corresponds to an agent installation package name with the installed version of the agent. You can show the id field within Cortex XDR Administrative Console >> Endpoints >> Endpoint Management >> Agent Installations.
3. The agent administrative password is then required
4. Click Check In Now on the agent console

We are working through some other scenarios so I will update this post accordingly in hopes to help out some other customers in our situation.

[thenetworksfine]

@btenberge See my reply to @Alexandre_Jodoin below.