topic Re: Cortex XDR 7.4.1 crashing server in Cortex XDR Discussions

Cortex XDR 7.4.1 crashing server

Jason_Castillo — Fri, 30 Jul 2021 19:43:18 GMT

After the installation of xdr 7.4.1, our domain controllers began crashing, and even after a reboot they would lock up. Has anyone had any issues with the 7.4.1 release on Windows Server 2012 R2?

Based on a Windows Dump, Microsoft reported the following:

findings- We have checked high Contention Count threads where we were able to see Palo Alto Networks driver tedrdrv.sys in the stack trace which is getting loaded at the time of issue when hang happened.

Request you to remove it to isolate the issue, and check if the issue is happening or not.

Since then, we have uninstalled 7.4.1 and reinstalled 7.4.0 and the issues went away. I have created a ticket with Palo Alto to investigate MS claim.

I have searched the forums and net but have not found any issues related to what we are experiencing. If anyone has had similar issues, please report here. I will keep this updated with new information as it arises. Thank you.

Update 7/30:

Palo Alto tech support has confirmed other cases involving AD and DC servers where performance is being affected by agent 7.4.1. Since this is a newly found bug, we are currently testing a deployed fix that occurred within the past 30 mins via our data cortex tenant. For those that are having issues, Palo Alto recommends rolling back to the 7.4.0 agent.

Content update 191-66972 is being tested on our data cortex tenant.

Will keep you posted on future updates. Thank you.

Re: Cortex XDR 7.4.1 crashing server

jcandelaria — Wed, 28 Jul 2021 04:33:35 GMT

Hi Jason,

Definitely a valid case since the issue went away if you remove 7.4.1 and put 7.4.0.

Palo Alto support should be able to isolate or possibly reproduce the issue.

Do you happen to remember the exact minor version number? 7.4.1.xxxx ?

Re: Cortex XDR 7.4.1 crashing server

wkasak — Thu, 29 Jul 2021 13:33:56 GMT

Any updates to this issue? I think we are experiencing something very similar.

Re: Cortex XDR 7.4.1 crashing server

Shmuel — Thu, 29 Jul 2021 13:43:27 GMT

I also had a server malfunction with version 7.4.1 I spoke to the support and they told me to install back to version 7.4.0 and at the moment everything is normal.

Re: Cortex XDR 7.4.1 crashing server

mbahen — Thu, 29 Jul 2021 14:59:24 GMT

Just put a ticket in with support. They acknowledged the bug causing DCs with the 7.4.1 client to lock up randomly. They are working to resolve. In the meantime, they recommended downgrading to 7.4.0. It is scheduled to be fixed in the 7.4.2 release

Re: Cortex XDR 7.4.1 crashing server

Jason_Castillo — Fri, 30 Jul 2021 19:40:24 GMT

Hi,

The agent version is 7.4.1.31675. They are currently testing our environment and, if successful, will deploy a fix soon. Will keep everyone posted.

Re: Cortex XDR 7.4.1 crashing server

JasonPeterson — Fri, 30 Jul 2021 23:05:05 GMT

Does anyone have the process on how to downgrade?

Re: Cortex XDR 7.4.1 crashing server

wkasak — Sat, 31 Jul 2021 11:23:53 GMT

We do the uninstall from the Cortex Admin console then create an .msi package, copy it locally to the server, and install as administrator.

Re: Cortex XDR 7.4.1 crashing server

Jason_Castillo — Mon, 02 Aug 2021 14:19:31 GMT

Hello,

There are two ways to do the downgrade: 1) from the console, or 2) using Msiexec

The more direct approach is using Msiexec especially dealing with only a few endpoints. Here is the link on how to complete this process: https://docs.paloaltonetworks.com/cortex/cortex-xdr/7-4/cortex-xdr-agent-admin/cortex-xdr-agent-for-windows/uninstall-the-cortex-xdr-agent-for-windows.html

Re: Cortex XDR 7.4.1 crashing server

Jason_Castillo — Mon, 02 Aug 2021 14:23:28 GMT

8/2 Update:

Thus far no issues have been reported on our DCs on the fix provided by tech support on 7/30. I Will post once the word is official.

For those that are on 7.4.x, Palo Alto recommends downgrading to 7.3 (Although the 7.4.0 agent worked fine for us once we downgraded).

Re: Cortex XDR 7.4.1 crashing server

Jason_Castillo — Wed, 04 Aug 2021 20:46:35 GMT

Update 8/4/2021:

We have closed the ticket with Palo. The fixed worked. They will have it ready in the next release. No ETA when that will occur. Thank you for all your follow-up on this case.

Re: Cortex XDR 7.4.1 crashing server

stefan.tomasevic — Mon, 25 Sep 2023 09:11:55 GMT

Hi, what is currently stable version of XDR for windows file server?

I am about to install xdr on several servers (not all at once) and I am about to use new policy and profile (exploit, malware, agent blades) with all default setting. Are there any option that I should change before putting xdr into production environment?