topic Re: cyvrtrap.dll causing spoolsv.exe crashes? in Cortex XDR Discussions

cyvrtrap.dll causing spoolsv.exe crashes?

kindzma — Mon, 05 Aug 2024 18:54:35 GMT

We updated Cortex XDR agent on a number of VMs and on some of them the Print Spooler service (spoolsv.exe) started crashing repeatedly, causing disruptions to operations.

Is this a known issue? Are there available workarounds or ways to resolve it short of downgrading the agent?

Sample events:

Log Name: Application Source: Application Error Date: 7/31/2024 7:59:28 AM Event ID: 1000 Task Category: (100) Level: Error Keywords: Classic User: N/A Computer: V******a.*****.COM Description: Faulting application name: spoolsv.exe, version: 10.0.17763.4644, time stamp: 0xacbcf874 Faulting module name: cyvrtrap.dll, version: 8.5.0.624, time stamp: 0x667afdda Exception code: 0xc0000005 Fault offset: 0x00000000000175d1 Faulting process id: 0xf28 Faulting application start time: 0x01dae2a0fe85bd33 Faulting application path: C:\Windows\System32\spoolsv.exe Faulting module path: C:\Windows\System32\cyvrtrap.dll Report Id: 8a26e6e7-e8e7-4dc9-9cdb-dce6c0798d81 Faulting package full name: Faulting package-relative application ID:

Log Name: Application Source: Application Error Date: 8/1/2024 7:29:24 AM Event ID: 1000 Task Category: (100) Level: Error Keywords: Classic User: N/A Computer: V****a.****.COM Description: Faulting application name: spoolsv.exe, version: 10.0.17763.4644, time stamp: 0xacbcf874 Faulting module name: cyvrtrap.dll, version: 8.4.0.51691, time stamp: 0x667afdda Exception code: 0xc0000005 Fault offset: 0x00000000000175d1 Faulting process id: 0x2f50 Faulting application start time: 0x01dae35a42e79f3d Faulting application path: C:\Windows\System32\spoolsv.exe Faulting module path: C:\Windows\System32\cyvrtrap.dll Report Id: 90dc4222-bee6-42fd-a6a7-5c4f076c9e99 Faulting package full name: Faulting package-relative application ID:

P.S. Downgrading from 8.5 to 8.4 seems to help but does not completely eliminate the crashes.

The version prior to 8.4 and 8.5 was 8.2 or lower - and that one didn't seem to cause these crashes at all.

The host OS is WS2019.

Thank you!

Re: cyvrtrap.dll causing spoolsv.exe crashes?

neelrohit — Tue, 06 Aug 2024 02:21:00 GMT

Hi @kindzma ,

Seems this was reported by another customer on another thread as well and its recommended to open a case with the support team.

Link to discussion: https://live.paloaltonetworks.com/t5/cortex-xdr-discussions/xdr-8-5-0-print-servers-error/td-p/593625

Re: cyvrtrap.dll causing spoolsv.exe crashes?

tlmarques — Tue, 06 Aug 2024 10:01:21 GMT

I've encountered the same issue, but in my case, I have 15 print servers, and the problem appears on all of them when upgrading to version 8.5.0.

When I downgrade to version 8.4.0, everything works fine.

Today, a Cortex system crash occurred on a print server (Version 8.4.0.51691, Content Version 1430-86494).

At this moment, my question is whether the problem might be related to the content version rather than the agent version, since the content version is the same in both versions.

My HostOS is W2022

Re: cyvrtrap.dll causing spoolsv.exe crashes?

Fm12345 — Tue, 06 Aug 2024 10:43:46 GMT

With agent release 8.5 and 3.11, we have an option in device configuration profile to control print jobs in the environment. Try to check if it is enabled..if it is then disable that and see if it solves the issue. See the release notes accordingly.

Capability should not crash the service generally but check with support if it is the root cause.

Re: cyvrtrap.dll causing spoolsv.exe crashes?

tlmarques — Tue, 06 Aug 2024 11:33:13 GMT

Yes, but by default, this is disabled. To enable it, you need to assign an Extensions Profile with the required settings.

In my case, I don't use Extensions Profiles and the problem persists.

I've opened a case, and support advised me to disable the Logical Exploits Protection module in the respective Exploit Profile.

Re: cyvrtrap.dll causing spoolsv.exe crashes?

kindzma — Tue, 06 Aug 2024 15:06:47 GMT

We've got 20 (nearly identically configured WS2019 VMs where print spooler service needs to run, and where if it crashes, users usually call to let us - the IT helpdesk - know). That - in addition to a bunch of other servers that need to print and where Cortex XDR is running - yet we're only seeing the adverse impact on those specific LoB servers.

Some notes:

The default print spooler service configuration is to auto-restart twice on a crash

... which means not all crashes will get noticed - at least in our env - only ones that fail to start after 2 retries.
After we updated Cortex XDR to 8.5 across the board (200+ servers and workstations or so), the only immediate adverse impact (crashed print spoolers) was on those specific types of servers, and then - not all of them - about 5 initially, with 5 more joining the party a week later. We still have about 10 of them with Cortex XDR 8.5 that do not exhibit any crashes, and don't have those application errors mentioning both cyvrtrap.dll and spoolsv.exe. (I know, a mystery. 🤷)
When downgraded to 8.4, there may be one application error like the above - yet the service recovers if it's configured to auto-retry, and then the errors seem to go away. I.e. so far (knock on wood) 8.4 fixes the issue.

Re: cyvrtrap.dll causing spoolsv.exe crashes?

tlmarques — Thu, 08 Aug 2024 17:06:59 GMT

I've 6500 devices with 8.5...only print servers have the issue when upgrade to version 8.5.0.

But another mystery, I've one server on lab, with 8.5.0 and disable exploit module, and problem disappear...right know my question for support is, what is the root cause ...because i don't see any alert or incident??

Re: cyvrtrap.dll causing spoolsv.exe crashes?

tlmarques — Mon, 02 Sep 2024 16:38:42 GMT

solutions is "Disable PrintMonitor for the Windows Spooler service" exploit module.

Re: cyvrtrap.dll causing spoolsv.exe crashes?

kindzma — Mon, 02 Sep 2024 18:36:31 GMT

... or downgrade to 8.4? ("Downgrading" isn't quite the right term as it seems to require a full re-install of the XDR agent?)

Is there a doc on how to do this?

@tlmarques wrote:

solutions is "Disable PrintMonitor for the Windows Spooler service" exploit module.

Re: cyvrtrap.dll causing spoolsv.exe crashes?

tlmarques — Tue, 03 Sep 2024 19:40:28 GMT

Yes, there's no downgrade option in XDR... The only option is to remove the agent (uninstall) via the tenant and then install version 8.4.

to do exception, import the json file and insert the same on rules...

https://docs-cortex.paloaltonetworks.com/r/Cortex-XDR/Cortex-XDR-Pro-Administrator-Guide/Add-a-Support-Exception-Rule