Credential agent crashes LSASS

[palousermatthew]

Setup a 2016 RODC so I could use the Credential Agent.

As soon as I try starting the agent as system, the server pops a message that I will be force restarted in 1 minute. It non-gracefully reboots in 1 minute. I tried agent v10 and v9. Perms and settings appear fine afaik, and suppressing a/v didn't help. Palo sent me a suggestion to roll back patches before Jan or even before July of last year but that doesn't seem right, plus Jan is the baseline in my template. Has anyone experienced a similar issue and had any luck?

Faulting application name: lsass.exe, version: 10.0.14393.4704, time stamp: 0x615be0cd
Faulting module name: samsrv.dll, version: 10.0.14393.4886, time stamp: 0x61d5262e
Exception code: 0xc0000096
Fault offset: 0x000000000000bac6
Faulting process id: 0x298
Faulting application start time: 0x01d82bfd507c5710
Faulting application path: C:\Windows\system32\lsass.exe
Faulting module path: C:\Windows\SYSTEM32\samsrv.dll
Report Id: bf2a0ead-8af1-4d85-b595-2509ddf94f46
Faulting package full name:
Faulting package-relative application ID:

----------

The process wininit.exe has initiated the restart of computer RODC-3 on behalf of user for the following reason: No title for this reason could be found
Reason Code: 0x50006
Shutdown Type: restart
Comment: The system process 'C:\Windows\system32\lsass.exe' terminated unexpectedly with status code -1073741674. The system will now shut down and restart.

----------------

A critical system process, C:\Windows\system32\lsass.exe, failed with status code c0000096. The machine must now be restarted.

[kiwi]

Hi @palousermatthew ,

Sounds like a recent MS update broke the credential detection agent. Any update/workaround from support if they are planning a fix or is rolling back currently the only option ?

Cheers,

-Kiwi.

[dennispci]

Hi Matthew, I have exactly the same issue. The RODC is running in Azure with the latest patches (I presume, it is a fresh one).

I have a case running with PaloAlto, hopefully there will be a fix fast....

[NaheedVisram]

We are having the same issue as well and had to revert patches to get it running again. Tried both v9 and v10 as wekk

[palousermatthew]

Which exact patch did folks have to revert to fix this? Just the Feb CU?

[NaheedVisram]

We have had the issue with both the January and February CU updates 

[JeffCrowell33]

We are experiencing this same issue. We tried to patch the server in Jan and Feb. This caused the server to go into a loop. Once the User-ID Agent service launches it triggers the server to reboot in 1 min. I disabled the User-ID agent so the server is now stable. I looked at the patch and the logs on the server. The windows patch has an update to the Lsass.exe in it. The error log right after the User-ID Agent is triggered it give the error "A Critical system process, C:\windows\system32\lsass.exe, failed with status code c0000096. The machine much now be restarted." When I rolled back the patches on the RODC server I kept getting a BSOD and had to disable the driver on the boot to get the server to load. Once this was up I had to turn the nic back on and I was allowed to launch the User-ID service. I'm trying the March patch now. I'll update what happens after the March patches are installed.

[NaheedVisram]

we had the same issues with the march patches as well

[JeffCrowell33]

Still no good after the March update. If I remove the updates the server give a BSOD on boot but I can get the paloalto UserID agent to run. Apply the updates and the server boots normal but once I turn on the UserID agent the server crashes. Hope Paloalto is looking into this. 

[btroudy]

We are having the same issue, we also have an open support ticket on this.  Only resolution at this point is to remove Jan/Feb/Mar CU. 

[palousermatthew]

Thanks for all the replies. This saved me from sinking additional hours into troubleshooting.

[brucegarlock]

I found this via a link in the paloaltonetworks subreddit, and was subjected to the couple of hours trying to figure out what was going on.  Luckily, I knew about the botched MS patches that caused bootloops, so uninstalling the patch tipped me off.  I then noticed that the cred agent was not connected properly to the FW, so I disabled the service and the server was stable.  As soon as I started it, I got the message that the server was going to reboot in one minute.  It doesn't even cleanly shutdown, it acts like issuing a reset in VMware.

The last patches that seem to work are December 2021.  Since our users continue to fail phishing test emails, I'll keep the Dec 2021 patches on the RODC for now until we get a fix.  I did open a case and recommended others do as well so it receives some priority.

[palousermatthew]

Thanks for the reply and sorry I didn't reply faster on that thread. Hope this helps gain visibility.

[NaheedVisram]

anyone else hear back from support? Ive just been told theyre investigating....

[brucegarlock]

I wish...  I put in a S2 ticket last Wed, the 16th.  I did not even have my case assigned until the 18th.  Way behind on the SLA for a S2.  The current workaround is to reduce the security of the RODC, or disable the Cred agent.  I hope PAN fixes support.  Certainly a good problem to have (many new customers needing support), but not good for those of us paying for premium support, and not receiving it.