Credential agent crashes LSASS

[palousermatthew]

Setup a 2016 RODC so I could use the Credential Agent.

As soon as I try starting the agent as system, the server pops a message that I will be force restarted in 1 minute. It non-gracefully reboots in 1 minute. I tried agent v10 and v9. Perms and settings appear fine afaik, and suppressing a/v didn't help. Palo sent me a suggestion to roll back patches before Jan or even before July of last year but that doesn't seem right, plus Jan is the baseline in my template. Has anyone experienced a similar issue and had any luck?

Faulting application name: lsass.exe, version: 10.0.14393.4704, time stamp: 0x615be0cd
Faulting module name: samsrv.dll, version: 10.0.14393.4886, time stamp: 0x61d5262e
Exception code: 0xc0000096
Fault offset: 0x000000000000bac6
Faulting process id: 0x298
Faulting application start time: 0x01d82bfd507c5710
Faulting application path: C:\Windows\system32\lsass.exe
Faulting module path: C:\Windows\SYSTEM32\samsrv.dll
Report Id: bf2a0ead-8af1-4d85-b595-2509ddf94f46
Faulting package full name:
Faulting package-relative application ID:

----------

The process wininit.exe has initiated the restart of computer RODC-3 on behalf of user for the following reason: No title for this reason could be found
Reason Code: 0x50006
Shutdown Type: restart
Comment: The system process 'C:\Windows\system32\lsass.exe' terminated unexpectedly with status code -1073741674. The system will now shut down and restart.

----------------

A critical system process, C:\Windows\system32\lsass.exe, failed with status code c0000096. The machine must now be restarted.

[JeffCrowell33]

I added a ticket and they wanted some logs. Still nothing at this time. 

[Magnus_App]

We have a case as well but nothing so far. There is this KB but no more info really.

https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA14u0000004NJKCA2&lang=en_US%E2%80%A...

[brucegarlock]

I got an update on our case and PAN is saying that this is a MS issue.  My questions back on the case to PAN are something like the following.  Right now, they are pointing at MS as the cause, but I am not clear on the following:

1.  Is this a bug in the patch?

2.  Is this a new policy MS is now enforcing on domain controllers that would prevent the User ID Cred agent from working properly?

3.  Is MS accepting this as a bug and going to fix it?

[AndreasThanner]

Is there a workaround known to anyone? (Except of uninstall of patches on DCs 🙂 )

[JeffCrowell33]

This is disappointing you are correct. In a change meeting with all our IT staff we discussed this issue and it was 100% we will not remove the patches on the DC to make this feature work. We will wait for a fix or abandon Paloalto all together and find something else that will provide these services. Someone on here reported that Paloalto is blaming it on Microsoft. I reached out to Microsoft about this and their response was "It was an issue with one of their application and it fixed their product." Meaning they are not going to change this patch because it resolved an issue they had with their own product. They don't care about Paloalto. Hopefully Paloalto finds a workaround soon otherwise we will be finding a new product. 

[brucegarlock]

Do any other vendors offer a feature like this?  (i.e. prevent users from submitting domain creds to sites?)

I am going to raise this further with our SE.  I saw in the PAN Reddit thread that someone was a new customer setting this up for the first time.  That leaves a poor impression.  I really hope PAN can resolve this.  It seems like the change MS implemented broke the PAN agent, and if PAN is pointing at MS, maybe this is no longer technically possible? 

[Magnus_App]

MS will have a similar service in windows 11. Looks pretty good and will notify the user directly

[brucegarlock]

I am wondering if anyone has tested with a 2019 or 2022 RODC to see if the LSASS related patches might not be an issue with the cred agent after the Jan 2022 patches from MS.  I just tried April 2022 patches on our 2012R2 RODC, and it still goes into a bootloop.  We are only licensed for up to 2016, so the best I can try is an eval edition of 2019 or 2022, but if others are seeing this issue with those server versions, than I am not going to waste my time.

[NaheedVisram]

We have tried this on a 2019 server and it still goes into a loop

[Jay_Moock]

2019 has never worked.
USER-ID CREDENTIAL AGENT FAILS TO EXTRACT CREDENTIALS ON WINDOWS SERVER 2019 RODC 

[dennispci]

PaloAlto needs to come up with an update. Our case is still being processed... No real update... I looked at the service Magnus_App talked about, this is not what our customers want.

quote: This new feature will help protect users from phishing attacks by identifying and alerting users when they are entering their Microsoft credentials into a malicious application or hacked website. 

I don't want a user being alert when they enter their Microsoft credentials into a malicious app or hacked site, I don't want their company credentials be entered anywhere, and I want to be able to control/audit that.

Come on PaloAlto, please give us an update.

[Jay_Moock]

The last update I received from TAC (after some prodding from our account manager) was on 4/21.

It has been reported to our Engineering team and tracked on Jira ID: WINAGENT-830
At the moment, our development team is still analyzing the root cause and we do not have an update on a fix version yet.

[brucegarlock]

Glad to hear this is with engineering.  This is one feature when I talk to others in IT that really raises eyebrows.  Almost everyone I talk to would love to have a feature like this, and I think it's a great selling point for PAN firewalls.  I really hope engineering can come up with a solution.  

[Eric_Ratermann]

Is there a Bug ID for this?

[brucegarlock]

The Jira case ID is WINAGENT-830