topic Re: User ID causing heavy load on domain controllers in General Topics

User ID causing heavy load on domain controllers

Maxstr — Mon, 28 Sep 2015 14:23:25 GMT

My domain controller is seeing very high CPU and RAM usage caused by the event log settings (as required by User ID). It's currently at 427,000 events and it's using up about 60% CPU.

Is this normal?

Re: User ID causing heavy load on domain controllers

cpainchaud — Mon, 28 Sep 2015 14:48:17 GMT

Eventually yes,

Number of logs per seconds (not just logon events but all events) and resource constraints (CPU+RAM) on your domain controller may explain this.

I suppose you are using FW embdded agent, Windows agent works differently and shall put less pressure on your domain controller.

Re: User ID causing heavy load on domain controllers

Maxstr — Mon, 28 Sep 2015 15:10:29 GMT

Yes, I started using the firewall agent after the standalone agent stopped communicating with the PA. For some reason the PA doesn't appear in the list, and I have tried re-installing it

Re: User ID causing heavy load on domain controllers

cpainchaud — Mon, 28 Sep 2015 15:12:30 GMT

do you have stats of logs/second on your Domain Controller ?

can you describe its hardware ?

Re: User ID causing heavy load on domain controllers

Maxstr — Mon, 28 Sep 2015 15:58:27 GMT

It is a virtual machine, Win2008 R2, with 1vCPU and 8 GB ram. I was hoping not to throw more resources at it, especially since there are a total of 4 domain controllers. The other 3 do not seem to have this issue with the amount of logs, which leads me to believe there may be a configuration issue

Re: User ID causing heavy load on domain controllers

cpainchaud — Mon, 28 Sep 2015 16:01:05 GMT

It's common to see some DC take a lot more load than others. many factors can explain that.

anyway, you may want to add at least 1vCPU to your DC VM. if it's really busy then it's going to help it.

but you are right, you should keep investigating what is being logged there, and the volume/hour

Re: User ID causing heavy load on domain controllers

kbiswas — Mon, 28 Sep 2015 18:41:46 GMT

Hi Max,

Agentless User-ID utilizes WMI to connect directly from the Palo Alto Networks firewall to an AD server (or servers) and obtain user IP information.
On some older servers (for example, Windows 2003), the memory allocation for WMI may be constrained, which then prevents the system from parsing the server security logs.
Do take a look at the below article :
https://live.paloaltonetworks.com/t5/Management-Articles/Agentless-User-ID-Error-quot-failed-to-parse-security-log-buf/ta-p/58815

You also have the option to use the User-ID Agent, which is a software application that runs on your DCs if agentless User-ID is not feasible for your network

You can install the agent directly on domain controller or another server where security logs will be read from.
This is much lesser resource intensive for both the PA firewall and the Domain Controller, as it uses Microsoft RPC- which is native to Microsoft unlike WMI.

I was going through a Microsoft sites and came across issues being reported with "wmiprvse.exe" service in Windows server 2003 and ntdll.dll service when an external service tried to interact with these services. There is hot fix released to address wmiprvse service causing high CPU usage.
The link for the fix is below:
http://www.microsoft.com/en-us/download/details.aspx?displaylang=en&id=1157

For the ntdll.dll service, there have been reported crashes of this service in windows 2008 R2 server and necessary steps and links to the documents addressing this issue has been provided in the following link:
http://social.technet.microsoft.com/Forums/windowsserver/en-US/164c5cc5-810a-47f5-97ba-91fa7982c123/ntdlldll-keeps-crashing-on-windows-2008-r2
There have been issues reported with these processes. Check for any errors related to ntdll.dll service in the windows 2008 server ?

Thanks and Regards,

Kunal