Identify syslog type for User-ID parse

[JermaineScott]

I'm in the process of implementing User-ID and want to parse syslog logs. the predefined parse profile don't appear to be a match, as I'm looking to pull syslog from my domain controller. However, my Active Directory team can't provide me with a sample of the syslog event or tell me the type of syslog (regex or field) being generated. So I'm looking from some additional information about to identify the type of syslog file to help push this process along. Anything would be helpful. Thank you.

[S.Cantwell]

The UserID agent  can already parse traffic from the Security log on your DC.

Can you please explain the need for syslog parsing, when the solution already has the native ability to do it?

You would use the syslog parsing for Linux/Unix (PAMs or Portable Authentication Module), or Aerohive APs, and the rest of the default parsers.

For you, a service account with Event Log Readers, Service Operator Priviledges, and Distributed COM User is what is needed to read Microsoft logs, natively.

[JermaineScott]

Hey @S.Cantwell 

My rationale is that of the default Syslog filters (Aerohive AP, Unix, Citrix, etc.), Microsoft Domain Controllers are not listed. My goal is to leverage syslog events to conduct user to IP mappings. I didn't adding all default files to a server monitor would provide the expected results. Please let me know if I missed something here.

As far as a service account: I've found with my AD team and it doesn't that I'm getting access to Service Operator Privileges. According to Palo Alto, I should "need" access to Server Operator for User-ID to function.

[ddelcourt]

@JermaineScott, what @S.Cantwell is saying is that there is no MS AD Syslog parser because we don't do general Syslog scraping for MS AD. PAN-OS actually talks natively to the DC to gather the information as it is much more effective and efficient. In short, take a look at the User-ID documentation, https://docs.paloaltonetworks.com/pan-os/9-0/pan-os-admin/user-id.html is PAN-OS 9.0 doc, as there is some great information on how to configure User-ID to talk to your AD DCs and gather the appropriate information.

There are special Syslog parsers, as identifed, that have been created to look for specific information from these specific sources because that is the only way to gather information needed to do IP Mapping. It is also then leaning on receiving Syslog messages, not the most reliable mechanism.

If you have further questions after reviewing the documentation please do post, a lot of smart folks out here... 🙂

[JermaineScott]

@ddelcourt  @S.Cantwell 

Thanks for your responses, I realized that misunderstood what you both were saying and probably didn't provide enough insight into my situation.

So to start, I do understand that the Palo Alto is able to speak directly with the Active Directory servers. I've implemented that setup on multiple occasions. During this new setup instance running into a little hang up. I'm getting very limited mappings, about 100 users from what I expected to be a couple thousand users. So as we troubleshoot we find out.

1) The Activate Directory team is not allowing the Service account access to Server Operator privileges. However, speaking with Palo TAC, it's said that the function should still work.

2) After troubleshooting with Palo TAC we found out the Active Directory servers do not Audit successful logon events. which would be required for the Palo Alto to read the logs. Apparently the servers can't handle the load of that many logs, and there's no plan to upgrade any time soon. This is a bit confusing, because I wondered why I'm getting any logs at all.

So now management wants to know (because our syslog server (Qradar) has logs/events that contain username and IP address coming from the domain controller), if the Palo Alto can pull the same syslog events being sent from the Domain Controller to the syslog server into the firewall to be used for User-ID mapping. That's why I have been trying to parse syslog files from the domain controller. 

Does this make sense to any of you? An if so, am I going about it the right way?

[S.Cantwell]

@JermaineScott 

Glad to assist you.

Are you using the builit in UserID agent or the standalone version (that you would have installed on your DC or another domain'd server)

Why is the MS not allowed ServerOperator Privs?  I understand that the UserID can limp along without, but what is the security concern from the MS team?  Did they have any issues with providing Distributed COM User?  You should go back and check this one as well.

If you are missing mappings, then I would use the standalone version.  I would configure the timeout timer to be 240 minutes (4 hours) and ALSO, your DHCP scopes should be reduced to 1 day (8 hour to 12 hours) and not 3 to 8 days (as I have seen it before).

The standalone also looks at the last 50k of security logs (looking for login/logff requests).  Are you stating that the Security Log (part of the Event Logs on the DC) cannot keep up with the amount of users logging into the DC?   That is very strange, as the PANW solution can handle 100 DC/Exchange servers, so yes, the limiting factor could be your servers.

The probing function is another way to find missing windows machines (no userID info), but that requires the Distributed COM User privilege.

You could configure GlobalProtect with always on (and not provide an internal gateway) and now all users would authenticate to the FW (and to your LDAP server) so that you would get logs.

You could enable authentication policies (a good thing!!!) to safely/securely allow access to internal web servers, after a person authenticated.

There are many different ways to get log info... Maybe a different UID technique is needed. 

[JermaineScott]

@S.Cantwell 

Thanks for your feedback. See my answers to your inquiries.

Are you using the builit in UserID agent or the standalone version (that you would have installed on your DC or another domain'd server)

I'm using the Built-in User ID Agent.

Why is the MS not allowed ServerOperator Privs?  I understand that the UserID can limp along without, but what is the security concern from the MS team?  Did they have any issues with providing Distributed COM User?  You should go back and check this one as well.

The AD team has a security concern that a Service Account with that level of access can begin opening and closing user sessions. So the they feel that could be a threat, especially if someone or some thing was able to get access to the service accounts password. After troubleshooting, Palo TAC said that access to Server Operator wasn’t necessary for User-ID to work, so I lost my ground in that battle. Might be worth revisiting if I can prove that this is the only way to get User-ID to function properly.

No issues getting access to Distributed COM.

If you are missing mappings, then I would use the standalone version.  I would configure the timeout timer to be 240 minutes (4 hours) and ALSO, your DHCP scopes should be reduced to 1 day (8 hour to 12 hours) and not 3 to 8 days (as I have seen it before).

There are 13 domain controllers in three different location. So, would I have to install a standalone on each server, or just one per location? (like I planned with the built-in agents). 

As much trouble as I'm having collaborating between teams, I'd rather not rely on having access to another teams server, if I don't have to.

The standalone also looks at the last 50k of security logs (looking for login/logff requests).  Are you stating that the Security Log (part of the Event Logs on the DC) cannot keep up with the amount of users logging into the DC?   That is very strange, as the PANW solution can handle 100 DC/Exchange servers, so yes, the limiting factor could be your servers.

The argument from the AD team’s perspective is that “to activate Auditing for successful logon/logoff would mean that the server would store successful logon/logoff information for a period of time, and given the amount of users on the network, they fear that the stored logs would be more than the servers could handle.

The probing function is another way to find missing windows machines (no userID info), but that requires the Distributed COM User privilege.

I read in multiple places enabling probing was no recommended and an older practice. Fortunately I’m aware of all the Windows Domain controllers in the network

You could enable authentication policies (a good thing!!!) to safely/securely allow access to internal web servers, after a person authenticated.

I considered this with Captive Portal, but the limitation I encountered was that I’m mappings of North/South traffic of users accessing the internet but not east/west amongst the data centers.

There are many different ways to get log info... Maybe a different UID technique is needed.

Syslog parsing is my second option. If this get the FW to pull syslogs from the DC or have the Syslog server forward traffic to the FW (sounds backwards) l have to approach this another way, maybe Kerberos.

Thanks again. Any feedback is helpful.

[S.Cantwell]

@JermaineScott 

Thanks for the info...

So, what I see/feel/believe is that your Built-in, is not able to keep up with the amount of LDAP traffic that is being used.

The recommendation (surprised that TAC did not mention this) would be using the standalone version (where it will be installed on a DC or a server that is part of the domain). 

I also believe (in my opinion) that this warrants escalation with your company to ease the lines of communication for collaboration.

Our communication here, has been back and forth for about 1 week or so... which (IF) you had the support of your team, would have been able to get these resolved.

I do not have the experience/skill to try and parse from the Syslog profile (what we originally chatting about).

Maybe someone else can provide their input... and that would be your MS team and/or MS TAC themselves, to help you parse out what the syslog profile should be.  PANW TAC may not be able to assist you.

I guess.. good luck and keep us in the loop.

[JermaineScott]

@S.Cantwell 

Thanks a ton.

FYI: I didn't specify that I'm using the built-in on a PA-7050. So I (we) didn't think processor power with be a limiting issue. Unless you're saying it's the functionality of the built-in itself that can't support it.

I'm still chasing down some theories on how to import syslog events but If I can't get that to give the results I need I've got a couple takeaways from our conversation that I'm going to try:

1) Secure a server to install a stand alone User-ID agent.

2) Reach out to AD teams to conduct do a temporary test of adding our Service Account to the Server Operators group for 24 hours to see fixes the issue. 

Thanks again for your help. I will definitely provide an update when I find a solution.

[S.Cantwell]

@JermaineScott 

Always glad to assist you.

To be more accurate in my comment about the Server Operator Privs, I agree it will not prevent UserID from working, however.... it is one of 3 techniques used by PANW, when working with LDAP.

1) Event Log Reader (to read the security logs)

2) Server Operator Privs (which is ONLY used by the FW to confirm an IP is maintaining its file/print share sessions. If your MS has issues with it, have them escalate it up to PANW, but in the meantime, it really is NOT a security concern, because your MS team controls the password.. It is not on the FW.

3) Probing... (which is not a bad thing....) using WMI.  I was surprised that your MS group provide you with an elevated privilege (normally reserved for Domain Admins...yet fumble on a more secure ServerOpPriv service. ) so that you can probe unknown IPs.

Other techniques that you should be investigating... Syslog profile from your wireless controllers, enabling GP (always ON with no internal gateway) so that users can authenticate to (and through) the FW.

Thanks.. and happy firewalling!!! 

[asilliker]

If you have events that your can access in qradar that contain both a valid username and that IP address then you will be able to forward those events to a firewall for parsing of those messages.

It doesn't make sense that there would be a built in parser for this, the firewall won't know what your message format is. If there was going to be a default, it would be for the same messages that the User-ID agent would see, which do not exist in your environment as you are not auditing auth success messages. 

This is why you can write a regular expression to parse them. 

If you get a pcap of what the log messages look like on the wire, or a sample log from qradar, you can use something like regex101.com to help you appropriately craft the regular expressions required to capture the username and IP out of the message. 

If you can massage the info before you forward it out of qradar, you may also be able to use field matching if you can simplify the message format. 

[JermaineScott]

@asilliker 

Thanks for the response. I've been able successful log event in Qradar and grab sample of the log to use for a custom parse. A few things I ran into.

1) When going through Palo documentation I read that that "syslog senders were the network services that authenticate users. I took this as, the syslog sender had to be the Domain Controller and I'd have to pull the logs directly from it. Thus, I haven't tried to go down the path of having the Qradar forward syslog events to the User ID Agent. Did I interpret that incorrectly?  

2) Another sticking point: When attempting to create a custom parse file, Palo requires the identification of a regex vs field type. I don't know enough about these formats to understand which is which by just looking at the Qradar log. Do you know how to identify the type? 

3) Does the syslog server log into the domain controller to pull logs, or does the Domain Controller forward its logs to the syslog server? I found an article that send syslog listeners will log in and pull information, so I've setup the agent with a service account that has access to the domain controller but I haven't gotten anything, so I'm wondering if the DC needs to be told to forward log traffic to my User ID agent. I haven't gotten much feedback from my team on how the setup actually works so still trying to figure it out.

Thanks for your feedback.

[S.Cantwell]

Typically, the device that supports syslog messages sends the messages to the syslog server.

So think about any network piece of hardware (switch/router, wireless access controller, etc)

These devices would typically fwd to the IP of a syslog server.

In this case, your syslog server should be sending them to another syslog server (in this case, the IP of the FW mgmt)

What does a log message look like in QRadar, when the DC successfully fwds the messages to the syslog server?

[JermaineScott]

@S.Cantwell 

Got it.

As for the log message in Qradar:

<13>Nov 27 10:37:56 x.x.10.37 AgentDevice=WindowsLog AgentLogFile=Security PluginVersion=7.2.9.72 Source=Microsoft-Windows-Security-Auditing Computer={Domain.Controller.Name} OriginatingComputer=x.x.10.37 User= Domain= EventID=4624 EventIDCode=4624 EventType=8 EventCategory=12544 

RecordNumber=959957311 TimeGenerated=1574869076 TimeWritten=1574869076 Level=Log Always Keywords=Audit Success Task=SE_ADT_LOGON_LOGON Opcode=Info Message=An account

was successfully logged on.  Subject:  Security ID:  NULL SID  Account Name:  -  Account Domain:  -  Logon ID:  0x0  Logon Information:  Logon Type:  3  Restricted Admin Mode: -  Virtual

Account:  No  Elevated Token:  Yes  Impersonation Level:  Impersonation  New Logon:  Security ID:  {Domain\User-ID}  Account Name:  {User-ID here}  Account Domain:  {Domain.Name.Here}  Logon ID:  0xA1D8416  Linked Logon ID:  0x0  Network Account Name: -  Network Account Domain: -  Logon GUID:  {40724BD3-9F5D-A463-A21A-367F5AA824FF}  Process Information:  Process

ID:  0x0  Process Name:  -  Network Information:  Workstation Name: -  Source Network Address: x.x.196.115 Source Port:  49203  Detailed Authentication Information: 

Logon Process:  Kerberos  Authentication Package: Kerberos  Transited Services: -  Package Name (NTLM only): -  Key Length:  0  This event is generated when a logon session is created. It

is generated on the computer that was accessed.  The subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server

service, or a local process such as Winlogon.exe or Services.exe.  The logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3

(network).  The New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.  The network fields indicate where a remote logon request

originated. Workstation name is not always available and may be left blank in some cases.  The impersonation level field indicates the extent to which a process in the logon session can

impersonate.  The authentication information fields provide detailed information about this specific logon request.  - Logon GUID is a unique identifier that can be used to correlate this

event with a KDC event.  - Transited services indicate which intermediate services have participated in this logon request.  - Package name indicates which sub-protocol was used among the

NTLM protocols.  - Key length indicates the length of the generated session key. This will be 0 if no session key was requested.