Ingest DHCP logs using XDR collector

[PeterBengtslin]

Hi,

I am having issues with ingesting DHCP log from our DCs. We are using the XDR Collector app. I suspect that the issue is with the filebeat.yml file but cannot figure out what the problem is. I have tried and followed the guide below and copy-paste the example code but no logs are showing up. The yml has been checked and the syntax is correct. 

https://docs.paloaltonetworks.com/cortex/cortex-xdr/cortex-xdr-pro-admin/cortex-xdr-collectors/add-a...

Is there anyone that has successfully managed to ingest those logs? 

Thanks!

[bbarmanroy]

Hi @PeterBengtslin have you checked if the path to the DHCP logs are valid? Is the issue persistent if you use a filebeat collector?

[PeterBengtslin · Accepted solution]

Hi @bbarmanroy Thank you for your reply. The path is valid but there is something in the filebeat configuraion that is causing it to inactivate the service. I have been doing some labs on this and found out that this configuration is giving an acceptable result: 

filebeat.inputs:

  - type: log

    enabled: true

    paths:

      - c:\Windows\System32\dhcp\DhcpSrvLog*.log

processors:

  - add_fields:

      fields: 

        vendor: "microsoft"

        product: "dhcp"

  - drop_event.when.not.regexp.message: "^[0-9]+,.*"

  - dissect: 

      tokenizer: "%{id},%{date},%{time},%{description},%{ipAddress},%{hostName},%{macAddress},%{userName},%{transactionID},%{qResult},%{probationTime},%{correlationID},%{dhcid},%{vendorClassHex},%{vendorClassASCII},%{userClassHex},%{userClassASCII},%{relayAgentInformation},%{dnsRegError}"

This will store the information in dataset microsoft_dhcp_raw and the content will be split into fields defined in the tokenizer statement. 

Best regards,

Peter

[Honza_Linhart]

Hi, I found the problem causing service inactivation. There is a \n inserted between "tokenizer:" and its parameter that has to be removed. The result has to be on one line.

[KarlHalpin]

Hi Peter,

I had this issue with the the collector as well and thought it was the yml file also.

I thought when you install the collector it installs the file beat service with it but it dosent.

I had to go to C: > Program data>XDR Collector > Content > filebeat-windows-x86_64  run the install-service-filebeat from powershell and then start the service from powershell as well.

Once i did that it worked for me. 

[JahidAliyev]

Hello,
I created XDR collector package and installed it on my windows server. But it does not show up on Administration tab where i am supposed to see all xdr collector agent lists. What can be the reason for this? 

[Honza_Linhart]

Hello @JahidAliyev 

this looks like communication issue - server (XDR collector on the server) is not able to reach XDR management console. Try to check FW outbound rules (local FW on the server and any FW in the path to XDR management). 

[JahidAliyev]

When I am in organization network, it cannot communicate. When I am using my own internet, it directly connected. If I switch to my organization network, will it be disconnected again?

[JahidAliyev]

And, do I need to write parsing rules or Cortex XDR will automatically bring DHCP logs under "dhcp" dataset?

[JahidAliyev]

@Honza_Linhart If you answer these two, I would be so happy

[Honza_Linhart]

For proper function, you need XDR collector connected to XDR management console all the time. You're using this connection to ingest data from on-premise server to cloud data lake. So yes, if you put it back to the network that doesn't allow connection, it will be disconnected again.

If you will have your XDR collector up and connected, use config file mentioned in this thread and you will get your DHCP logs into new dataset in Cortex XDR. There is no need to write your own parsers. 

[JahidAliyev]

@Honza_Linhart Surprisingly, when I connect back to my organzition network, it does not become disconnected. 

If I use this as config file, what will be the name of dataset if I want to look at these logs on queries?

And, is it okey to install this collector agent on any windows or we need to install it on DC and DHCP server? Because I simply installed it on usual endpoint which is not DC or DHCP server.