topic Re: DHCP feed to Cortex XDR in General Topics

DHCP feed to Cortex XDR

PeterMS — Tue, 10 Oct 2023 12:47:01 GMT

Dear all,

I have troubles to feed our DHCP logs into Cortex XDR.

I watched this Video:

https://www.youtube.com/watch?v=rxmn1sYzIlY

and for the installation I used this manual:

https://docs-cortex.paloaltonetworks.com/r/Cortex-XDR/Cortex-XDR-Pro-Administrator-Guide/Ingest-Logs-from-Windows-DHCP-using-Elasticsearch-Filebeat?tocId=1oUsTnJzhhrKS4esy_2enw

I created a XDR Collector for Windows and installed it on two DHCP servers (the servers are visible under "Administration").
I created a "Windows DHCP" instance under "Configurations > Data Collection > Collection Integrations".
I created a Filebeat XDR Collector profile*.
I created a XDR Collector policy and added the two DHCP servers and the XDR collector profile.
Result: no logs are feeded into Cortex XDR:

* here are the profile settings:

# ============================== Filebeat inputs =============================== filebeat.inputs: # Each - is an input. Most options can be set at the input level, so # you can use different inputs for various configurations. # Below are the input specific configurations. - type: log # Change to true to enable this input configuration. enabled: true # Paths that should be crawled and fetched. Glob based paths. paths: - c:\Windows\System32\dhcp\DhcpSrvLog*.log # ============================== Elasticsearch Output =============================== output.elasticsearch: enabled: true # Array of hosts to connect to. hosts: ["https://xxxxx.xdr.eu.paloaltonetworks.com:443/logs/v1/filebeat"] # Protocol - either `http` (default) or `https`. protocol: "https" compression_level: 5 # Authentication credentials - either API key or username/password. api_key: "xxxxx" # ================================= Processors ================================= processors: - add_host_metadata: when.not.contains.tags: forwarded - add_fields: fields: vendor: "microsoft" product: "dhcp" - drop_event.when.not.regexp.message: "^[0-9]+,.*" - dissect: tokenizer: "%{id},%{date},%{time},%{description},%{ipAddress},%{hostName},%{macAddress},%{userName},%{transactionID},%{qResult},%{probationTime},%{correlationID},%{dhcid},%{vendorClassHex},%{vendorClassASCII},%{userClassHex},%{userClassASCII},%{relayAgentInformation},%{dnsRegError}" - drop_fields: fields: ["message"] - add_locale: ~ - rename: fields: - from: "event.timezone" to: "dissect.timezone" ignore_missing: true fail_on_error: false - add_cloud_metadata: ~ - add_docker_metadata: ~ - add_kubernetes_metadata: ~

I tried different configuration styles. I also used the "filebeat.yml" template, but nothing helped.

Do I miss anything or did a mistake?

Do you have some ideas what I can check next?

Thanks,

Peter

Re: DHCP feed to Cortex XDR

BPry — Tue, 10 Oct 2023 13:19:26 GMT

@PeterMS,

So one small thing that you probably want to doublecheck is your path to the log files. Many organizations that I've worked with change this path so it logs to another drive instead of C:\ and don't utilize the default file path for logs. Additionally ensure that your firewall is actually allowing the traffic; infrastructure services like DHCP nodes are usually highly restricted from a network traffic standpoint.

Re: DHCP feed to Cortex XDR

PeterMS — Tue, 10 Oct 2023 14:07:51 GMT

@BPry

Thank you for your reply.

Path at the server: C:\Windows\System32\dhcp\DhcpSrvLog-Tue.log
That should be suitable with the config: c:\Windows\System32\dhcp\DhcpSrvLog*.log

Firewall rules: I searched for blocked traffic before, but there was none. So, all was good for me.

Now, I searched also for existing traffic, and there is also none... (thanks for the push)

Trough this, I took a look at the "C:\ProgramData\XDR Collector\Logs\scouter.log" and there I found this every five minutes:

2023-10-10 15:52:03,221 <ERROR> content_manager.py:get_status:163 - filebeat is not running , error code: 1, b"Exiting: error unpacking config data: more than one namespace configured accessing 'output' (source:'C:\\ProgramData\\XDR Collector\\Data\\content\\filebeat-windows-x86_64\\filebeat.yml')\n"

Is that "filebeat not running" the problem? Because the service "XDR Collector" is running.

Regards,

Peter

Re: DHCP feed to Cortex XDR

PeterMS — Mon, 23 Oct 2023 06:10:43 GMT

Dear all,

with the test:

filebeat.exe test config -c filebeat.yml

I recieved this error message:

Error initializing beat: error initializing processors: each processor must have exactly one action,
but found 2 actions (when,add_host_metadata)

I searched at the Elastic Website and KB, but did not find any solution that works.

If someone of you have any idea...

Regards,

Peter

Re: DHCP feed to Cortex XDR

PeterMS — Wed, 01 Nov 2023 20:07:58 GMT

Dear all,

first of all, it was my fault.

In fact, the installation itself was much easier than described in the documentation.

https://docs-cortex.paloaltonetworks.com/r/Cortex-XDR/Cortex-XDR-Pro-Administrator-Guide/Add-an-XDR-Collector-Profile-for-Windows

All information is there.

But I was very confused by the structure of this section and information that where useless for me.

What worked for me:

I created an XDR Collector installer under Settings > Configurations > XDR Collectors > Installers
- Gave a name
- Selected the OS (Windows)
- Selected the Version (newest)
I downloaded and installed the XDR Collector on the corresponding server. The server was then visible under Settings > Configurations > XDR Collectors > Administration.
I checked if that "Filebeat Status" is "Active".
Under Settings > Configurations > XDR Collectors > Profiles, I created a new Filebeat profile:
- Right clicked the Windows Filebeat Default profile and clicked Save as new
- Gave a name and description
- Clicked below in Select template... and select DHCP
- Clicked Add and Create
I did the same with the Windows Settings Default profile and adjusted it. There I only set Enable under Collector Auto-Upgrade
Under Settings > Configurations > XDR Collectors > Policies I created a new policy:
- Selected + Add Policy
- Gave a name and description
- Select the platform (Windows)
- Selected the new Filebeat and Collector settings profile that I created under point 3. and 4.
- Clicked Next
- Selected the corresponding endpoint
- Clicked Next
- Clicked Done
- Clicked Save
After some minutes I went to the Query Builder, selected XQL Search and used this command:
dataset = microsoft_dhcp_raw
and saw the long awaited data.

I hope that maybe this will help someone else as well.

Regards,

Peter