Parsing and Mapping 3rd party log source logs

[paIoaItonetworks]

Hi,

We’re in the process of ingesting logs from multiple third-party systems into Cortex XDR, but the current documentation on user-defined parsing rules and dataset mapping isn’t clear enough. Is it possible to get a detailed step-by step plan on how to properly:

1. Define and register a new dataset (dell_powerprotect_data_manager).
2. Write and apply parsing rules to extract fields from raw syslog entries (currently landing in unknown_unknown_raw).
3. Map parsed events into their dedicated dataset.

At the moment, there is a parsing rule set (photo attached) and in the query builder, I've run a query with a target_dataset parameter which moved logs from unknown_unknown_raw logs to a custom dell_powerprotect_data_manager dataset. When a query "dataset = dell_powerprotect_data_manager | sort desc _time" completed, the mapped logs are shown, BUT there is a problem that these logs are only those who appeared on XDR until the date/time when the query with 'target_dataset' was run and no new logs are being moved from unknown_unknown_raw to dell_powerprotect_data_manager. What would be the solution to make those logs move in real time between these two datasets? I've heard that there is a possibility to create some sort of preset that move logs in real time to preferred dataset?? Also, do I only need to use the [INGEST] section when setting up the rule or do I also need to use [COLLECT/CONST/RULE] sections?

Thanks in advance.

[eluis]

Hi ! 

At XQL query builder, you can set the run of your query that populates your custom dataset with the periodicity you want, so the dataset will be updated with the frequency you need. And you can set the query to overwrite the dataset or to append data at the end. 

As per your second question, you can use just the INGEST (for the dataset) and RULE (for the logic). 

If you feel this has answered your query, please let us know by clicking like and on "mark this as a Solution". Thank you.

KR, 

Luis

[A.Elzedy]

Hi @paIoaItonetworks , 

I'm assuming you're using vm broker because it's landing in unknown_unknown_raw, so in order to adjust that 

go to Configurations - > Data Broker -> Broker VMs and then you could assign a custom port ( if you're receiving more than one category on the same port ) , then assign a vendor and product 

[paIoaItonetworks · Accepted solution]

Is it also possible to somehow delete default parsing rules to which my custom datasets were saved? I can delete newly created datasets from dataset management, but default parsing rules are read-only, therefore I can't modify/delete them. I'd like to delete them and freshly reconfigure the parsing rule.