Using Regex in Cortex XDR XQL query

[Kavurisowmya]

I want to extract a particular string from a particular field value from the Cortex XDR endpoint category and use it in XQL query. Is it possible to do that?

For example from the below string (it is a field )

arn:aws:ec2:eu-abcd-1:123456789:instance/i-000000000000

I would like to extract the number 123456789 in an XQL query and use it to get the count of all the hosts which has the same value as 123456789.

How can I do that? 

Cortex XDR  Endpoint Protection

[jtalton]

Hi Kavurisowmya, 

Yes, you may. Vendor Logs ingested from some unsupported external data sources sent to Cortex XDR may not match the format of the data already present in XDR from other sources.  In this case, you can use additional XQL functions to transform the data within your custom parsing rules. Reference Ingest External Alerts • Cortex XDR Pro Administrator Guide • Reader • Palo Alto Networks documentat...

You may use regex to convert the fields to extract the data to facilitate matching with Cortex XDR format. For example, say you need to map the port but your source field contains both the IP address and port (192.168.1.200:8080). To extract everything after the :, use the following regex:

^[^:]*_

Here is an example with Zscaler logs where this statement performs a regular expression match on any group of characters except whitespace characters, the resulting array will then have each field in an index, and we can simply choose the array index that corresponds to the field we want to assign. 

String with a delimiter

dataset = zscaler_nsssaasfilelog_raw 

| fields _raw_log

| alter date = arrayindex(regextract(_raw_log, "[^\s]+"),0)

In the example, the 0th array index (recall that arrays are 0-indexed data structures, so the first value is stored at index 0) corresponds to the first value in the log format.

In your example the delimiter is : so you will need to pull selected value for the regex to pull the data values you would like to extract. 

Reference

• https://regexr.com/
• regex101: build, test, and debug regex - the regex implementation used by XDR is in Golang. This change affects the regex syntax and supported options. When using regex101, make sure you select the correct "Flavor" on the left.
• regextract • Cortex XDR XQL Language Reference • Reader • Palo Alto Networks documentation portal - XQL uses RE2 for its regular expression implementation which supports single group capturing, look behind and other regex features are not supported. 
• XQL Functions Reference

If you found this answer helpful, please like and select Accept as Solution. 

Thank you!

[seb_rupik]

Hi there,

I love regex, but don't have any experience with XQL, but I think the bit of code you are looking for is:

| alter uid = arrayindex(regextract(_raw_log, "(\S+?:){4}(\d+)"),2)

...assuming regextract returns an array with three elements: the match and two groups. Hence you want the second group at index '2'.

https://regex101.com/r/24tFJl/1

cheers,

Seb.