topic Re: Question About Custom Logs Time Field in Cortex XDR Discussions

Question About Custom Logs Time Field

H.Fukuda — Wed, 04 Sep 2024 01:20:29 GMT

Question

I want to replace _time field value with original timestamp, but I can not find way to do this.
Please tell me how to replace _time field value or Is this not possible due to specifications?

Background

When we collect logs from XDR Collector, which ingest three fields which related time.

First one is _time, which is generated by XDR Collector.

Second is _insert_time, which is generated by Cortex XDR.

Last one is original timestamp which recorded in log ( which included _raw_log or single dedicate field using parsing rule or filebeat setting)

For example, if I ingest apache http severlog, then it shows like this.

Between _time and datetime(which was created by parsing rule from _raw_log field), there are some gaps around 1 to 10 seconds.
I want to erase these gaps.

Re: Question About Custom Logs Time Field

jmazzeo — Thu, 05 Sep 2024 18:42:55 GMT

Hi @H.Fukuda, thanks for reaching us using the Live Community.

The _time fields is a system field that takes the value from the data entry's timestamp. If unknown, then the value is the time the data entry was added to the database. In your case you have a timestamp value in the logs, and looks like is accurate.

Is your "datetime" field rounding to zero the seconds?

Re: Question About Custom Logs Time Field

H.Fukuda — Fri, 06 Sep 2024 00:19:25 GMT

Hi Jmazzeo,

>Is your "datetime" field rounding to zero the seconds?

No.

To show the gap, I build apache server, and generate log with using shell script which generate log each 10 seconds.

So original datetime field's seconds data will be zero.

Re: Question About Custom Logs Time Field

paIoaItonetworks — Tue, 10 Sep 2024 12:15:28 GMT

Unfortunately, these fields will always be shown