topic [INT] Alternative for Data Lake in General Topics

[INT] Alternative for Data Lake

paIoaItonetworks — Mon, 26 Aug 2024 06:58:21 GMT

Re: Alternative for Data Lake

Brandon_Wertz — Wed, 03 Jul 2024 14:10:56 GMT

@paIoaItonetworks wrote:

Are there any cheaper alternatives of Data Lake? Let's say, our retention is 6 months and we'd like to store incidents and all related information for 1-2 more years somewhere else. Has anyone used/using something like that, e.g. Amazon S3

I'm not 100% sure if they're the same. You talk about CDL logs, then also INCs. If it's just CDL log retention you can log forward out of CDL to a different destination. I'm not sure about INCs though, I assume you mean something from XSOAR or XSIAM? If it's either of these 2 platforms I'm not sure about the export ability from these consoles.

Re: Alternative for Data Lake

BPry — Wed, 03 Jul 2024 19:43:03 GMT

@paIoaItonetworks,

If you wanted to store incident details that's probably something better stored via capturing all of the incident details via the API and copying it elsewhere for longer retention. As an example you could utilize the ExportIncidentsToCSV script that is readily available that you could utilize as a base and store the CSV of the incident elsewhere to allow for longer retention periods in a cheaper solution.

Re: Alternative for Data Lake

paIoaItonetworks — Thu, 04 Jul 2024 13:03:28 GMT

I'm talking about Cortex XDR, I didn't mention it, but there's a tag. The new requirement is to not export incidents, logs etc to the cloud because it'd be too expensive and also that place, where we'd like to store all data related to incidents should have search engine.