Where are my XDR logs stored?

[DannyMulheran]

Can anybody explain the storage method used by XDR for external data (and internal for that matter) better than the Beacon training and XDR admin guide. I have AWS S3 logs which I can see under Dataset management with the dataset name 'amazon_aws_raw' (imported using the Amazon S3 integration). I was under the impression these are stored in my data lake but when I access my data lake via the apps gateway I see nothing in it.

Does the XDR instance have its own Datalake that we cannot see from the apps portal?

Is the Datalake provided by the Pro per TB license the one shown in the apps portal?

In the apps portal I have a datalake. From the CSP under assets > cloud services I can see my XDR tenant is associated with my Datalake. Is this my Pro per TB datalake, or an additional datalake?

I have reviewed the architecture diagram in the documentation which appears to make sense until I try to find my logs using the explorer app and the datalake app.

Any additional information on the storage architecture for XDR would be very much appreciated.

[aleksandar.astardzhiev · Accepted solution]

Hi @DannyMulheran 

As you can see from the description here - https://apps.paloaltonetworks.com/marketplace/explore Explorer is intended to allow you to search only the logs that are generated by the PAN FWs that are sending logs to your Datalake instance.

There are not separate or additional datalakes, but it just that Explorer doesn't have an option/it doesn't allow you to look at the other logs apart from PAN FW/Prisma Acess.

If you want to browser/search your AWS logs you need to use the XQL queries in  XDR console and specifying the dataset you want to use.

[DannyMulheran]

Hi Astardzhiev

Many thanks for the reply and information, it is really appreciated.