log forwarding to CDL is generating high traffic volume

[Carracido]

Dear community!

We are sending logs to cortex data lake and we noticed high traffic volume for the sessions concerning log forwarding, with peaks up to 200GB of data sent. 

Do you know if this volume of traffic can be normal?

Also, is there any documentation on how logs are being sent to CDL or how would you troubleshoot this issue?

Thank you in advance!

[OtakarKlier]

Hello,

Check your log sources for what they are sending. If its the Firewalls, make sure you are only logging at session end. Also you can filter what is sent to the data lake, if you wish to limit the data, but its a data lake so I say the more the merrier.

Regards,

[LAYER_8]

If you're a Cortex Pro customer as well, the usage by Data Lake will be high. Just QoS the app on the box, give it a set bandwidth if it's causing troubles. (docs)

[Remo]

@Bearden 

Sorry for this question, but could you explain thia again? I don't understand what you are trying to tell us.

[Remo]

Hi @Carracido 

Over what time do you see these 200 GB peaks or also where? Is this from the ACC tab on the firewall? In that case you might see the end of a session which was open for days or even weeks and these 200 GB were the result of this very long session.

[Carracido]

Yes, those are sessions that stay alive for over a week.

In some sessions I see also that bytes received from CDL are almost the same as bytes sent from the firewall. Is this because of log acknowledgment? If yes, how possible is so big the ack?

Thank you!