Log collectors Incoming logs per second capacity

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Log collectors Incoming logs per second capacity

L0 Member

In a log collector group, we have two log collectors and redundancy enabled option is turned on. This results in the incoming logs per second capacity reduced to half. 

However, wanted to confirm that in order to use the incoming logs capacity of both the log collectors optimally- should we change the preference list on the managed devices so that some managed devices have log collector 1 on the top of the preference list and some managed devices have log collector 2 on the top of the preference list. So that both the log collectors are used by the managed devices to send the logs. I understand that the first log collector in the preference list will always receive the logs from the firewall and then the hash algorithm will decide which log collector should write it and then a copy is sent to the second log collector - but initially the first log collector in the preference list will receive the logs from the firewall. If the same log collector is used on top of the preference list for all the managed devices, then only one log collector out of two in the log collector group will receive the logs from all the devices which could result in waste of some capacity of log collector 2 and bottle neck for log collector 1? appreciate some insight into this? thank you.

 

1 REPLY 1

Cyber Elite
Cyber Elite

from a connectivity bottleneck perspective it makes sense to spread connections to both collectors by setting one group to prefer collector1 and the other collector2

As you mention this does not 'improve' overall logging rate when you have redundancy enabled as the collectors will be writing logs amongst themselves so your top ingestion rate is halved on each collector. I do recommend distributing inbound connections so any peaks can be distributed and wont congest a single collector 

distributing the connections should lead to slightly better performance/resilience as each collector has less inbound logs to process so has more cycles to account for redistribution

Tom Piens
PANgurus - Strata specialist; config reviews, policy optimization
  • 1523 Views
  • 1 replies
  • 0 Likes
Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!