About hburton

hburton · ‎04-20-2020

Both answers have been helpful. Thank you. I'm mostly interested in the Palo Alto logs though. Specifically, the traffic logs. The traffic logs are 99+% of what I'm currently sending to log management, and the bulk of the load that I would like to reduce. Currently, every security policy is configured to log and forward those logs to log management. I feel like that is probably more than what we realistically need, and I'm curious what other people are keeping.

hburton · ‎04-20-2020

Thanks Patrick. I appreciate your response. I haven't read through all of these guidelines, because I've only been reading for about 35 years, but I have googled both PCI and HIPAA log retention policies and they are about as vague as standards/guidelines tend to be. They tend to state how long to maintain logs, and how to store them pretty concisely. However, for which logs to store, they say "user logins, ..., firewalls logs." What I'm hoping to get is an idea which logs people tend to care about, so that I can whittle down what I'm sending to log management. My log stream is around 5Mb/s right now. The Palo Alto firewall isn't the only system logging to it, but it's one of just a few, and the others log a handful of entries per minute, so that 5Mb/s is almost entirely my firewall logs. This is also the reason that we aren't logging these to Splunk. The firewalls alone would chew up our total ingress budget.

hburton · ‎04-19-2020

Hello. I'm looking for suggestions for what to log out to ELK. I understand that this will be different for every environment and should be based on existing log retention policies etc. However, we have no such policies, and I am currently logging far too much, I'm sure. In order to get traffic visibility > 2 days, I rolled up an ELK stack on a server with 17TB storage, and pointed all system, traffic, threat, and config logs to it. I setup every security policy to forward to this collector, and the traffic logs, in just a few days, have grown to over 129GB. The space is there to support that. However, Elasticsearch isn't happy and extracting the data is extremely slow. It could be that I need to tune the collector, but I think more likely, I need to be more realistic/logical about what I'm logging out. I would appreciate any general suggestions. For example, is logging blocked traffic useful enough to justify it? I can see where it may be useful in an audit trail, but if it was blocked/dropped, how much should we care about it?

Member Since	‎08-01-2016 03:56 PM
Date Last Visited	‎04-20-2020 05:08 PM
Posts	3

Online Status	Offline
Date Last Visited	‎04-20-2020 05:08 PM

About hburton

Re: Log Retention Policies and Best Practices

Re: Log Retention Policies and Best Practices

Log Retention Policies and Best Practices

Re: Log Retention Policies and Best Practices

Re: Log Retention Policies and Best Practices

Log Retention Policies and Best Practices