Suggestions for Splunk Search/Report

[davehaertel]

We have the Palo Alto app for Splunk logging everything correctly, I'm basically looking for suggestions on solid search reports to eliminate most of the noise.  I've been combing through some of the Splunk forum posts but nothing jumping out at me so far. Thanks in advance.

[BPry · Accepted solution]

@davehaertel,

I would really recommend diving into creating custom datasets and filtering for anything that you really care about that way, this also allows you to schedule the reports. For example I have a dataset configured to look at my threat logs and gather all of action and client_ip information so that I can quickly see if there is any single IP that is generating a large amount of threats or DoS policy alerts. 

[davehaertel]

Thanks, I haven't created any datasets, just done specific reports based on the criteria I've been interested in seeing regularly.  I'll have a look at the dataset creation portion and see what I need to do.  

My biggest problem is meshing the palo alto on the perimeter and the ASA(s) that operate the DMZ.  Between the 2 of them they generate an enormous amount of material, especially with the multiple entries that VPN access creates.  It's really pretty overwhelming trying to figure out what is noise and what is something to be concerned about.  

[BPry]

@davehaertel,

In my experiance, and mind you I'm no expert at Splunk, Splunk is a great tool if you know what you are looking for and that's about it. Just reviewing the logs you'll find a ton of stuff that doesn't really hold any value or doesn't really matter. 

[davehaertel]

I agree completely.  If you aren't careful you can drown in meaningless data, looking for that tiny little bit that actually indicates that there's a hole that needs to be plugged lol.  

I spend an hour every morning just going through the windows logs and I think I have that finally narrowed down to just the basic stuff that I'm concerned about, but moving forward to the Cisco and Palo Alto additions, I'm going to easily have 2 hours a day just going through my checklists.  

Oh well time to ask for an assistant LOL!