How do I use the query feature of the CLI show log traffic command?

[jsilvia]

I'm trying to perform a complex log export operation from the command line, as the web GUI seems to be drastically underpowered and slow to respond (hours to see, longer to export in CSV), and from the command line I can't perform queries using terms to extract data in a date range, or figure out how to do that.

Ideally, I'd love to execure the query:

"( port.dst eq 80 ) and ( zone.dst eq inet ) and (packets eq 1) and (time_generated geq '2011/10/21 00:00:00') and (time_generated leq '2011/10/22 00:00:00')"

and save the output in a CSV file that I can download via SCP.

There is an option for "query" show in the CLI guide - if this permits me to an actual web query, that would be ideal, however, I cannot seem to implement or even find any examples of how the query feature is used.

Please note that this underscores one of my major complaints with this product - the documentation tends to follow a very Linux "Man Page" approach - the documentation is accurate in it's presentation of the commands, but provides no examples of syntax to help people actually use the features.

Am I on the right track?

[skrall]

What software version are you running? My 4.0 system seems to be lacking a "query" command. We do have a "show log" command but it displays on the CLI and does not export to CSV.

I have a security policy named "SKRALL-test1"

Below is a query based on that security rule in the threat logs for a range of dates.

skrall@Corp-FCS-vwire> show log threat rule equal SKRALL-test1 start-time equal 2011/10/21@15:14:45 end-time equal 2011/10/31@12:00:00
Time                App             From            Src Port   Source
Rule                Action          To              Dst Port   Destination
Severity            Src User        Dst User        Threat
===============================================================================
2011/10/21 15:14:45 web-browsing    corphstuntrust  80        98.137.88.36
SKRALL-test1        alert           corphsttrust    59899     10.16.0.26
info                                paloaltonetwork HTTP response data URI scheme evasion attempt(33127)

Can you do a "show system resources" and see if any of the processes are consuming 700M of memory or greater? Restarting these processes may speed up the box and make the GUI more usable.

Steve Krall

[jsilvia]

I'm on version 4.0.5 - connected to my Panorama server.

+ action         action

+ app            app

+ csv-output     csv-output

+ direction      direction

+ dport          dport

+ dst            dst

+ dstuser        dstuser

+ end-time       end-time

+ from           from

+ query          query

+ receive_time   receive_time

+ rule           rule

+ sport          sport

+ src            src

+ srcuser        srcuser

+ start-time     start-time

+ to             to 

|              Pipe through a command 

<Enter>        Finish input
> show log traffic

Understand that I'm trying to pull as much log history as I can out of the box so that I can perform external analysis - PA's log analysis doesn't provide what I need - and in my Check Point past I was able to export the desired columns, then process in excel and visualize in ggobi - this can help find the needles hiding in the haystack of the network - in my case, infected needles that are performing outbound infection sweeps on port 80 - by searching for one packet sessions I can get these - however, by the time I can pull these from the GUI, I'll probably loose most of todays logs just trying to get to them!

[bpappas]

Another option you might consider is setting up a custom vulnerability signature that looks for port 80 sweeps and sends and alert email for each infected host that is discovered.

This may be more effective for you than trying to perform log exports and data analysis.

-Benjamin

[jsilvia]

I seem to have dug it out with some outside vendor help - turns out the query language is a query without parenthesis.

I was ultimately able to perform this:

scp export log traffic query "packets eq 1 and zone.dst eq inet" to user@hiddenip:filename.csv end-time equal 2011/10/22@00:00:00 start-time equal 2011/10/21@00:00:00

[jsilvia]

My stuff draws prettier pictures though!

I'll look into making the signature too.

Thanks!