Correlating PAN-OS syslog with indicators

Printer Friendly Page

MineMeld can analyze PAN-OS syslog messages and match them against live indicators. In case of match, the result of the match (full session info and full details of the indicator) can be sent to a logstash (https://www.elastic.co/products/logstash) instance for forwarding to an external database.

 

1. Message flow

The following diagram illustrates the message flow of the syslog analysis process.

syslog-schema.png

NOTE: while rsyslog is already installed and configured by the MineMeld loader on the VM, logstash is not installed by default. To save matching results in an external collector logstash must be installed and the syslog should be configured to send results to the logstash instance.

 

2. Adding a syslog analyzer node

Under CONFIG, click + to add a new node. Select prototype stdlib.localSyslog if you don't need to send results to an external logstash instance, select prototype stdlib.localSyslogToLogstash if you plan to send results to a local logstash instance. Select an aggregator as input.

 

NOTE: syslog analyzer node supports only one aggregator per type as upstream node. If you have multiple aggregators, you can use multiple syslog analysis nodes or create an additional generic aggregator just for syslog analyzer.

 

Screen Shot 2016-02-03 at 16.58.52.png

 

3. Configuring syslog forwarding on PAN-OS

Please refer to the PAN-OS Administration guide for instructions on how to configure log forwarding to a syslog server on PAN-OS (https://www.paloaltonetworks.com/documentation/70/pan-os/pan-os/monitoring/configure-log-forwarding....). By defeault the syslog server on the VM listens for PAN-OS syslog messages on port tcp/13514.

Screen Shot 2016-02-03 at 17.06.40.png

4. Checking counters on the syslog node

Under NODES, click on the syslog analyzer node. Select STATS in the left menu, and check the counters.

Screen Shot 2016-02-04 at 11.42.59.png

 

SYSLOG.PROCESSED is the number of syslog messages received from PAN-OS devices and processed. TOTAL_MATCHES is the number of matching indicators found. LOGSTASH.SENT is the number of results sent to the logstash instance (if configured).

 

5. Evaluating indicators sources

Under NODES, click on the syslog analyzer node. Select SOURCES in the left menu to see the which feeds have produced matching indicators.

Screen Shot 2016-02-04 at 11.43.14.png

 

6. Configuring logstash (optional)

If at step 2 you have chosen prototype stdlib.localSyslogToLogstash, the syslog analyzer node will send matching results to logstash on port 5514/tcp. Use the following logstash snippet to let logstash listen on port 5514 for messages from MineMeld syslog nodes.

input {
tcp {
port => 5514
host => '127.0.0.1'
codec => 'json_lines'
}
}

NOTE: logstash is not installed by default on the VM. Please refer to the following link for instructions on how to install logstash and an ELK stack on the VM or on an external VM:

- official site https://www.elastic.co/

- step by step tutorial from DigitalOcean community https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-el...

 

Kibana view of the matching results:

Screen Shot 2016-02-04 at 11.52.08.png

Comments

hi, which log that I need to select in the log forwarding profile ? is traffic log enough to match an indicator ? see screencap below.

CapturFiles-14-58-2016_12.58.05.jpg

Hi bartoq,

Traffic should be enough in most cases, but it really depends on the scenario. Both traffic and threat are supported.

 

Luigi

Is it possible to send matching results to a standard syslog server (instead of logstash)?

Hi SamKear,

you can configure logstash to forward messages to an external syslog server using the logstash syslog plugin: https://www.elastic.co/guide/en/logstash/current/plugins-outputs-syslog.html 

lmori-

 

Are domains supported for syslog matching? I see in the comments URL's are not; is this a roadmap feature by chance?

 

Regards,

Erik Yunghans

Hi Erik,

yes, domains are supported. ER minemeld-core#43 has been opened to track support of URLs (https://github.com/PaloAltoNetworks/minemeld-core/issues/43). Could you unicast me some details about use cases ?

 

Hi Luigi,

 

is it possible to correlate data from two syslog sources for user-id? So for instance receive the IP + MAC address from a DHCP server and MAC address + username from a WLAN controller to correlate the data based on the MAC address to get the IP address to username mapping?

 

Thanks

Lars

Hi @LarsAtConsigas,

no, this is not possible out of the box. You need a miner for the syslog messages of the WLAN controller and a node to correlate the two streams. This is doable with some additional development.

Hi, I'm running minemeld Version: 0.9.11-3build1 and configured syslog node as described here:

 

minemeld0.png

 

but I can't see syslog metrics and sources.

 

minemeld2.pngminemeld3.png

 

 

I've configured a PaloAlto 3020 to forward traffics logs to minemeld and the connection is established as listed bellow:

 

# netstat -ntaup|grep 13514
tcp 0 0 0.0.0.0:13514 0.0.0.0:* LISTEN 10544/rsyslogd
tcp 0 0 10.0.0.63:13514 10.0.0.11:48068 ESTABLISHED 10544/rsyslogd
tcp 0 0 10.0.0.63:13514 10.0.0.10:34568 ESTABLISHED 10544/rsyslogd
tcp6 0 0 :::13514 :::* LISTEN 10544/rsyslogd

 

Any hint on what am I missing?

@supseg

I assume since you're listening on 13514 that you have the specific minemeld rsyslog configuration?


Do you have the palo alto rule base for mmnormalize?

@GarrettMartin, hi.

 

Yes, rsyslog is configured by the default instalation. I followd this guide [1].

 

================================

$ cat 60-syslog-minemeld.conf


module(load="imtcp")
module(load="pmpanngfw")
module(load="mmnormalize")
module(load="omrabbitmq")

$template alljson,"%$!all-json%\n"

ruleset(name="pan-ngfw" parser=["rsyslog.panngfw", "rsyslog.rfc5424", "rsyslog.rfc3164"]) {
action(type="mmnormalize" rulebase="/etc/rsyslog.d/palo_alto_networks.rb" userawmsg="on")
if strlen($!unparsed-data) == 0 then {
if $!log_subtype == "url" then set $!url = $!misc;
*.* action(type="omrabbitmq"
host="localhost"
virtual_host="/"
user="guest"
password="guest"
exchange="mmeld-syslog"
routing_key=""
exchange_type="fanout"
delivery_mode="1"
auto_delete="0"
template="alljson")
}
*.* stop
}

input(type="imtcp" port="13514" ruleset="pan-ngfw")

================================

 

And yes, I have the palo_alto_networks.rb file on the rsyslog.d/

 

Best.

 

 

[1] https://live.paloaltonetworks.com/t5/MineMeld-Articles/Manually-install-MineMeld-on-Ubuntu-Server-14...

Hi Garrett,

 

Can you push the palo_alto_networks.rb file please ?

 

THX in Advanced

@AG2RSecOp 

 

Use this script and dump into a file named palo_alto_networks.rb

 

https://gist.github.com/jtschichold/87f59b99d98c8eac1da5

Is it possible to send matching results to cloud collector

Ask Questions Get Answers Join the Live Community
Version history
Revision #:
4 of 4
Last update:
‎02-04-2016 02:53 AM
Updated by:
 
Contributors