No metrics showing up in a syslog analyser node

[borising]

Hi,

I followed this post the other day and have been forwarding logs from my firewall for 2 days now, but without any hits, so I am wondering if I have done something wrong? I can see in a tcpdump dump on the minemeld server, that logs are received on port 13514/TCP. Also, the logs that are sent to minemeld are dropped traffic from an EDL, so the indicators should be present.

https://live.paloaltonetworks.com/t5/MineMeld-Articles/Correlating-PAN-OS-syslog-with-indicators/ta-...

I am using the stdlib.localSyslog prototype, as I just want to know whits lists I hit.

Any ideas on how to troubleshoot this?

I'm using:

PAN-OS 8.0.3-h4

Minemeld v 0.9.40

[lmori]

Hi @borising,

please, could you attach a screenshots of the stats of the syslog miner node ? (Nodes > <syslog miner nodes> > Stats tab on the right)

Thanks,

luigi

[borising]

Hi Luigi,

I have attached the screenshot, and a screenshot of the sources tab and how it looks on the Nodes page.

Regards,

Bo

[lmori]

Hi @borising,

could you double check the logs rsyslog in /var/log/rsyslog to see if there are errors in loading the rabbitmq modules ?

luigi

[borising]

Hi Luigi,

There are only 2 rsyslog.log files, which I have cat'ed below. Rsyslogd is running, as you can see.

xxx@minemeld01:/var/log$ cat rsyslog.log
xxx@minemeld01:/var/log$ cat rsyslog.log.1
Jul  2 21:44:32 minemeld01 rsyslogd: [origin software="rsyslogd" swVersion="8.17.0" x-pid="5101" x-info="http://www.rsyslog.com"] start
Jul  2 21:44:32 minemeld01 rsyslogd: rsyslogd's groupid changed to 104
Jul  2 21:44:32 minemeld01 rsyslogd: rsyslogd's userid changed to 101
Jul  2 21:45:14 minemeld01 rsyslogd: [origin software="rsyslogd" swVersion="8.17.0" x-pid="5101" x-info="http://www.rsyslog.com"] exiting on signal 15.
Jul  2 21:45:14 minemeld01 rsyslogd: [origin software="rsyslogd" swVersion="8.17.0" x-pid="5420" x-info="http://www.rsyslog.com"] start
Jul  2 21:45:14 minemeld01 rsyslogd: rsyslogd's groupid changed to 104
Jul  2 21:45:14 minemeld01 rsyslogd: rsyslogd's userid changed to 101
Jul  4 06:53:12 minemeld01 rsyslogd: [origin software="rsyslogd" swVersion="8.17.0" x-pid="5420" x-info="http://www.rsyslog.com"] rsyslogd was HUPed
xxx@minemeld01:/var/log$ ps xau | grep rsyslogd
xxx       430  0.0  0.1  11764  1960 pts/1    S+   17:08   0:00 grep --color=auto rsyslogd
syslog    5420  0.0  0.0 378196   708 ?        Ssl  Jul02   0:03 rsyslogd

And the rsyslog version:

xxx@minemeld01:/var/log$ dpkg -l | grep rsyslog
ii  rsyslog                            8.17.0-0adiscon2trusty1                    amd64        a rocket-fast system for log processing
ii  rsyslog-minemeld                   8.16-0                                     amd64        minemeld modules for rsyslog
ii  rsyslog-mmnormalize                8.17.0-0adiscon2trusty1                    amd64        The rsyslog-mmnormalize package provides log normalization

[lmori]

Hi @borising,

could you check the output of this command:

sudo rabbimq_ctl list_queues | grep -i syslog

[borising]

Here you go:

xxx@minemeld01:~$ sudo rabbitmqctl list_queues | grep -i syslog
localSyslog:rpc 0
mbus:directslave:localSyslog:rpc        0
mbus:slave:localSyslog:rpc      0
xxx@minemeld01:~$

[borising]

Hi @lmori,

Any luck of finding out what causes this problem?

Thanks!

[lmori]

Hi @borising,

I am adding new counters in syslog matcher to help troubleshooting this, they will make into the next release. If you are in a hurry drop me an email at lmori@paloaltonetworks.com and we can have a webmeeting to debug this together.

Luigi

[borising]

Hi @lmori,

When do you think the next release will be available?

Regards,

Bo

[lmori]

Hi @borising,

should be in a week.

Luigi

[borising]

Hi @lmori,

Then I'll just wait, and update this message, when I have tested further 🙂

Regards,

Bo

[lmori]

Hi @borising,

which PAN-OS version are you using ?

Thanks,

luigi

[borising]

Hi @lmori,

I started out with 8.0.3-h4, but have upgraded to 8.0.4 along the way.

Regards,

Bo

[borising]

Hi again @lmori

I just did a debug session of when rsyslogd receives a syslog message, as I had a suspicion that it might be an issue parsing the 8.x logs.

Though I can't really get much out of it, you might 🙂 

So if you think you could use the output for further debugging, then let me know.

Regards,

Bo Rising