I have a active-standby panorama cluster version 8.1.17 that manages about 40 firewalls. The active-cluster panorama is also a log collector-group.
20 firewalls send traffic/threat/URL logs to active panorama and the other 20 firewalls send traffic/threat/URL logs to the standby panorama. From there, I configure panorama to forward these logs to syslog splunk. I have PAN TAC support look at the configuration and they confirm the setup is good.
Here is the issue. When I use the command "less mp-log syslog-ng.log", I can see the drop increment from panorama to the syslog splunk every 30 minutes or so. The counter is measured every ten minutes. On the syslog Splunk side, they confirmed that the traffic never arrived in tcpdump (syslog is clear text so we can decode the missing logs).
I've opened a ticket with PAN support and waiting to hear back from them but it is currently with the first tier level TAC support so not much hope so far.
Why would panorama stop forwarding log to external syslog splunk? Has anyone seen this issue before?
@reaper: What is the command do you recommend? I am using "debug log-collector log-collection-stats show log-forwarding-stats | match syslog" and I am seeing this:
syslog enqueued count: 3260998077
syslog sent count: 3260769863
syslog dropped count: 422974378
syslog Queue depth: 0
What do you mean by the log rate is too high? My panorama is running in AWS with the biggest available EC2 available.
I use a python to log into Panorama every 5 seconds and run these three commands and pipe them into an ascii file:
show clock | match GMT
Sat Mar 20 07:40:04 GMT 2021
> debug log-collector log-collection-stats show log-forwarding-stats | match "syslog dropped count"
syslog dropped count: 23026208
> debug log-collector log-collection-stats show incoming-logs | match "Incoming"
Incoming log rate = 1389.45
Then I use grep and awk to find out if the count the diff in the dropped count and Incoming log rate base on the timestamp.
PAN TAC support also has something similar but they run it in Teraterm for Windows. Real engineers use Linux 🙂
Click Accept as Solution to acknowledge that the answer to your question has been provided.
The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!
These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!
The LIVEcommunity thanks you for your participation!