11-24-2021 09:49 AM
Here is the set up. Palo FW HA pairs send logs to Panorama and Log Collectors. Log Collectors send logs to long term archival (LTA) such as LogRhythm.
Here is the issue, long term storage is not seeing the latest logs. I guess what I don't understand is the timing. When/how often are logs sent from the FW's to Panorama/Loggers and then when/how often do the Loggers send them off to LTA?
I get that logs may roll on Panorama based on disk size, but was under the impression that latest logs should be at least reasonably close if I search for a recent for events on Panorama traffic logs and LTA.
11-25-2021 04:37 PM
Thank you for posting question @RobertShawver
Based on my experience, the logs from Firewall to Panorama are sent almost instantly. Under normal circumstances there should not be a significant time gap between logs on Firewall and Panorama. In my case, I can see logs in Panorama within a few minutes. If you see this issue, could you check the output on log collector from this command:
debug log-collector log-collection-stats show incoming-logs
If you see any value than 0 under: Fails, this might indicate an issue.
If you have multiple log collectors ensure that there is latency less than 10ms between each log collector. Here is corresponding KB: https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000CmUnCAK
Could you also check the logging rate and health of Elastic Search:
show log-collector-es-cluster health
show log-collector detail | match logs
Could you also make sure that Firewall and Panorama/Log Collector are using the same time zone?
Regarding syslog exporting from Panorama to 3d party server, I can see some time gap which varies through out the day. Likely depending on load. I have not found any good way to troubleshoot /debug it to narrow down a root cause. Every time, I saw that server does not get any data, i took a pcap on log collector to confirm it is being sent by log collector or not.
Kind Regards
Pavel
11-25-2021 04:37 PM
Thank you for posting question @RobertShawver
Based on my experience, the logs from Firewall to Panorama are sent almost instantly. Under normal circumstances there should not be a significant time gap between logs on Firewall and Panorama. In my case, I can see logs in Panorama within a few minutes. If you see this issue, could you check the output on log collector from this command:
debug log-collector log-collection-stats show incoming-logs
If you see any value than 0 under: Fails, this might indicate an issue.
If you have multiple log collectors ensure that there is latency less than 10ms between each log collector. Here is corresponding KB: https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000CmUnCAK
Could you also check the logging rate and health of Elastic Search:
show log-collector-es-cluster health
show log-collector detail | match logs
Could you also make sure that Firewall and Panorama/Log Collector are using the same time zone?
Regarding syslog exporting from Panorama to 3d party server, I can see some time gap which varies through out the day. Likely depending on load. I have not found any good way to troubleshoot /debug it to narrow down a root cause. Every time, I saw that server does not get any data, i took a pcap on log collector to confirm it is being sent by log collector or not.
Kind Regards
Pavel
Click Accept as Solution to acknowledge that the answer to your question has been provided.
The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!
These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!
The LIVEcommunity thanks you for your participation!
