Panorama Logging Behind

[Nicole.Perez]

Hi all,

The issue that I am having is that all my firewall logs in Panorama are behind in time.  I have tried multiple KB articles and support basically went through all the KB articles I found such as restarting the management service, stopping and starting the logging service, and even rebooted our Panorama.  After rebooting our Panorama I started to see logs in the GUI of Panorama from a Firewall that I didn't see before.  I went to the firewalls and verified that I am connected to our Panorama instance.  The log collector on my Panorama is also in sync.  I am not getting much traction with support at this point and need these to view the logs because it is annoying to have to go into each firewall to troubleshoot what is going on at the moment that I push a rule.  Any ideas?

My management CPU seems to be at 40-63 percent and right now I am stuck on what to look at.  Also everything was working fine until we had tunnel issues with our Data Center because they were having routing issues (we were down for like 10 days and no workaround for this... crazy right).  4 days before their change the logging broke on our Panorama.  Up until that point everything was working fine and now Panorama doesn't seem to know how to recover itself.  I do have a support case open but again very slow movement and nothing much that they are doing at this point other than the KB articles that I have found.

[Nicole.Perez]

Just to note, my Panorama is still working and everything is in sync and connected without any issues.  I can still push rules and everything with little to no effect.  

[Mick_Ball · Accepted solution]

could it simply be that the firewalls are trying to catch up with log transfer.  we had a similar issue when panorama was out of action for about a week.

when it was back up and running the same thing happened but it did catch up eventually.

[Nicole.Perez]

Mick, 

So I suspect we had a huge amount of data that was queued.  I think Panorama did need reboot as something was stuck and not displaying one of our firewalls logs even though logs were being sent because after the reboot I could see the logs at least.  It has to be a process that is associated perhaps with the web GUI which I thought the management service would take care of, either way a roboot fixed the logs we were not seeing.  It just took about 2 plus days for it get back to normal.  I checked it last night and 4 of my firewalls were about caught up and the 1 firewall which was almost 9 hours behind at the time started to catch up faster.  This morning I checked it and everything is back where it should be and my CPU is resting at about 3 percent (this expectation is normal).  I thought the queues would have caught up faster than it had but just glad everything is back to normal.  Thanks for your reply.