Panorama can't show up traffic log

[JeffKim]

We have two panorama and HA.

Today our active panorama was broken caused by VM datastore issue .

Secondary Panorama taken active role , but We can't see any traffic log at the Monitor tab.

[JeffKim]

jkim3@lvnv-now-mgt-pan(secondary-active)> show log-collector all

Serial CID Hostname Connected Config Status SW Version IPv4 - IPv6
---------------------------------------------------------------------------------------------------------
000710xxxxx  2 esca-now-mgt-pan no Out of Sync 10.1.3-h1 10.53.9.54 - unknown

Redistribution status: none
Last commit-all: none, current ring version 3
SearchEngine status: Active
md5sum updated at 2022/01/13 14:01:27

Certificate Status:
Certificate subject Name: 0e070ba7-7aec-4663-ab53-7a2ea571fec6
Certificate expiry at:
Connected at:
Custom certificate Used:
Last masterkey push status: Unknown
Last masterkey push timestamp: none

Serial CID Hostname Connected Config Status SW Version IPv4 - IPv6
---------------------------------------------------------------------------------------------------------
00071xxxxx 5 lvnv-now-mgt-pan yes In Sync 10.1.3-h1 10.93.9.54 - unknown

Redistribution status: none
Last commit-all: commit succeeded, current ring version 3
SearchEngine status: Inactive
md5sum updated at ?

Certificate Status:
Certificate subject Name:
Certificate expiry at: none
Connected at: none
Custom certificate Used: no
Last masterkey push status: Unknown
Last masterkey push timestamp: none

[PavelK]

Thank you for the post @JeffKim

Are your managed Firewalls configured to use both Panorama's IP addresses under: Device > Setup > Panorama Settings > Panorama Servers ?

Is your secondary Panorama using the same time/time zone as managed Firewalls?

Could you confirm that logs are coming by: show logging-status device <Firewall S/N>

Kind Regards

Pavel

[JeffKim]

1. Yes

2.Yes

3.

Log Collector
'Log Collection log forwarding agent' is active and connected to 10.93.9.54

config Not Available Not Available 0 0 0
system 2022/01/13 14:28:19 2022/01/13 16:46:25 2971756 2971746 4015
threat 2022/01/13 14:27:25 2022/01/13 16:46:25 1766377018 1766377008 2448368
traffic 2022/01/13 14:36:52 2022/01/13 16:48:07 1534220641 1534210553 11715931
hipmatch Not Available Not Available 0 0 0
gtp-tunnel Not Available Not Available 0 0 0
userid 2022/01/13 14:21:37 2022/01/13 16:46:25 169621989 169621979 569726
iptag Not Available Not Available 0 0 0
auth Not Available Not Available 0 0 0
sctp Not Available Not Available 0 0 0
globalprotect 2022/01/13 14:20:13 2022/01/13 16:46:25 3114386 3114376 24217

~
~

[JeffKim]

Working with Paloalto support and we found the root cause is ElasticSearch doesn't work between log-collectors.

We couldn't fixed the issue and need to clarify it .

[2ac3d44d4e6a1ff84088e56a25275be4]

What was the solution? We probably have the same problem.

[JeffKim]

when we had the issue , there was no other way to recover faulty panorama as soon as possible.

the minimum master node is n/2 + 1.
we need 3 log collectors to avoid this issue.

we still use two log collectors.

[PavelK]

Sorry to step in, the requirement for n/2 + 1 was introduced with PAN-OS 10 and later:

https://docs.paloaltonetworks.com/pan-os/10-0/pan-os-release-notes/pan-os-10-0-release-information/c...

The behavior with one faulty log collector is therefore expected.

Kind Regards

Pavel