Someone school me on Syslog and Panorama

[drewdown]

I have Panorama managing roughly 10 firewalls.  I have logging setup on those FWs to send to Panorama, email and send syslog to a 3rd party host.  I struggled immensely trying to get everything configured correctly on PANORAMA and the FWs themselves to even send logs.   The syslog server is getting the logs but they are all coming from PANORAMA vs the local FWs and I can't seem to figure out why.  

I have a syslog server profile configured on PANORMA and pushed/created the same on every firewall. I don't have templates for every fw (why I don't know - dont get me started on templates) so I manually created a matching syslog server profile on all them because if I didn't the push would complain about that profile not existing on them. 

I also have a Log Forwarding object on all of them that includes sending syslog to the syslog server profile:

Now when the logs reach the syslog server they are coming from the PANORAMA and not the fws themselves.  I thought having the Forward Method option of "Panorama/Logging Service" unchecked would mean the FWs themselves would send the logs but that doesn't seem to be the case.  

Can someone explain what that option entails and the best way to go about this?  I would think I would want the fws to send syslogs themselves so that the fw IPs would be seen as the sender but I can't get them to do that.  They are in the actual log but the reporting IP is PANORAMA. I ran a pcap for 514 on one of the firewalls and it shows sending them to the syslog server but the security team reports that no logs come from the individual fw IPs.  

admin@3020(active)>
tcp        0      0 10.100.1.40:44984           10.129.2.99:514            ESTABLISHED
admin@3020(active)> tcpdump filter "port 514" snaplen 0
Press Ctrl-C to stop capturing

tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
^C2 packets captured
4 packets received by filter
0 packets dropped by kernel
admin@3020(active)> view-pcap mgmt-pcap mgmt.pcap
10:17:14.544409 IP 3020.domain4u.com.44984 > ec222.domain4u.com.shell: . ack 1879714823 win 115
10:17:14.560967 IP ec222.domain4u.com.shell > 3020.domain4u.com.44984: . ack 1 win 211

If I run the same commands from the PANORAMA I see this:

admin@m100> tcpdump filter "port 514" snaplen 0
Press Ctrl-C to stop capturing

tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
^C6 packets captured
17 packets received by filter
0 packets dropped by kernel
admin@pan-m100-cs> view-pcap mgmt-pcap mgmt.pcap
10:49:56.794253 IP panorama.domain4me.com.53305 > ec222.domain4u.com.syslog: SYSLOG user.info, length: 573
10:49:56.794299 IP panorama.domain4me.com.53305 > ec222.domain4u.com.syslog: SYSLOG user.info, length: 573
10:49:56.794348 IP panorama.domain4me.com.53305 > ec222.domain4u.com.syslog: SYSLOG user.info, length: 573
10:49:56.796670 IP panorama.domain4me.com.53305 > ec222.domain4u.com.syslog: SYSLOG user.info, length: 572
10:49:56.796705 IP panorama.domain4me.com.53305 > ec222.domain4u.com.syslog: SYSLOG user.info, length: 572
10:49:56.796756 IP panorama.domain4me.com.53305 > ec222.domain4u.com.syslog: SYSLOG user.info, length: 572

And I do have PANORAMA > Syslog Server Profile configured as well, so is that what is sending the logs? If so what are the local firewalls doing if anything at all?  I just can't seem to wrap my head around the right way to do this.  Meaning how do you do this from PANORAMA if you don't have Templates for every device?  Note all of my polices originate from PANORAMA and they all have the same logging configuration so why are am I no seeing event logs from the firewalls when the cli shows its sending them? 

[S.Cantwell]

Hello there.

Generally speaking, it is considered inefficient to duplicate logs from FWs to Panorama AND the Syslog server profile.
It is best to send ALL logs to the Panorama from the FWs, and then a single stream of syslog messages are sent from Panorama TO the syslog server.  You are doing it correctly.

As for templates and Panorama. it sounds you have some info in Panorama and some stuff set as local.

You do better importing your current local configs into the Panorama

https://docs.paloaltonetworks.com/panorama/9-1/panorama-admin/manage-firewalls/transition-a-firewall...

This may help you, as an initial step, to get all FWs into the Panorama.