Hi @mb_equate ,
Based on your explanation my first guess is that IPs were probably configured locally on the firewall while the template were empty.
During device-group commit and push, someone has clicked on "Force template values" which tells the firewall to accept what is configured in the template.
I would start by comparing commit revisions - locally on the firewall and Panorama, mainly the revision before and after the device-group commit. This will show the changes that were applied with this commit and I would expect to see the removal of the IPs.
"...Technically this is not a valid configuration as routes rely on next hops in the interface subnets..."
This is not entierly try. I cannot test at the moment, but I am alsmost certain that you can configure static route with next-hop IP and interface, without assigning IP to that interface. You defintely will see commit warning - like those you have pasted above, but those are warning and not errors.
Little background - Device-Groups and Templates have different logic for resolving conflicting config. With device-group (just for contrast) you cannot create two rules (locally and pushed by panorama) with exact same name, you can override address object locally, but not a rule. This is on purpose to ensure the order of rule pushed by Panorama
Template on other hand allow without issues to push different values for exact same config that is being set locally. If you have device or network config that is being pushed by panorama and configured locally on the FW, FW will always prefer the local config. The tricky part is that pushing from Panorama wouldn't indicate in any way if the settings from the Template are indeed used or there is local config. The only way to apply Panorama config is to either "force template values" when pushing from Panorama, or clicking on "revert" for each setting locally on the firewall (which will require local commit).
PanOS 11.0.1 is fairly new any who know what weird bug you may face, but before chaising bugs I would suggest to eliminate obvious reasons by comparing config revisions around the time of the incident.
... View more
