Multiple remote site firewall commit errors/failures after Panorama 10.2 upgrade

[NeonNetSec]

Hey all,
Recently step-upgraded Panorama from 9.1.14-h4 to 10.2.4-h4. No issues upgrading Panorama. This panorama manages 180+ remote site firewalls. Ever since the upgrade we have *a few* remote site firewalls that are failing to commit properly in 2 ways:

1. commit failures related to particular configuration items, mostly specific interfaces and dhcp configurations, that should work, are present on the device, and have worked prior to upgrade (example below)


2. if we do some exhaustive troubleshooting, try to remove and re-import/connect the devices to panorama then commits will succeed again without error but most Template settings, like Network tab, will NOT propagate down to the remote site firewall

We've worked w/ Palo TAC a bit on this. Originally we found that some of the problematic firewalls were on 9.1.x versions so there were configuration transforms happening that could have been problematic, but upgrading these remote site firewall to 10.2.x did not resolve the issue either.

Only other interesting item found is error messages related to Xpath error : invalid expression for a particular interface:

Has anyone seen this before or have any thoughts? Thanks.

[felipeorozco]

I have a question, Do you validate if was have same errors before the upgrade? I think same issue but I don't know it's necessary apply commit (after upgrade apply commit) for the all users admin and appears the commit successfully.

[TomYoung]

Hi @NeonNetSec ,

I have not seen this before.

To answer #2, to push template values to a newly imported NGFW, you need to select Force Template Values.  I would definitely have Automated Commit Recovery enabled before this as it will override your network values.

With regard to #1, it looks like most of your errors are Network related.  Open up the GUI, override and check the config, then save.  Sometimes this will fix the syntax error in the XML.  Do this everywhere you get an error.  You could also try looking at the config in the CLI.  Sometimes the syntax errors are easy to spot and fix there.

I try not to keep my NGFWs and Panorama versions too far apart.  A bigger difference between versions means a bigger chance of a commit error.  I upgrade Panorama and then upgrade NGFWs for each version.  I know this is not much help for you now.

Thanks,

Tom

[NeonNetSec]

Neither solution helped, unfortunately. Issue (commit failures) and errors persist as before.

[TripleThreat]

@NeonNetSec did you ever find a resolution for this?  Have a similar situation with panorama and devices at 11.02.