5200 upgrade from 8.1.5 to 9.0.6 and HA2 won't come up

[nextgenhappines]

Perform an upgrade from 8.1.5 directly to 9.0.6 yesterday on A/P pair of 5250.  The HA2 link won't come up on 9.0.6

This is from TAC,

Check the pan_dha.log in dp0-log and dp1-log for this error,

I was able to see the following errors that explain as to why HA2 would not come up:

pan_dha.log
++++++++++++++
Error:  pan_dha_config_connection_load(pan_dha_config.c:483): invalid peer ha2-ip addres
++++++++++++++

What might have happened, is that after the reboot once 9.0.6 was installed, the full configuration might not have been validated, including the HA2 config,  for one internal reason or another. Physically, the interface port was healthy throughout the upgrade process, but it looks like it was an internal configuration issue. I myself have experienced issues that a commit after upgrading would fix issues that arose after the upgrade reboot.

Hope this helps others.

E

[OtakarKlier]

Hello,

Check the config on both PAN's and make sure they are correct. If still not working, try changing the HA2 config so you know its not correct, commit the changes and then change them back.

Just some thoughts.

[nextgenhappines]

Hello @OtakarKlier ,

The issue is not related to HA2 config of the firewall, as TAC explained, something went wrong in auto commit. 

Suggestion 1 , commit force.

Suggestion 2, reboot the firewall.

I rolled back both firewall to 8.1.5 and upgraded to 8.1.13, then to 9.0.0, and then upgraded to 9.0.6.  HA2 link stay up.   Just want to share my experience to save others time and pain to do many many upgrades.

E

[BPry]

@nextgenhappines,

So you went from a non-recommended upgrade path to mirroring the recommended upgrade path and your issues went away, funny how that works out 😉 ; - )

In all seriousness, this is exactly why I stress following the actual recommended upgrade path as much as I do. 95% of the time it won't matter and everything will work perfectly fine, but then 5% of the time something breaks and can cause an outage. It's better that you follow the proper process and need a bit more time for the maintenance window than have an issue and cause an unexpected outage or unexpected extended maintenance. 

[nextgenhappines]

@BPry 

Change window is difficult to request.  Don't want to fill out additional paper works to explain what happened.    "Recommended" upgrade. takes 3 times long.   It takes 35 minutes for a firewall to reboot (I do miss the good old days work on screenos) and I have to do 6 upgrades instead of 2..  hmm.. that is a hard sales for me.  

If I knew the TAC tricks,  1) try commit force, or 2) reboot the firewall on the version that you wanted.  That would have save so much time as well.   Just want to share my experience and hope to save others; time and stress.