Failed active/passive HA Upgrade from 8.1.4-h2 to 9.0.3

Reply
Highlighted
L1 Bithead

Failed active/passive HA Upgrade from 8.1.4-h2 to 9.0.3

Hello community - I have a case open with support, but I am looking to see if anyone else has an idea for me while they are looking at my tech support files.

 

I attempted to upgrade an active/passive HA pair following the Palo Alto Doc. I upgraded the secondary from 8.1.4-h2 to 9.0.3 and rebooted. I am now at the step where I would suspend the primary and fail over to the secondary. But my HA pair is broken.

 

Here is what shows on the Dashboard of my primary ...

 

ModeActive-passive
LocalActive
Peer (172.17.1.11)Unknown
Running ConfigSynchronized  
App VersionUnknown
Threat VersionUnknown
Antivirus VersionUnknown
PAN-OS VersionMatch
GlobalProtect VersionUnknown
HA1Down
HA1 BackupDown
HA2Down

 

I find it odd that it shows a version match for the PAN-OS Version, when in fact, the secondary is running 9.0.3?

Highlighted
L1 Bithead

Also - my secondary now shows "HA not enabled" on the Dashboard, even though it's still configured?

Highlighted
L1 Bithead

I ended up upgrading (or in this case *downgrading*) my secondary to 9.0.0 and that worked. So then I was able to upgrade my primary to 9.0.0 without any issues. None of my VPN tunnels came back up on their own, which was a little disheartening. I had to go into the CLI and do a "test vpn ike-sa gateway <tunnel name>" on every single one of them.

 

But my upgrade is complete and functional for now.

Highlighted
Cyber Elite

@SteveBallantyne,

Just out of curiosity, what did you upgrade path actually look like? 

In this situation you should have followed the following path to meet best practices:

8.1.4-h2 -> 8.1.9 (As the latest maintenance relase) You do not need to restart (I would anyways)

8.1.9 -> 9.0.0 Install and Reboot

9.0.0 -> Target Maintenance Release (9.0.3) Install and reboot 

Highlighted
L0 Member

i too have this condition while upgrading from 8.1.9 to 9.0.3-h3.  the release notes and upgrade guide state i can upgrade directly to 9.0.3-h3 without the intermediate 9.0 step.  however, this post led me to downgrade to 9.0 from 9.0.3-h3 and re-attempt a non-impactful upgrade.  this did not fix my situation but i was able to continue my upgrade path, just with impact to user traffic.

 

upgrade to 9.0 was impactful, but after both devices upgraded, HA2 came online and synchronization was successful.  upgrade to 9.0.3-h3 from 9.0 was hitless and uneventful.

 

also, i needed to upgrade my logging server to 9.x before logs would start showing up in panorama for this set of firewalls.

Highlighted
L0 Member

Bug 128629 is what we ran into when upgrading from 8.1.6 to 9.0.5

Only the HA2 HSCI link was down for us.

setup a HA2 Backup link.

then was able to proceed without interruption.

PAN-128269

PA-5250, PA-5260, and PA-5280 firewalls with 100GB AOC cables only

Fixed an issue where after you upgraded the first peer in a high availability (HA) configuration to a PAN-OS 9.0 release, the High Speed Chassis Interconnect (HSCI) port did not come up due to an FEC mismatch until after you finished upgrading the second peer.

Highlighted
L4 Transporter

If you read the upgrade guidance "carefully", you will see it is recommended to upgrade to the latest code train release before jumping versions.

 

https://docs.paloaltonetworks.com/pan-os/9-0/pan-os-new-features/upgrade-to-pan-os-90/upgrade-the-fi...

"Download and install the latest preferred 8.1.x maintenance release and reboot."

 

8.1.4 -> 9.0.3 is a HUGE jump that I would never attempt.  I would have gone to at least 8.1.11 or whatever was available first, then you can leap over 9.0 (as long as it's downloaded) and go straight to 9.0.3 (current recommended it 9.0.5).

Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!

The Live Community thanks you for your participation!