HA upgrade oddness (no preempt)

[VK9H13]

Hello,

just seeing if anyone has an explanation or has experienced this before.

I followed the HA firewall upgrade guide and experienced this unexpected behavior.

i suspended the primary, ( secondary went active), installed the new os, and rebooted. 
Upon reboot, it was no longer suspended, came up and went active and caused a split brain. After it recovered, it stayed active, and secondary assumed passive. 
I then continued following the guide and suspended the now passive secondary, installed new os, and rebooted, and again, this one also came up active, split brain occurred, and recovered with primary staying active, and secondary assuming passive.

i cannot explain this whatsoever, there is no preempt configured and the guide indicates the suspended state should have stayed upon reboot. I have done upgrades in the past and suspended survived a reboot but not this time?

any one seen this before?

[reaper]

so you suspended and upgraded the primary? 

it is normal to come out of the reboot unsuspended, but it's upgraded state (OS mismatch) should prevent it from becoming active as it should normally go into 'non-functional

unless there is an external factor like HA1 disconnect, this is your root cause

why is the ha1 getting disconnected, is the ha1 link passing a firewall that could be blocking first packets? 

are you using dedicated links

did you enable ha1 backup/heartbeat?

[VK9H13]

Yes, suspended primary (active) and it went passive, while secondary took over as active.

heartbeat backup is enabled, ha1 and ha2 are dedicated links, directly connected to eachother. 
I looked through logs to make sure I wasn’t crazy, and sure enough, after suspending, installing, and rebooting it came up as active and secondary went to passive. 

Question on step 9 of Palo ha upgrade documentation, why is it sometimes I see “make device functional again” is not necessary? I see a lot of times it reboots and should just come up as passive, the only option under operational commands is to “suspend local device for high availability” again rather than “make local device functional for high availability”. ALOT of walk throughs I see, that are not Palo Alto documentation, this is not a required step. Even on Palo Alto’s documentation, step 8, after reboot, it shows a screenshot of the device as passive, not non functional. So step 9 is confusing.