topic Advice on upgrading HA pair in General Topics

Advice on upgrading HA pair

migration — Wed, 13 Apr 2011 16:02:51 GMT

I have read the couple of docs regarding the upgrading oh HA pairs, but I was more interested in actual user experience with the process. Does anyone have any sage advice for me as I plan my own upgrade event? I will be taking my PA500's from 3.1.6 to 3.1.8. I thought about moving to 4.0.1, but I hesitate to go to a .1 version of anything and prefer to wait until 4.0.2 comes out before making that leap.

Thanks!

Mike

Re: Advice on upgrading HA pair

MGoodnow — Wed, 13 Apr 2011 16:21:26 GMT

Hi Mike,

I got to say we have been extremely impressed with the failover ability of the PAN devices. We lose one ping during the transition. We have also done streaming audio and video tests during the failover and we don't lose a noticeable frame of content. I have become so comfortable and confident in the process, that I update our Internet HA's fairly quickly after new code release.

I start a continuous ping to www.yahoo.com then "suspend" HA1 so that HA2 takes over (confirm my one lost ping with yahoo). Upgrade and reboot HA1. Before I upgrade HA2, I like to watch the processes on HA1 after it comes up the first time and wait till all the processes have calmed down before moving on (show system resources follow). Then I perfrom a failover on HA2 - upgrade and reboot. The entire process usually takes me about 20 minutes and we have no downtime. We follow a change management procedure - but I dont' do any wide spread stake holder's message or wait till after hours. I typically do the update mid morning and have never had any compliants or issues (thus far). We started witth 3.0 code and have done all the 3.0.x updates in this fashion. I have not done 4.x yet on the HA pair - but most likely when .2 is released as you mentioned.

Cheers,

Mike

Re: Advice on upgrading HA pair

migration — Wed, 13 Apr 2011 16:39:21 GMT

It sounds, then, like you tackle your main, active, firewall first by suspending it, thus forcing the passive (HA2) to become active. Upgrading and rebooting HA1 leaves it in a passive state. Then you suspend HA2(the current active one), forcing HA1 into active status. Upgrade and reboot of HA2 which will leave it in its original condition of passive.

Do I have it about right? Have you ever just tried running the install on the first, active firewall(HA1) and letting it make its own failover decisions?

-mike

Re: Advice on upgrading HA pair

MGoodnow — Wed, 13 Apr 2011 17:29:00 GMT

Yes that is the exact process I follow. No haven't tried your question. But I know that is an option. I like controlling the failover. However, we had some experiences initially on the early 3.0 code (all been resolved) that did cause a couple of "auto failover" events. And during this, the failover worked perfectly and we didn't lose the Internet. I personally have not tried to trigger the auto fail during an upgrade though.

Re: Advice on upgrading HA pair

dagibbs — Thu, 14 Apr 2011 00:58:19 GMT

mwaters31 wrote:

Do I have it about right? Have you ever just tried running the install on the first, active firewall(HA1) and letting it make its own failover decisions?

-mike

Mike.

I've done exactly that on two 2050's in a HA configuration - twice now (from 3.1.4 to 3.1.6 and from 3.1.6 to 3.1.8).

I run the software uopgrade on the active first and then simply reboot it. Lgk on to the console of the secondary (now active), wait until the HA status shows the cluster is back online with the other node as passive, then repeat the process and reboot the second.

Log back on to the original, wait for the secondary to come back online, and you're laughing.

The *only* service interruption I noticed was if someone was logged in to the SSL VPN the session dropped and had to be re-established - everything else just kept on ticking without missing a beat. I'm amazed at how well these things failover and back - we have an inbound FTp server that has lots of connections - upwards of 1000 active at a time - and not *one* of them dropped out during either failover.

Cheers.

Re: Advice on upgrading HA pair

migration — Thu, 14 Apr 2011 14:43:42 GMT

Upgraded the pair last night. The whole process took approximately 34 minutes.

I installed the new OS on the active firewall (all using the GUI) first. The installation took about 4 minutes and then it prompted for a reboot, which I did. At that point the passive FW became active as the new changes were loaded into the first FW. During that failover, I lost about 8 pings to Yahoo.com. I think this has to do with the 8000ms timer to which that the PA500 is restricted to or that I don't have PortFast enabled on the switch interfaces due to the trunking we are doing into the FW's.

About 12 minutes later the first FW went through another failover, this time back to active, automatically. This caused the running active to return to its original passive status. During this switch back, I lost about 12 pings to Yahoo.com.

After everything had stabilized about 5 minutes later, I began the upgrade of the passive FW. This time it went a little quicker since there was only the one reboot. After each reboot, each FW took about 12-13 minutes to complete its software installation.

Now, last night I do a software "refresh" and lo and behold, there is version 4.0.2 tempting me to install. I resisited this temptation. :>)

Mike