topic Re: Azure HA Failover not working in VM-Series in the Public Cloud

Azure HA Failover not working

RyanJohnstone1144 — Thu, 22 Oct 2020 09:13:47 GMT

Hello,

We have a pair of VM300 PAs in Azure set up in Active-Passive. They are running 9.0.7 code with VM Series plug in 1.0.8.

There was an issue in Azure on 19/10/20 which caused a failover and recovery (we use pre-emption). Post this issue the PAs were up and running but not passing traffic. we found that the secondary IP addresses (i.e. floating IPs) had been moved to the Azure VM for the Passive firewall (PA2). Hence no traffic flowing as this firewall was passive. We failed from the Active (PA1) over to this Firewall (PA2) and some traffic started to flow but everything was incredibly slow. we tried restarting VMs, failing back over, etc but nothing would change the state of the secondary IP addresses, they were locked to PA2.

Eventually we completely powered down the VM which was running PA1 and things started to run ok again. we then configured PA2 to always be Active and powered PA1 back up. PA1 came back up, re-established HA and things were running fine, PA2 Acitve, PA1 Passive. We then suspended PA2 to trigger failover and again we had issues with secondary addresses. The secondary addresses on the untrust VM interface floated over to PA1 correctly, but the secondary address on Trust VM interface disappeared completely from both PA1 and PA2. No failovers, restarts etc recovered this address.

We powered down PA2 and had to manually re-create the secondary address on Trust on PA1 to restore service. this is the state we are now in.

can someone please assist / recommend next steps? Failover it seems is broken.

thanks

Ryan

Re: Azure HA Failover not working

Ravindra692 — Wed, 28 Oct 2020 13:12:26 GMT

Does any of the interfaces has a Public IP's associated with them?

I also had a similar issue but in my case I had public IP's associated with the interface and I used the Standard SKU for it. Once I changed it to Basic SKU the failover is working fine for me. However, the failover time was 6 to 8 mins each time.

In your Active/Passive Scenario Do you have L2L VPN tunnels configured?

Re: Azure HA Failover not working

RyanJohnstone1144 — Mon, 09 Nov 2020 10:16:38 GMT

Hello, thanks for response. yes we have public IPs on one interface, i will ask our CSP for guidance on this setup.

Yes we have L2L tunnels configured as well, are there issues here which could be cause?

Thanks again for reply

Ryan

Re: Azure HA Failover not working

Ravindra692 — Mon, 09 Nov 2020 13:36:20 GMT

I ran the failover test like 20 times in the Active Standby Scenario. Two things I observed there.

1. A couple of times that tunnel took 20 mins to failover.

2. Three times the tunnel didnt come up after the failover, I had to do a force re-negotiate

Re: Azure HA Failover not working

JoshuaKovach — Fri, 20 Aug 2021 18:51:01 GMT

Was there ever a solution found for this issue? If so, could you please share?

Re: Azure HA Failover not working

RyanJohnstone1144 — Tue, 31 Aug 2021 08:24:49 GMT

HI there, no solution no. am pending some assistance from PA TAC once i can arrange an outage. We have tried multiple code versions and plug ins but problem remains. i will update once i have an update, hopefully in the coming month.

Re: Azure HA Failover not working

RyanJohnstone1144 — Wed, 06 Oct 2021 15:51:48 GMT

still no "solution" to this but carried out a controlled failover last week and it worked, first time it has worked as it should have done since the PAs were stood up about 2 years ago. FYI the code we are on is 9.0.13 Azure plugin 1.0.13.

have some upgrades coming up over next few months to get to 9.1 so be interesting to see if any issues occur then. otherwise can only assume something was fixed in the background Azure end or maybe i just got lucky...

Re: Azure HA Failover not working

JimMcGrady — Fri, 17 Dec 2021 07:10:45 GMT

I have always found this failover mechanism within Azure to be impractically slow. I look forward to the day when it improves.