Manual failback for PBF

[cburke]

Is there a way to force PBF rules to have to be manually failved back? As it is now, if our primary ISP fails, we failover to a secondary ISP using PBF. However, once the primary is back up, things fail back to it immediately. We would like to prevent the immediate fail back and not use a timer. ISP recoveries often times flap for a period of time, so we just want to wait it out until the failed ISP is deemed stable and manually fail back. Ideas? Thanks!

[rmonvon]

Hi...When there is a failover to the backup link, you can manually change the PBF rule so it does not fail back to the main link until you're ready.  There will be 2 manual steps but you'll get to control when the fail back occur.  Thanks,

[cburke]

So how does one configure such a thing?

[Raido_Rattameister]

On the Policy Based Forwarding rule there is checkbox "Disable this rule if next hop is unreachable"

Try if this fullfills your requirements.

[rmonvon]

Yes, use the 'disable this rule' option as suggested by Raido.  Create 2 PBF rules

rule1 - send all traffic to primary link with 'disable this rule' option

rule2 - send all traffic to backup link 

when primary is down & backup link is active, you need to login and manually disable rule1 to ensure primary is not use until you're ready.  When ready, enable rule1 for fail back to occur.

[cburke]

Seemingly not. That was the first thing I tried. On my primary ISP I ping 2 separate external polling IPs. If both external polling IPs fail pings (both rules have "Disable this rule if next hop is unreachable" checked), the firewall fails over to the other ISP as expected. This all works perfectly. However, as soon as either external IP comes back up, it fails back automatically. We want to prevent the fail back behavior. Thanks!

[rmonvon]

it fails back because that's by design.  Hence, I am suggesting that you manually disable the PBF rule such that it does not fail back and manually enable the rule when ready.

[jvalentine · Accepted solution]

It's not elegant, but it's possible.  

You'll have to loop-in an external system in order to accomplish this... for example, you could have the firewall syslog the system events to an external server.  That server could parse the logs looking for the pbf nh-down events like these:

When the next-hop monitor fails and the PBF rule is bypassed, that server can use an API call to take some sort of action, such as:

 - disable failed PBF rule, commit... or

 - modify object group membership used in PBF rule, commit... or

 - modify dynamic object group membership used to match PBF rule (no commit required!)

 - etc.  

[abjain]

I agree with the suggestions made. There is no way to keep the PBF rules for primary ISP disabled for some time. There has to be some manual intervention or external intervention.

But there should not be any issues with the existing sessions I believe if "wait-recover" is used in the monitoring profile. Is there a problem if new sessions start using primary ISP?

[cburke]

Ok, that makes more sense. So, really more of a procedure than something we can configure. The API call outs are interesting and what we were hoping to avoid, but this is super helpful.