Active/Passive HA Sync Issues

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Active/Passive HA Sync Issues

Not applicable

I'm in the process of testing out two PAN-M-100's in the lab and more specifically testing the HA functionality at this point.

The issue that I am running into:

I have changed the Primary to Passive and the Secondary to Active, made a change to the Active/Secondary and then reverted the M-100's back to Active/Primary - Passive/Secondary. After doing this, instead of the Active/Primary pulling the latest config from the Passive/Secondary, it tries to overwrite the config with it's own. So in a nut shell, when we are failed over to our secondary M-100, all the changes we make will have to be redone on the Primary upon fail back.

Running version 5.1.3 (STIG compliance disallows us to upgrade, trust me I wish I could).

Any thoughts?

24 REPLIES 24

Could you share the output of "show high-availability all" from both firewalls?

These are not firewalls, they are M-100 Panoramas (managers). Below is the the "show high-availability all" of our current VM enviroment. The M-100's that we are testing in the lab have been uploaded with the exact configuration snap-shot of our live VM Panoramas. I cannot provide the read out from the M-100s but it shouldn't matter since everything is the same (Config, licneses, software, dynamic updates, etc).

PRIMARY:

High-Availability:

  Local Information:

    Version: 1

    State: active

    Device Information:

    Election Option Information:

      Priority: primary

      Preemptive: yes

      Promotion Hold Interval: 2000 ms

      Hello Message Interval: 8000 ms

      Heartbeat Ping Interval: 1000 ms

      Preemption Hold Interval: 1 min

      Monitor Fail Hold Up Interval: 0 ms

      Addon Master Hold Up Interval: 7000 ms

    Version Information:

      Build Release: 5.1.3

      URL Database: Not Installed

      Application Content: 445-2292

      Threat Content: 0

      Anti-Virus: 1317-1787

    Version Compatibility:

      Software Version: Match

      Application Content Compatibility: Match

      Threat Content Compatibility: Match

      Anti-Virus Compatibility: Match

  Peer Information:

    Connection status: up

    Version: 1

    State: passive

    Device Information:

      Connection up; Primary HA1 link

    Election Option Information:

      Priority: secondary

      Preemptive: yes

    Version Information:

      Build Release: 5.1.3

      URL Database: Not Installed

      Application Content: 445-2292

      Threat Content: 0

      Anti-Virus: 1317-1787

  Configuration Synchronization:

    Enabled: yes

    Running Configuration: synchronized

SECONDARY:

High-Availability:

  Local Information:

    Version: 1

    State: passive

    Last suspended state reason: User requested

    Device Information:

    Election Option Information:

      Priority: secondary

      Preemptive: yes

      Promotion Hold Interval: 2000 ms

      Hello Message Interval: 8000 ms

      Heartbeat Ping Interval: 1000 ms

      Preemption Hold Interval: 1 min

      Monitor Fail Hold Up Interval: 0 ms

      Addon Master Hold Up Interval: 7000 ms

    Version Information:

      Build Release: 5.1.3

      URL Database: Not Installed

      Application Content: 445-2292

      Threat Content: 0

      Anti-Virus: 1317-1787

    Version Compatibility:

      Software Version: Match

      Application Content Compatibility: Match

      Threat Content Compatibility: Match

      Anti-Virus Compatibility: Match

  Peer Information:

    Connection status: up

    Version: 1

    State: active

    Last suspended state reason: User requested

    Device Information:

      Connection up; Primary HA1 link

    Election Option Information:

      Priority: primary

      Preemptive: yes

    Version Information:

      Build Release: 5.1.3

      URL Database: Not Installed

      Application Content: 445-2292

      Threat Content: 0

      Anti-Virus: 1317-1787

  Path Monitoring Information:

    Enabled: yes

    Failure condition: any

    Groups:

      No path monitoring groups

  Configuration Synchronization:

    Enabled: yes

    Running Configuration: synchronized

Again, the preemptive function works correctly up to the point where it doesn't Sync the recently config changes from the secondary to the primary. I have to manually run the sync command from the secondary CLI in order to make the sync work.

Looks pretty good to me (I have lots of A/P devices; but my VMware Panorama is just standalone - so can't compare directly).  Clocks synced between the two devices OK?

Yes, the clocks are synced. It's almost as if there is a bug with the software version that isn't allowing changes made on the secondary device (while in active mode) to transfer to the primary device upon reinstatement.

What's even stranger, is that after turning OFF the preemptive mode on both devices (committing, etc), the primary device still automatically becomes active upon reinstatement to the network. It's like there is no difference between preemptive and non-preemptive.

L3 Networker

Ok I overlooked that you said it's a M-100. The passive/secondary isn't able to manage policies and if the active/primary comes online again it will sync it's configuration because it's the only device which is allowed to manage the policies.

Panorama Administrator's Guide 6.0 (English)  Page 127

I think you have to put them in active/primary and active/secondary if you want the secondary device to be able to sync the configuration.

Whoops my mistake. That is not possible. If this still doesn't work if preemptive is deactivated on both devices you should open a case.

I can't test this here and see how a suspended device behaves if the other peer commits a configuration. Could you see if there is a job running on the suspended device after you commit the configuration on the secondary?

If the guide is right the secondary should still be active after you make the primary functional again and preemptive is disabled on both devices.

I don't think you guys are picking up what I'm putting down here...I'm not making policy changes on a Passive/Secondary device, I'm making the policy changes on an ***ACTIVE***/Secondary device while the Primary device is offline. Once the Primary device comes back online, it automatically becomes the active, which is all good and gravy...however, any/all changes made on the previously ***ACTIVE***/Secondary device, don't sync with the NOW Active/Primary.

The only way I can get the changes on the previously ***ACTIVE***/Secondary to sync with the Primary device after reinstatement, is to run the sync command from the CLI of the Secondary device.

Suspending one device is probalby the same as a split brain situation. Understanding Split Brain in Panorama

>I have changed the Primary to Passive and the Secondary to Active, made a change to the Active/Secondary and then reverted the M-100's back to Active/Primary - Passive/Secondary.

If they are still able to see each other like in your first test, you should see a commit job on the passive device. You should open a case if you don't see a commit job on the passive device after you commited changes on the active one.

For one, the commit job doesn't automatically kick off like it should (that's an issue in it's self). When we go to kick off the sync (commit), the Primary overwrites any changes that were made on the secondary. I have opened a case with Palo Alto, so hopefully they can figure out where the bug is. Thanks for all of the inputs everyone!

Do keep us posted on the final resolution and bug number when assigned.

Also could you clarify the exact procedure you are using as the work around to get the committed changes on secondary onto the restored primary?

Steve Puluka BSEET - IP Architect - DQE Communications (Metro Ethernet/ISP)
ACE PanOS 6; ACE PanOS 7; ASE 3.0; PSE 7.0 Foundations & Associate in Platform; Cyber Security; Data Center

Steven,

There are two different ways that you can get the config from the Secondary device to sync with the Primary device.

1. Suspend HA on on the Primary and then immediately turn it back on, this will leave the primary in "Passive" mode for a couple minutes allowing you to push the sync from the Secondary.

2. (The easier way) Push the manual sync command in the Secondary device's CLI.

Both of these steps worked in getting config changes from the Secondary to the Primary.

  • 9574 Views
  • 24 replies
  • 0 Likes
Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!