07-11-2014 09:15 AM
I'm in the process of testing out two PAN-M-100's in the lab and more specifically testing the HA functionality at this point.
The issue that I am running into:
I have changed the Primary to Passive and the Secondary to Active, made a change to the Active/Secondary and then reverted the M-100's back to Active/Primary - Passive/Secondary. After doing this, instead of the Active/Primary pulling the latest config from the Passive/Secondary, it tries to overwrite the config with it's own. So in a nut shell, when we are failed over to our secondary M-100, all the changes we make will have to be redone on the Primary upon fail back.
Running version 5.1.3 (STIG compliance disallows us to upgrade, trust me I wish I could).
Any thoughts?
07-14-2014 09:42 AM
Could you share the output of "show high-availability all" from both firewalls?
07-14-2014 10:16 AM
These are not firewalls, they are M-100 Panoramas (managers). Below is the the "show high-availability all" of our current VM enviroment. The M-100's that we are testing in the lab have been uploaded with the exact configuration snap-shot of our live VM Panoramas. I cannot provide the read out from the M-100s but it shouldn't matter since everything is the same (Config, licneses, software, dynamic updates, etc).
PRIMARY:
High-Availability:
Local Information:
Version: 1
State: active
Device Information:
Election Option Information:
Priority: primary
Preemptive: yes
Promotion Hold Interval: 2000 ms
Hello Message Interval: 8000 ms
Heartbeat Ping Interval: 1000 ms
Preemption Hold Interval: 1 min
Monitor Fail Hold Up Interval: 0 ms
Addon Master Hold Up Interval: 7000 ms
Version Information:
Build Release: 5.1.3
URL Database: Not Installed
Application Content: 445-2292
Threat Content: 0
Anti-Virus: 1317-1787
Version Compatibility:
Software Version: Match
Application Content Compatibility: Match
Threat Content Compatibility: Match
Anti-Virus Compatibility: Match
Peer Information:
Connection status: up
Version: 1
State: passive
Device Information:
Connection up; Primary HA1 link
Election Option Information:
Priority: secondary
Preemptive: yes
Version Information:
Build Release: 5.1.3
URL Database: Not Installed
Application Content: 445-2292
Threat Content: 0
Anti-Virus: 1317-1787
Configuration Synchronization:
Enabled: yes
Running Configuration: synchronized
SECONDARY:
High-Availability:
Local Information:
Version: 1
State: passive
Last suspended state reason: User requested
Device Information:
Election Option Information:
Priority: secondary
Preemptive: yes
Promotion Hold Interval: 2000 ms
Hello Message Interval: 8000 ms
Heartbeat Ping Interval: 1000 ms
Preemption Hold Interval: 1 min
Monitor Fail Hold Up Interval: 0 ms
Addon Master Hold Up Interval: 7000 ms
Version Information:
Build Release: 5.1.3
URL Database: Not Installed
Application Content: 445-2292
Threat Content: 0
Anti-Virus: 1317-1787
Version Compatibility:
Software Version: Match
Application Content Compatibility: Match
Threat Content Compatibility: Match
Anti-Virus Compatibility: Match
Peer Information:
Connection status: up
Version: 1
State: active
Last suspended state reason: User requested
Device Information:
Connection up; Primary HA1 link
Election Option Information:
Priority: primary
Preemptive: yes
Version Information:
Build Release: 5.1.3
URL Database: Not Installed
Application Content: 445-2292
Threat Content: 0
Anti-Virus: 1317-1787
Path Monitoring Information:
Enabled: yes
Failure condition: any
Groups:
No path monitoring groups
Configuration Synchronization:
Enabled: yes
Running Configuration: synchronized
Again, the preemptive function works correctly up to the point where it doesn't Sync the recently config changes from the secondary to the primary. I have to manually run the sync command from the secondary CLI in order to make the sync work.
07-14-2014 12:03 PM
Looks pretty good to me (I have lots of A/P devices; but my VMware Panorama is just standalone - so can't compare directly). Clocks synced between the two devices OK?
07-14-2014 12:12 PM
Yes, the clocks are synced. It's almost as if there is a bug with the software version that isn't allowing changes made on the secondary device (while in active mode) to transfer to the primary device upon reinstatement.
What's even stranger, is that after turning OFF the preemptive mode on both devices (committing, etc), the primary device still automatically becomes active upon reinstatement to the network. It's like there is no difference between preemptive and non-preemptive.
07-15-2014 12:31 AM
Ok I overlooked that you said it's a M-100. The passive/secondary isn't able to manage policies and if the active/primary comes online again it will sync it's configuration because it's the only device which is allowed to manage the policies.
Panorama Administrator's Guide 6.0 (English) Page 127
I think you have to put them in active/primary and active/secondary if you want the secondary device to be able to sync the configuration.
Whoops my mistake. That is not possible. If this still doesn't work if preemptive is deactivated on both devices you should open a case.
I can't test this here and see how a suspended device behaves if the other peer commits a configuration. Could you see if there is a job running on the suspended device after you commit the configuration on the secondary?
If the guide is right the secondary should still be active after you make the primary functional again and preemptive is disabled on both devices.
07-15-2014 05:07 AM
I don't think you guys are picking up what I'm putting down here...I'm not making policy changes on a Passive/Secondary device, I'm making the policy changes on an ***ACTIVE***/Secondary device while the Primary device is offline. Once the Primary device comes back online, it automatically becomes the active, which is all good and gravy...however, any/all changes made on the previously ***ACTIVE***/Secondary device, don't sync with the NOW Active/Primary.
The only way I can get the changes on the previously ***ACTIVE***/Secondary to sync with the Primary device after reinstatement, is to run the sync command from the CLI of the Secondary device.
07-15-2014 05:32 AM
Suspending one device is probalby the same as a split brain situation. Understanding Split Brain in Panorama
>I have changed the Primary to Passive and the Secondary to Active, made a change to the Active/Secondary and then reverted the M-100's back to Active/Primary - Passive/Secondary.
If they are still able to see each other like in your first test, you should see a commit job on the passive device. You should open a case if you don't see a commit job on the passive device after you commited changes on the active one.
07-15-2014 05:53 AM
For one, the commit job doesn't automatically kick off like it should (that's an issue in it's self). When we go to kick off the sync (commit), the Primary overwrites any changes that were made on the secondary. I have opened a case with Palo Alto, so hopefully they can figure out where the bug is. Thanks for all of the inputs everyone!
07-15-2014 02:06 PM
Do keep us posted on the final resolution and bug number when assigned.
Also could you clarify the exact procedure you are using as the work around to get the committed changes on secondary onto the restored primary?
07-17-2014 05:46 AM
Steven,
There are two different ways that you can get the config from the Secondary device to sync with the Primary device.
1. Suspend HA on on the Primary and then immediately turn it back on, this will leave the primary in "Passive" mode for a couple minutes allowing you to push the sync from the Secondary.
2. (The easier way) Push the manual sync command in the Secondary device's CLI.
Both of these steps worked in getting config changes from the Secondary to the Primary.
Click Accept as Solution to acknowledge that the answer to your question has been provided.
The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!
These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!
The LIVEcommunity thanks you for your participation!
