Order to reboot devices in HA pair (passive)

Reply
Highlighted
Not applicable

Order to reboot devices in HA pair (passive)

We need to reboot our firewall due to some issues related to the traffic logging not working.  We have already attempted debug software restart log-receiver, syncing the devices etc and none of them have resolved the issue. We are pretty new to the device and have never had to reboot them. We have two PA-500's in an HA pair config. The backup is passive. It was suggested we reboot both devices.

What is the proper order if we intend to reboot both devices? If we reboot the main firewall will it initiate a reboot of the backup device or do we need to reboot each device separately?

Highlighted
L7 Applicator

Hello Bino,

Before re-start the PAN firewall, i would request you to follow  below mentioned steps:

Question-1:

>show logging-status 

Check: Last Log fwded and Last SeqNo. fwded counters. Make sure dates are showing correctly and sequence number is incrementing.

>debug log-receiver statistics

Check Log incoming rate and Log written rate are incrementing. Make sure below mentioned counters are not incrementing rapidly:

Log Forward discarded (queue full) count: 0  >>>>>>

Log Forward discarded (send error) count: 0  >>>>>>

Apply below mentioned command as per the sequence:

>debug software trace log-receiver

>debug software trace management-server

>debug software restart log-receiver

if no change still;

>debug software restart management-server 

Once you will restart the management-server process, it will take some time to come up, you will lose cli access for a few minutes and it will not impact to the production traffic through data-plane.

> debug dataplane pool statistics  >>>>>>>>> Verify Software pools are not depleted

> show system software status | match logrcvr  ( Restart may be required if not running/stopped)

Question-2: If the PA-500 HA pair is in a production environment, i would suggest you not to restart both firewalls at the same time. First restart the Active firewall, so the Secondary will become Active ( for the time being) and it will start passing production traffic. Once, it will become up, you may restart the second firewall.

Please let me know, if you have any other questions or concerns I can help address regarding this issue.


Thanks

Highlighted
L6 Presenter

Hi Bino,

Make sure partition is not full, that might be impacting logging.

Execute "show system disk-space" command.

admin@PA-500-Gia(active)> show system disk-space

Filesystem            Size  Used Avail Use% Mounted on

/dev/sda2             3.8G  1.4G  2.3G  38% /

/dev/sda5             7.6G  3.8G  3.4G  53% /opt/pancfg

/dev/sda6             3.8G  2.1G  1.6G  58% /opt/panrepo

tmpfs                 991M   67M  924M   7% /dev/shm

/dev/sda8             125G  2.3G  116G   2% /opt/panlogs--------------> Make sure this has space

Regards,

Hardik Shah

Highlighted
L6 Presenter

Hi Bino,

In HA, only active one does logging, and not passive unit. Than what is purpose of rebooting passive.

Passive might be able to do logging.

If you really want to reboot Active unit than follow bellow mentioned steps.

1. Suspend Active, now passive will take over.

request high-availability state suspend

2. Check if passive[New active] does logging for traffic logs.

3. If passive[New Active] does logging than reboot is not required.

4. At this point you can reboot active[new Passive] unit as its not passing traffic.

5. If passive[New Active] doesnt do logging than follow the same process.

Make sure you do not reboot both the boxes same time, at a time atleast one box should be passing traffic. That way you can avoid any kind of potential outage.

Regards,

Hardik Shah

Highlighted
Not applicable

Thank you for the info guys.

Here is the system disc space. it doesn't look too full

Filesystem            Size  Used Avail Use% Mounted on

/dev/sda3             3.8G  1.7G  1.9G  47% /

/dev/sda5             7.6G  2.9G  4.4G  40% /opt/pancfg

/dev/sda6             3.8G  1.2G  2.5G  33% /opt/panrepo

tmpfs                 963M   67M  896M   7% /dev/shm

/dev/sda8             125G   13G  106G  11% /opt/panlogs

Here is the logging status>

--------------------------------------------------------------------------------

---------------------------------------------

      Type      Last Log Created        Last Log Fwded       Last Seq Num Fwded

Last Seq Num Acked         Total Logs Fwded

--------------------------------------------------------------------------------

---------------------------------------------

> CMS 0

        Not Sending to CMS 0

> CMS 1

        Not Sending to CMS 1

>Log Collector

        Not Sending to Log Collector


Debug log-receiver statistics

admin@PA-500(active)> debug log-receiver statistics

Server error : An error occured. See dagger.log for information.

admin@PA-500(active)>

I've tried restart log-receiver several and restart management-server times a few days ago. it did get logging going for system etc but no effect for the traffic log.

>debug software trace log-receiver

>debug software trace management-server

>debug software restart log-receiver

if no change still;

>debug software restart management-server

This all relates back to this post which HULK helped me with.Logging stopped in Pan OS GUI

I did do show ntp as well

NTP state:

    NTP synched to 0.north-america.pool.ntp.org

    NTP server 0.north-america.pool.ntp.org connected: True

    NTP server 1.north-america.pool.ntp.org connected: True

I think a reboot is still needed to resolve this problem.

Highlighted
L7 Applicator

Hello Bino,

Instead of rebooting the PAN firewall, i would recommend you to open a ticket with PAN support and let them investigate this. So, you can prevent any future occurrence as well.

Thanks

Not applicable

Thanks Hulk.

Highlighted
L4 Transporter

As HULK said, you should definitely call support first. Before you suspend the active unit, it is recommended that you disable preemptive under election settings to prevent unwanted elections. Disabling the preempt configuration change must be committed on BOTH peers, and once completed, re-enabling must be committed on both peers. The setting is located in High Availability -> General Tab.


7-18-2014 11-54-35 AM.jpg

The inherent vice of capitalism is the unequal sharing of blessings; the inherent virtue of socialism is the equal sharing of miseries.
Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!

The Live Community thanks you for your participation!