PANCast™ Episode 43: Troubleshooting Commit Issues

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements
L4 Transporter
No ratings

 

Episode Transcript:

 

John: 

Hello PANCasters. Let’s welcome back Olivier to another episode. Hi Olivier.

 

Olivier: Olivier Zheng, PCNSE, is a Staff Support Engineer at Palo Alto Networks. As SME Management/Logging Reporting in Technical Assistance Centre Singapore, he is supporting customers and participating in multiple knowledge sharing initiatives by writing content in the Knowledge Base, by delivering training to internal engineers.  He is responsible for 1 issued patent.  Olivier holds a Master of Science Mobile and High Speed telecom networks from Oxford Brookes University, UK and a Master of Science in Computer Science and Information Technology from ESI SUPINFO Paris, France.Olivier Zheng, PCNSE, is a Staff Support Engineer at Palo Alto Networks. As SME Management/Logging Reporting in Technical Assistance Centre Singapore, he is supporting customers and participating in multiple knowledge sharing initiatives by writing content in the Knowledge Base, by delivering training to internal engineers. He is responsible for 1 issued patent. Olivier holds a Master of Science Mobile and High Speed telecom networks from Oxford Brookes University, UK and a Master of Science in Computer Science and Information Technology from ESI SUPINFO Paris, France.

Hello John, thank you for having me back in PANCast™.

Today I wanted to share with our audience my troubleshooting method to resolve commit issues.

Disclaimer: I resolve a lot of issues with that, however, it may fail in some corner cases.

 

John: 

Great. Thanks Olivier. So first, when can we use your troubleshooting method?

 

Olivier:

So as I said today, we are going to discuss commit issues. That can be a commit change on the firewall, or on a Panorama. But this is also the case for HA configuration synchronization issues or for when you are trying to push a config from Panorama to a managed device.

You can also use this method to troubleshoot content update installation issues, because the content installation involves a commit operation.

In all those situations, performing the checks I will share will help you to narrow down the issue, if it does not resolve it.

 

John: 

Excellent, so let’s say my commit fails, what should I do first?

 

First Troubleshooting Checks

 

Olivier:

The first thing to check is the change you are going to commit.
You can review the changes for any error : is it a first time change or it is an update of an existing object?

If it is the first time you are implementing the change, refer to the Documentation to make sure all the steps are correct. 

If it is an update of an existing object, you can revert the change and clone the object to update the value on the cloned object. Also if you are doing multiple changes, you will need to troubleshoot by checking individual change to narrow down to the change causing the issue.

 

Also one quick check is to make a “blank” commit on the firewall, so you know that you are working from the running configuration, which should be a safe point.

 

Finally, last point I think to note: since PAN-OS 10.1, and the replayDB introduction, make sure if you want to restart a process to commit the whole configuration before. So you won’t get any weird configuration appearing suddenly.

 

John: 

Ok, so we tried all your troubleshooting actions, but the commit issue persists.

What's next?

 

Review the Commit Error Message

 

Olivier:

The next thing to do is to review the error message, that may sound silly, but yes, the error message may indicate clues related to the commit failure.

 

So when you have a commit failure, the first thing to review is the error message, and the change you are trying to commit. Are they related or not? For instance, you do add a security policy named “new_rule”, and the commit error message indicates an error with something else, then you can suspect the issue is not related to the security policy “new_rule”. If you are still not convinced, you can even delete the newly created security policy, and perform a commit.

 

One thing to mention is that you should review the commit error on the device performing the actual commit, that means if you are deploying something from Panorama, the managed device is the device performing the commit.

 

Unfortunately, in some cases the commit error may not contain enough information to resolve the issue.

 

John: 

Ok so the error does not have enough information, what do we do next?
 

Not Enough Information in the Error Message?

 

Olivier:

In case the error message is too generic, and you need more information, you will have to connect to the device through SSH for the command line interface.

Then display the logs of the process ms or configd, depending on the PAN-OS version your device is running on, while committing a change.

 

To do so, 

  • Make sure your terminal will record enough lines, as the process is quite chatty.
  • You can also save the terminal session in a text file to review it with a notepad.
  • Once you are ready, start by running the command “tail follow yes mp-log ms.log” or “tail follow yes mp-log configd.log” to start to display real time logs of the process.
  • Then perform the commit on the device.
  • Finally stop the log display by pressing the keys CTRL and C.

 

Now, the game is to see what is happening in the logs, so finding the error in the logs can be something straightforward, but it also may require you to see other logs.

So in case you don’t see immediately the error in the log, you will need to see the first process sending a commit failure message and perform the same procedure for that specific process.

 

John: 

Ok got it.

So what are the takeaways for today?

 

Episode Key Takeaways

 

Olivier:

So the key takeaways for today :

  • Review the change you are going to commit
  • Review the error message on the device performing the commit
  • And if you need, review the logs directly on the device performing the commit

Finally, the last point about ReplayDB, make sure that there is no pending commit if you have to restart the device or a process.

 

John: 

Thank you Olivier, great info as always. PANCasters, remember to head to live.paloaltonetworks.com for the transcript and additional info.

 

Related Content:

Strata Cloud Manager NGFW Panorama 

Rate this article:
  • 596 Views
  • 0 comments
  • 1 Likes
Register or Sign-in
Contributors
Article Dashboard
Version history
Last Updated:
‎06-14-2024 11:26 AM
Updated by: