article PANCast™ Episode 43: Troubleshooting Commit Issues in PANCast™ Episodes

article PANCast™ Episode 43: Troubleshooting Commit Issues in PANCast™ Episodes https://live.paloaltonetworks.com/t5/pancast-episodes/pancast-episode-43-troubleshooting-commit-issues/ta-p/589039 <DIV class="lia-message-template-content-zone"> <P><div class="video-embed-center video-embed"><iframe class="embedly-embed" src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F_IRTet57uGk%3Ffeature%3Doembed&display_name=YouTube&url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D_IRTet57uGk&image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F_IRTet57uGk%2Fhqdefault.jpg&type=text%2Fhtml&schema=youtube" width="600" height="337" scrolling="no" title="PANCast™ Episode 43: Troubleshooting Commit Issues" frameborder="0" allow="autoplay; fullscreen; encrypted-media; picture-in-picture;" allowfullscreen="true"></iframe></div></P> <P> </P> <P><STRONG><I><EM>Episode Transcript:</EM></I></STRONG></P> <P> </P> <P><STRONG><I>John:</I> </STRONG></P> <DIV><SPAN>Hello PANCasters. Let’s welcome back Olivier to another episode. Hi Olivier.</SPAN></DIV> <P> </P> <P><STRONG><EM><FONT color="#FF6600"><I>Olivier:</I></FONT></EM></STRONG> <span class="lia-inline-image-display-wrapper lia-image-align-right" image-alt="Olivier Zheng, PCNSE, is a Staff Support Engineer at Palo Alto Networks. As SME Management/Logging Reporting in Technical Assistance Centre Singapore, he is supporting customers and participating in multiple knowledge sharing initiatives by writing content in the Knowledge Base, by delivering training to internal engineers. He is responsible for 1 issued patent. Olivier holds a Master of Science Mobile and High Speed telecom networks from Oxford Brookes University, UK and a Master of Science in Computer Science and Information Technology from ESI SUPINFO Paris, France." style="width: 240px;"><img src="https://live.paloaltonetworks.com/t5/image/serverpage/image-id/60356i030711E453B5774A/image-dimensions/240x236?v=v2" width="240" height="236" role="button" title="PANCast_Olivier-Zheng_palo-alto-networks.png" alt="Olivier Zheng, PCNSE, is a Staff Support Engineer at Palo Alto Networks. As SME Management/Logging Reporting in Technical Assistance Centre Singapore, he is supporting customers and participating in multiple knowledge sharing initiatives by writing content in the Knowledge Base, by delivering training to internal engineers. He is responsible for 1 issued patent. Olivier holds a Master of Science Mobile and High Speed telecom networks from Oxford Brookes University, UK and a Master of Science in Computer Science and Information Technology from ESI SUPINFO Paris, France." /><span class="lia-inline-image-caption" onclick="event.preventDefault();">Olivier Zheng, PCNSE, is a Staff Support Engineer at Palo Alto Networks. As SME Management/Logging Reporting in Technical Assistance Centre Singapore, he is supporting customers and participating in multiple knowledge sharing initiatives by writing content in the Knowledge Base, by delivering training to internal engineers. He is responsible for 1 issued patent. Olivier holds a Master of Science Mobile and High Speed telecom networks from Oxford Brookes University, UK and a Master of Science in Computer Science and Information Technology from ESI SUPINFO Paris, France.</span></span></P> <P><SPAN>Hello John, thank you for having me back in PANCast™.</SPAN></P> <P><SPAN>Today I wanted to share with our audience my troubleshooting method to resolve commit issues.</SPAN></P> <P> </P> <P><SPAN>Disclaimer: I resolve a lot of issues with that, however, it may fail in some corner cases.</SPAN></P> <P> </P> <P><STRONG><I>John:</I> </STRONG></P> <DIV><SPAN>Great. Thanks Olivier. So first, when can we use your troubleshooting method?</SPAN></DIV> <P> </P> <P><STRONG><EM><FONT color="#FF6600"><I>Olivier:</I></FONT></EM></STRONG></P> <P><SPAN>So as I said today, we are going to discuss commit issues. That can be a commit change on the firewall, or on a Panorama. But this is also the case for HA configuration synchronization issues or for when you are trying to push a config from Panorama to a managed device.</SPAN></P> <P><SPAN>You can also use this method to troubleshoot content update installation issues, because the content installation involves a commit operation.</SPAN></P> <P><SPAN>In all those situations, performing the checks I will share will help you to narrow down the issue, if it does not resolve it.</SPAN></P> <P> </P> <P><STRONG><I>John:</I> </STRONG></P> <DIV><SPAN>Excellent, so let’s say my commit fails, what should I do first?</SPAN></DIV> <P> </P> <H2 id="toc-hId-1736998815"><STRONG><FONT color="#FF6600">First Troubleshooting Checks</FONT></STRONG></H2> <P> </P> <P><STRONG><EM><FONT color="#FF6600"><I>Olivier:</I></FONT></EM></STRONG></P> <P><SPAN>The first thing to check is the change you are going to commit.</SPAN><SPAN><BR /></SPAN><SPAN>You can review the changes for any error : is it a first time change or it is an update of an existing object?</SPAN></P> <P><SPAN>If it is the first time you are implementing the change, refer to the Documentation to make sure all the steps are correct. </SPAN></P> <P><SPAN>If it is an update of an existing object, you can revert the change and clone the object to update the value on the cloned object. Also if you are doing multiple changes, you will need to troubleshoot by checking individual change to narrow down to the change causing the issue.</SPAN></P> <P> </P> <P><SPAN>Also one quick check is to make a “blank” commit on the firewall, so you know that you are working from the running configuration, which should be a safe point.</SPAN></P> <P> </P> <P><SPAN>Finally, last point I think to note: since PAN-OS 10.1, and the replayDB introduction, make sure if you want to restart a process to commit the whole configuration before. So you won’t get any weird configuration appearing suddenly.</SPAN></P> <P> </P> <P><STRONG><I>John:</I> </STRONG></P> <DIV> <P><SPAN>Ok, so we tried all your troubleshooting actions, but the commit issue persists.</SPAN></P> <P><SPAN>What's next?</SPAN></P> </DIV> <P> </P> <H2 id="toc-hId-1736998815"><STRONG><FONT color="#FF6600">Review the Commit Error Message</FONT></STRONG></H2> <P> </P> <P><STRONG><EM><FONT color="#FF6600"><I>Olivier:</I></FONT></EM></STRONG></P> <P><SPAN>The next thing to do is to review the error message, that may sound silly, but yes, the error message may indicate clues related to the commit failure.</SPAN></P> <P> </P> <P><SPAN>So when you have a commit failure, the first thing to review is the error message, and the change you are trying to commit. Are they related or not? For instance, you do add a security policy named “new_rule”, and the commit error message indicates an error with something else, then you can suspect the issue is not related to the security policy “new_rule”. If you are still not convinced, you can even delete the newly created security policy, and perform a commit.</SPAN></P> <P> </P> <P><SPAN>One thing to mention is that you should review the commit error on the device performing the actual commit, that means if you are deploying something from Panorama, the managed device is the device performing the commit.</SPAN></P> <P> </P> <P><SPAN>Unfortunately, in some cases the commit error may not contain enough information to resolve the issue.</SPAN></P> <P> </P> <P><STRONG><I>John:</I> </STRONG></P> <DIV><SPAN>Ok so the error does not have enough information, what do we do next?</SPAN></DIV> <DIV> </DIV> <DIV> <H2 id="toc-hId-1736998815"><STRONG><FONT color="#FF6600">Not Enough Information in the Error Message?</FONT></STRONG></H2> </DIV> <P> </P> <P><STRONG><EM><FONT color="#FF6600"><I>Olivier:</I></FONT></EM></STRONG></P> <P><SPAN>In case the error message is too generic, and you need more information, you will have to connect to the device through SSH for the command line interface.</SPAN></P> <P><SPAN>Then display the logs of the process ms or configd, depending on the PAN-OS version your device is running on, while committing a change.</SPAN></P> <P> </P> <P><SPAN>To do so, </SPAN></P> <UL> <LI><SPAN>Make sure your terminal will record enough lines, as the process is quite chatty.</SPAN></LI> <LI><SPAN>You can also save the terminal session in a text file to review it with a notepad.</SPAN></LI> <LI><SPAN>Once you are ready, start by running the command “tail follow yes mp-log ms.log” or “tail follow yes mp-log configd.log” to start to display real time logs of the process.</SPAN></LI> <LI><SPAN>Then perform the commit on the device.</SPAN></LI> <LI><SPAN>Finally stop the log display by pressing the keys CTRL and C.</SPAN></LI> </UL> <P> </P> <P><SPAN>Now, the game is to see what is happening in the logs, so finding the error in the logs can be something straightforward, but it also may require you to see other logs.</SPAN></P> <P><SPAN>So in case you don’t see immediately the error in the log, you will need to see the first process sending a commit failure message and perform the same procedure for that specific process.</SPAN></P> <P> </P> <P><STRONG><I>John:</I> </STRONG></P> <DIV> <P><SPAN>Ok got it.</SPAN></P> <P><SPAN>So what are the takeaways for today?</SPAN></P> </DIV> <P> </P> <H2 id="toc-hId-1736998815"><STRONG><FONT color="#FF6600">Episode Key Takeaways</FONT></STRONG></H2> <P> </P> <P><STRONG><EM><FONT color="#FF6600"><I>Olivier:</I></FONT></EM></STRONG></P> <P>So the key takeaways for today :</P> <UL> <LI>Review the change you are going to commit</LI> <LI>Review the error message on the device performing the commit</LI> <LI>And if you need, review the logs directly on the device performing the commit</LI> </UL> <P>Finally, the last point about ReplayDB, make sure that there is no pending commit if you have to restart the device or a process.</P> <P> </P> <P><STRONG><I>John:</I> </STRONG></P> <DIV> <P><SPAN>Thank you Olivier, great info as always. PANCasters, remember to head to live.paloaltonetworks.com for the transcript and additional info.</SPAN></P> </DIV> <P> </P> <P><STRONG>Related Content:</STRONG></P> <UL> <LI><A title="How to identify the commit failure reason when no error message is displayed in the GUI" href="https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000PMb2CAG" target="_blank" rel="noopener nofollow noreferrer">How to identify the commit failure reason when no error message is displayed in the GUI</A></LI> <LI><A title="Difference between standard commit and commit force" href="https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA14u0000001VVaCAM" target="_blank" rel="noopener nofollow noreferrer">Difference between standard commit and commit force</A></LI> <LI><A title="Triage Commit Issues on Panorama" href="https://docs.paloaltonetworks.com/panorama/11-1/panorama-admin/troubleshooting/troubleshoot-commit-failures/triage-commit-issues-on-panorama" target="_blank" rel="noopener nofollow noreferrer">Triage Commit Issues on Panorama</A></LI> <LI><A title="Persistent Uncommitted Changes on PAN-OS" href="https://docs.paloaltonetworks.com/pan-os/10-1/pan-os-new-features/management-features/persistent-uncommitted-changes-on-pan-os" target="_blank" rel="noopener nofollow noreferrer">Persistent Uncommitted Changes on PAN-OS</A></LI> </UL> <P><LI-PRODUCT title="Strata Cloud Manager" id="Strata_Cloud_Manager"></LI-PRODUCT> <LI-PRODUCT title="NGFW" id="NGFW"></LI-PRODUCT> <LI-PRODUCT title="Panorama" id="Panorama"></LI-PRODUCT> </P> </DIV> Fri, 14 Jun 2024 18:26:32 GMT ozheng 2024-06-14T18:26:32Z PANCast™ Episode 43: Troubleshooting Commit Issues https://live.paloaltonetworks.com/t5/pancast-episodes/pancast-episode-43-troubleshooting-commit-issues/ta-p/589039 <P><SPAN>This episode discusses about what to do when you are facing a commit issue.</SPAN></P> <P> </P> <P><span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="Title_PANCast Ep 43_palo-alto-networks.jpg" style="width: 960px;"><img src="https://live.paloaltonetworks.com/t5/image/serverpage/image-id/60371i297062BBF28AECF1/image-size/large?v=v2&px=999" role="button" title="Title_PANCast Ep 43_palo-alto-networks.jpg" alt="Title_PANCast Ep 43_palo-alto-networks.jpg" /></span></P> Fri, 14 Jun 2024 18:26:32 GMT https://live.paloaltonetworks.com/t5/pancast-episodes/pancast-episode-43-troubleshooting-commit-issues/ta-p/589039 ozheng 2024-06-14T18:26:32Z