Why is this commit not working ?

by ‎02-01-2017 07:47 AM - edited ‎05-11-2017 08:52 AM (679 Views)

We've all been there right ? Feeling confused when our computer had problems and all we would get were these inscrutable error messages. We had no idea what was wrong because it wouldn't tell us any more or it just didn't make much sense to us.

 

32f8e99979a30dfe3f919adef9ad399d.jpg

 

 

In addition to that, depending on your hardware platform or the load on your device, a commit can take a while before it will spit out an error.  I'm sure your time is as valuable as mine and you want to move forward as fast as possible without doing too much trial and error.

 

In this week's blog I'll try to answer your questions concerning commit issues.

  

The first place to look at would be the Commit Status screen.  The Details field usualy gives you a hint of where the problem might be located :

 

Commit Status DetailsCommit Status Details

 

Alternatively the Task Manager window can show you the same information, in addition to other tasks that might have been going on at the same time :

 

Task ManagerTask Manager

In the CLI you can also check the job status with the command 'show jobs all' and check for more details with the command 'show jobs id <value>' :

 

admin@PA> show jobs all

Enqueued              Dequeued           ID  PositionInQ                              Type                         Status Result Completed 
------------------------------------------------------------------------------------------------------------------------------------------ 
2017/02/01 18:14:27   18:14:27        19893                                         Commit                            FIN   FAIL 18:14:40  


admin@PA> show jobs id 19893

Enqueued              Dequeued           ID                              Type                         Status Result Completed 
------------------------------------------------------------------------------------------------------------------------------
2017/02/01 18:14:27   18:14:27        19893                            Commit                            FIN   FAIL 18:14:40  
Warnings:
Details:vsys1
    Error: Fail to count address groups
(Module: device)
Commit failed

Description: 

 

Sometimes the error messages can be quite cryptic or even non-existant.

So how can you go beyond that and get more information while committing ?

Did you know that you can follow each logfile live during a commit ?

 

With the simple CLI command "tail follow yes" you can specify which logfile on which plane you would like to follow.

 

 

admin@PA> tail follow yes 
+ lines           output the last N lines, instead of the last 10
> agent-log       agent-log 
> cp-log          cp-log 
> dp0-log         dp0-log 
> dp1-log         dp1-log 
> mp-log          mp-log 
> webserver-log   webserver-log 

 

 

For example, lets say you would like to follow the management-server logs on the management plane during a commit. You can simply do this with the following CLI command :

 

 

admin@PA> tail follow yes mp-log ms.log

 

The above example will spit out every new log entry in the ms.log file until you interrupt it.  This allows you to follow exactly what's going on during a commit and might provide you with some additional details as to what could be wrong.

You are not limited to do this on just one file.  Open up several windows and tail multiple files at once if needed.  You might want to correlate or compare logfiles next to eachother.

 

Note that some of these logs can be very chatty, especially if you have increased the debug level on them.  Therefor I recommend that you have a big enough buffer configured or save the output somewhere for analysis.

 

Commit failures are not limited to erronous configurations.  Your device might not have enough resources available to perform a succesfull commit.  A good example of such a usecase :

https://live.paloaltonetworks.com/t5/Management-Articles/Slow-or-Failed-Commits/ta-p/61172

 

There is another useful command, "show management-clients".  This command shows the status of all of running daemons that are used during the commit process.  

Notice the * at the end of the 'device' line, indicating there was an issue with the device server during our commit.  In some cases the problem can be resolved by restarting the daemon that experienced an issue.

 

 

admin@PA-VM> show management-clients

              Client PRI    State Progress
-------------------------------------------------------------------------
              routed  30 P1-abort        0               
            ha_agent  25 P1-abort        0               
              device  20 P1-abort        0   *           
              ikemgr  10 P1-abort        0               
              keymgr  10     init        0    (op cmds only)
             logrcvr  10 P1-abort        0               
               dhcpd  10 P1-abort        0               
             varrcvr  10 P1-abort        0               
              sslvpn  10 P1-abort        0               
              rasmgr  10 P1-abort        0               
             useridd  10 P1-abort        0               
                satd  10 P1-abort        0               
             websrvr  10 P1-abort        0               
              sslmgr  10 P1-abort        0               
               authd  10 P1-abort        0               
              pppoed  10 P1-abort        0               
           dnsproxyd  10 P1-abort        0               
             cryptod  10 P1-abort        0               
              dagger  10     init        0    (op cmds only)
             l2ctrld  10 P1-abort        0               

Overall status: P1-abort. Progress: 0
Warnings:
Errors:
device: vsys1
device:     Error: Fail to count address groups
device: (Module: device)

 

The reasons for commit issues can be almost infinite and it would be nearly impossible to discuss all of them here.  Please post questions you would like to have answered, comments or suggestions below !

 

But before doing that, you might want to check on our Live Community.  

 

Your question might already have been answered.  Below is just a small set of commit related articles on the Live Community :

https://live.paloaltonetworks.com/t5/Featured-Articles/Threat-Database-Handler-Commit-Error/ta-p/120...

https://live.paloaltonetworks.com/t5/Featured-Articles/DotW-Commit-process-hangs-at-99/ta-p/71576

https://live.paloaltonetworks.com/t5/Management-Articles/Commit-failed-warning-Fail-to-count-address...

https://live.paloaltonetworks.com/t5/Management-Articles/Decrypt-mirror-unexpected-here-error-on-com...

 

Cheers !

@kiwi

Ask Questions Get Answers Join the Live Community