There's a debug command that can help you clean up old logs automatically

by Community Manager ‎07-03-2018 06:13 AM - edited ‎07-06-2018 11:45 AM (16,820 Views)

Several of our customers have reported in the past that their systems were having trouble with available disk space on the management plane.

 

In most cases, it turned out that management process logs had become overweight and filled up more disk space than desired. This is because log records are not simply purged when the log file grows large, but an 'archive' is created that stores older logs up to a total of 4 additional versions. This is the expected behavior: if debugging is enabled on one or more of the management plane processes (device server, management server, ..), this will temporarily cause additional logs to be written and the log to grow in size more rapidly. Recent history is not immediately purged out, and some history can be retained before losing this information for future reference or troubleshooting purposes by creating an 'old' log and starting a fresh log.

 

admin@PA-5220> ls long-format yes mp-log mp-monitor*
-rw-r--r-- 1 root root   455144 Jul  3 05:45 /var/log/pan/mp-monitor.log
-rw-r--r-- 1 root root 10481820 Jul  3 04:58 /var/log/pan/mp-monitor.log.1
-rw-r--r-- 1 root root 10485513 Jul  2 09:54 /var/log/pan/mp-monitor.log.2
-rw-r--r-- 1 root root 10485393 Jul  1 14:54 /var/log/pan/mp-monitor.log.3
-rw-r--r-- 1 root root 10485585 Jun 30 19:50 /var/log/pan/mp-monitor.log.4

As you can see from the output above, some processes can be chatty in their logs and can retain several 'old' files so history is preserved for longer than a (few) day(s).

 

If several processes need the extra space at the same time, however, disk space may become scarce. An administrator can go in and delete older log files manually, but in case this task is cumbersome, frequent, and/or log retention is not crucial, a debug command has been introduced in PAN-OS 8.0.7 as PAN-79671  that can be set to automatically purge all 'old' logs when disk capacity reaches 95% of full:

 

debug software disk-usage aggressive-cleaning enable 
debug software disk-usage aggressive-cleaning disable 

When aggressive-cleaning is enabled, the system will not interfere with 'old' log files for as long as the disk capacity is below 95%. Once the high mark is reached, the system will automatically purge all the old (*.log.old , *.log.{1..4} ) files on the management plane to make room.

 

When the debug command is disabled, (default setting) the system will only purge any files that would go above *.log.4,

eg. *.log.4 is purged, *.log3 is renamed to *.log.4, *.log.2 is renamed to *.log.3 and so on, and a fresh *.log is started.

 

The debug is visible from the system state, once enabled.

 

admin@PA-5220> debug software disk-usage aggressive-cleaning enable 
This will automatically purge all old log files if disk hits 95% occupancy. Do you accept this potential loss of debuggability? (y or n) 

admin@PA-5220> show system state | match aggressive-cleaning
cfg.debug-sw-du.config: { 'aggressive-cleaning': True, }

 

 

 

Stay frosty,

Reaper

 

Comments
by dstjames
on ‎07-06-2018 05:18 AM

When I run this I get the following:

 


Server error : Failed to execute op command

by Community Manager
on ‎07-06-2018 11:54 AM

hi @dstjames

 

are you on 8.0.7 or later?

by dstjames
on ‎07-13-2018 06:59 AM

Yeah 8.1.1. 

 

 

by Community Manager
on ‎07-13-2018 07:22 AM

hi @dstjames

Have you tried restarting the management plane ( > request restart software ) ? May want to give that a try

If that doesn't help you may want to reach out to tac to have a look at what may be keeping you from exxecuting this command

 

by Mike18
on ‎10-05-2018 04:45 PM

like this post

very helpfull

Ask Questions Get Answers Join the Live Community
Labels