Disk usage for /opt/pancfg exceeds limit, 100 percent in use

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Disk usage for /opt/pancfg exceeds limit, 100 percent in use

L3 Networker

Hi All,

running panorama 10.1.10-h2 VM in Azure with 4 managed VM100 FWs.

keep getting these alerts and not sure what else i can do. 'Disk usage for /opt/pancfg exceeds limit, 100 percent in use'

what i have done thusfar:   - deleted any old panos versions / dynamic updates / gp clients etc etc - there is nothing more to delete that wont break the system. deleted snapshots also but these are quite small  in comparison so doesn't make much of a difference.

 

padmin@panorama> show system disk-space
Filesystem Size Used Avail Use% Mounted on
....

/dev/sda5 24G 23G 238M 99% /opt/pancfg
...

 

if i run a 'request mongo show databases instance mgmt' it does not show the size of the mongodb. but there seems to be a few.

 

padmin@panorama> request mongo show databases instance mgmt

mon_summary_2024-02-29
pancfg
mon_summary_2024-02-26
mon_summary_2024-02-28
mon_summary_2024-02-20
mon_summary_2024-02-24
pan_rulehit
mon_summary_2024-02-21
admin
mon_summary_2024-02-27
mon_summary_2024-02-25
pan_devmon
pl_azure
mon_summary_2024-03-02
mon_summary_2024-02-22
mon_summary_2024-03-01
pan_warnings
panmgmtcache
mon_summary_2024-03-05
config
panreplay
pan_oplog
mon_summary_2024-03-03
local
mon_summary_2024-03-04
mon_summary_2024-02-23

padmin@panorama>

 

can some be deleted? and if so how?

 

i cannot do show system 'pancfg-directory-usage' because of not avail in 10.1

 

any ideas how to see the size of the mongodb's or how to clear more space?

 

thanks in adv.

1 accepted solution

Accepted Solutions

L3 Networker

Ok so update on this issue.. after exhausting all avenues with deleting images, content updates, config etc.. no luck.

in the end ran the following command to clear out the mongo DB which resolved the issue.. disk-space now down to 54%

 

panorama>debug mongo clear instance mdb
Are you willing to delete the data? This operation can not be reverted (y or n)

Delete data '' under path: /opt/pancfg/mgmt/mongo/data
Process 'mdb' executing STOP

panorama>

note when running this it will stop the mdb services.. so you need to verify and if stopped, need to actively start it up again.

panorama>show system software status

Process mdb stopped (pid: -1) - User Stop

panorama>

panorama> debug software restart process mdb

Process mdb was restarted by user NTT_Admin
panorama>

 

Will keep an eye on it for now.

thanks

 

View solution in original post

6 REPLIES 6

Community Team Member

Hi @PA_nts ,

 

Can you try the following command to clean up pancfg:

>debug pancfg-directory-usage set optimize-database-usage

 If not, are you able to upgrade to the 10.2 preferred 10.2.8? 10.2.8 addresses disk space usage issues. 

LIVEcommunity team member
Stay Secure,
Jay
Don't forget to Like items if a post is helpful to you!

Please help out other users and “Accept as Solution” if a post helps solve your problem !

Read more about how and why to accept solutions.

Hi JayGolf,

I am in a catch-22 situation..

this command is only avail in 10.2 - but i cannot download anything at the moment not even a content update.

>debug pancfg-directory-usage set optimize-database-usage

 there is a fix in 10.1.12 also.. but same issue.. cannot download the image to perform the upgrade

at the moment i cleared out everything.. i only have 10.1.0 (base image) and 10.1.10-h2 images downloaded, 3 content updates (av, threat and wildfire) and my snapshots are less than 50mb total.

 

PAN-232800
Fixed an issue where critical disk usage for
/opt/pancfg
increased continuously and the system logs displayed the following message:
Disk usage for /opt/pancfg exceeds limit, <value> percent in use
.
 

 

thanks

L3 Networker

Ok so update on this issue.. after exhausting all avenues with deleting images, content updates, config etc.. no luck.

in the end ran the following command to clear out the mongo DB which resolved the issue.. disk-space now down to 54%

 

panorama>debug mongo clear instance mdb
Are you willing to delete the data? This operation can not be reverted (y or n)

Delete data '' under path: /opt/pancfg/mgmt/mongo/data
Process 'mdb' executing STOP

panorama>

note when running this it will stop the mdb services.. so you need to verify and if stopped, need to actively start it up again.

panorama>show system software status

Process mdb stopped (pid: -1) - User Stop

panorama>

panorama> debug software restart process mdb

Process mdb was restarted by user NTT_Admin
panorama>

 

Will keep an eye on it for now.

thanks

 

Hi,

 

I'm running into this issue now. Do you know what this command actually clears? Did you this in production during work hours? The command sounds pretty scary and I'm not too sure if it will brakre annything.

 

Thanks

L3 Networker

I'd suggest to log a TAC for this if you can.. in my instance the tac engineer had to connect to the root OS to check the elastic seach health and found unassigned 'shards' on the es health.. which they had to clear manually. this process can only be performed by the tac as we as normal admins dont have permissions to log into the pan-os root.

That's what I had to do. They had to get on it and use their TAC login, it took a few hours, but it clear the disk on panorama. We just rejoined the call once it was done.

  • 1 accepted solution
  • 6694 Views
  • 6 replies
  • 0 Likes
Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!