06-08-2021 11:29 PM - edited 06-14-2021 06:52 AM
When the Palo Alto partition is full depending on which partition is full differen issues may happen. The Palo Alto versions like 9.1.x and newer have much less issues with disk space like the 8.1 version, so an upgrade to newer version may help in many cases.
1. Generally I have seen issues becase of full disk where the HA communicaton is impacted, the commit operation fails, the GUI is not responsible and also the log correlation process is down as seen in the masterd log file in the managfment plane (usually this happens because of the log partition panlogs being full ). After cleaning the disk space use "show system software status" / "show system resources" (https://docs.paloaltonetworks.com/pan-os/8-1/pan-os-cli-quick-start/cli-cheat-sheets/cli-cheat-sheet...) to see if the process is still down and if needed restart it with "debug software restart process <name of the process>"
https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000ClaJCAS
https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000ClSjCAK
https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000ClgZCAS
2. If the pancfg partition is full on Palo Alto firewalls or Panorama you will simply not be able to install new software versions of the panos or dynamic updates. In rare cases the GUI can be impacted.
3. An issue where the root partition is full you can only delete core files or system or process debug logs but if the issue persists then open a TAC case as they can access the Linux that Palo Alto uses as a core operational system and clean things from there.
https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000ClRXCA0
https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000PMznCAG
4. In some rare cases a partition like the /dev/shm can be full and a reboot of the firewall could be needed. If the issue is still there the ultimate solution for non hardware palo alto issues is saving the config to external storage then factory default reset of the firewall and again importing the the config(the TAC does this many times).
https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000CldXCAS
https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000ClRcCAK
06-10-2021 01:53 AM
Hi @nikoolayy1 ,
This is awesome ! Thanks for your contributions !
Cheers,
-Kiwi!
08-16-2021 04:57 PM
Hi,
Our /dev/shm is at 97% after upgrading to version 10.0.6,what is the reason for this?
We have rebooted the device in past and it used to fix it.
However this time it is still high
08-17-2021 12:31 PM - edited 08-17-2021 12:31 PM
Because /dev/shm is used for temporary memory storage do you see any other partitions being full?
Maybe also check if your hard disk is ok?
How to Run a Hard Disk Drive Performance Test - Knowledge Base - Palo Alto Networks
I did not see any known bugs for /dev/shm for your version but maybe 10.x is too heavy for your device because of the all new features like ML machine learning etc. If you are using hardware appliance did you check if this is a supported version by device? If it is virtual edition do you see any system logs that may indicate that more hard this is needed by the virtual edition similar to PA-VM 4GB Root Partition 100 Percent Full. - Knowledge Base - Palo Alto Networks ?
Add Additional Disk Space to the VM-Series Firewall (paloaltonetworks.com)
08-17-2021 07:21 PM
It is VM-300
I checked the device on auto assistant tool and it throws the following warming
> show system disk-space Filesystem Size Used Avail Use% Mounted on /dev/root 7.0G 4.3G 2.4G 65% / none 7.7G 76K 7.7G 1% /dev /dev/nvme0n1p5 16G 4.7G 11G 32% /opt/pancfg /dev/nvme0n1p6 7.9G 1.9G 5.7G 25% /opt/panrepo tmpfs 4.8G 4.7G 179M 97% /dev/shm cgroup_root 7.7G 0 7.7G 0% /cgroup /dev/nvme0n1p8 21G 13G 7.3G 64% /opt/panlogs tmpfs 12M 0 12M 0% /opt/pancfg/mgmt/ssl/private
Rest of the disk space looks alright
Another thing I noticed the swap is zero used and free, is this normal?
What would be your recommendation on this? Can this be ignored as we are receiving no issues from it yet.
08-18-2021 01:07 AM - edited 08-18-2021 01:13 AM
I suggest checking with the support just in case (it could normal thing in the new versions but better they to confirm this) as you seem not the only one with such issues with the new 10.6:
LIVEcommunity - How to clean up /dev/shm - LIVEcommunity - 411602 (paloaltonetworks.com)
You also may check for memory alarms in the systems log or if there is a process with memory leakage. Also 10.0.7 is out so better test if upgrading to it if there is the same issue:
PAN-OS 10.0.7 Addressed Issues (paloaltonetworks.com)
The SWAP being 0 means that there is no disk space that can be used for memory extension if needed, so this is reverse to the /dev/shm and it could be because of /dev/shm being full there is less memory available and the SWAP gets to overused because of this. With most linux systems /dev/shm only uses real memory and if it gets full the SWAP is not used to extend /dev/shm but with Palo Alto and the relation between them only the TAC may tell this.
How to Interpret: show system resources - Knowledge Base - Palo Alto Networks
If the support tells you this is a bug or normal behavior, please share it with us.
Click Accept as Solution to acknowledge that the answer to your question has been provided.
The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!
These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!
The LIVEcommunity thanks you for your participation!
