Panorama VM 8.1.5

[Shadow]

Hi all,

just got this case, where the client shutdown the Panorama 8.1.5 for maintenance. But once reboot, and login to the device he will not see the ">". if he performs any commands there are outputs. But the prompt is missing. He reloaded the device and has been is this state for over 24hr now.

Usual CUP load is 5% on their network. But now its over 37%. The memory and disk looks fine.

I have come across this issue on my VM lab. Same thing. When tried to login it times out, then I would not see the prompt. But after a while (15min) everything is fine, which tells me there some processors still starting. But it doesn't look like the same case with this client.

I cannot confirm whether he has access via WebGUI. But that's to be confirmed.

Any idea?

Thanks in advance

Lance

[reaper]

the prompt ">" will appear once the management server (mgmt-srv) is fully loaded. After reboots and depending how much config etc need to be loaded (and available resources etc; there are several different background processes that need to start before the mgmtsrv is fully 'up) , it can take a while before the management process becomes available

you can check if there are extraordinary delays during the boot process by checking the ms.log file

[Shadow]

I did come across this solution, to the description I have to add, once you log in there was no prompt ">" + if you run some show commands there were no output(black screen).

with help of TAC and tanks to Pradip, following was discovered and fixed

==============Knowledge share===================================

As the configuration of the ESXI is same and it does not seems to be a VM issue.
>> We are not able to access Panorama GUI.
>>We are able to get CLI but whenever we type command it does not return any output or error.
>> We tried restarting Panorama and it did not work.
>> We visited MAINT mode and tried reinstalling the PAN OS and tried reverting to older PAN OS both options did not worked.

configd.log
--------------

<-------SNIP------> Error: pan_mongo_output_error(pan_mongo.c:515): [MongoDB Failure] mongo_insert failed: 18446744073709551615(0),
<-------SNIP------> Error: pan_cfg_mongo_clear_unfinished_jobs(src_cms/pan_cfg_mongo_tables.c:15632): Failed to insert record for key

+ We ran the commands on root shell to repair the MongoDB.

# mongodump -d pancfg -o /tmp/db_backup

# mongo
> use pancfg
> db.dropDatabase()
> exit

# cd /tmp/db_backup
# mongorestore --drop -d pancfg pancfg (you must run this from /tmp/db_backup)

# mongo
> use pancfg
> db.repairDatabase()

+ We tried to SSH to the device, connect to WebUI and do a commit. All of them were successful

================================

PLEASE REMEMBER, by using the above commands you are responsible for your firewall,Palo Alto/TAC engineer Pradip or I are not responsible for your mess up