Enhanced Security Measures in Place: To ensure a safer experience, we’ve implemented additional, temporary security measures for all users.
09-10-2020 12:15 PM
We filled up the drive by doing scheduled log exports, we then added a new disk and moved log exports to it, the only thing I had to fix was PanReadOrders was showing as stopped and when we tried to start it, it would just stop again. So I found an article where it was recommended to remove and re-add Rabbitmq and that got the Jobs and Task manager to start again
But now things don't seem well, I tried to pull down Contents again in Devices and everything shows as Updating or Downloading, and there are 4 pending jobs in Task Manager.
I checked the panReadOrders.log file around the time when it ran out of space and ever since then the bottom errors about a non-object are just repeating for 99% of the file...
Also in Projects things don't seem to be working, ML and Rule Enhancement both seem to think no devices are added, and under Devices the Traffic tab never shows anything.
Is Expedition hosed?
09-10-2020 12:20 PM
When this issue happened to us, we tried to replace the RabbitMQ as well, but that didn't solve our problem. Once we saw that this was going to be a much more involved process for restoring our expedition server, I went back to the VMWare snapshot that we had and restored from there. That was much more time-efficient for us.
However, even if we didn't have the snapshot, we would have just re-installed via the OVA fiile. Most of what we currently had in expedition was research related and could be re-created fairly easily. I know that doesn't really solve the issue, but it's the path that we took.
09-10-2020 01:25 PM
Hello @BRS-Alex
You can always check your volume with "df -k" to confirm where most of your space is being consumed and confirm the size is no longer adequate. Also once you confirm that my recommendation would be to re-size the VM, if you are unable to do that then I would mount another drive to the VM.
09-10-2020 01:44 PM
We did just that, and mounted another drive and resolved the space issue, then came the Rabbitmq issue and since then the Task Scheduler seems to be broken.
09-10-2020 01:45 PM
I was thinking that would be my next course of action, but before going that route I thought I'd throw it out there, maybe there was another step in restoring RabitMQ that I'm not aware of... but yeah it's a good bit of work we would be throwing away since the last snapshot.
09-10-2020 02:48 PM
You can try the following commands to fix rabbitmq
apt-get remove rabbitmq-server && apt-get purge rabbitmq-server
then reinstall it
apt-get install rabbitmq-server
Sudo rabbitmq-server start
Click Accept as Solution to acknowledge that the answer to your question has been provided.
The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!
These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!
The LIVEcommunity thanks you for your participation!
