Expeditions stopped accepting Syslog

[ChrisHammock]

I had this all setup and working, following the "Log Analysis Features of Expedition" tutorial.  The setup is, all FWs managed by single Panorama, logs forward from FWs to panorama.  I have setup panorama collector to forward the firewall logs to Expedition via syslog TCP.

This all worked fine for a few days then when I tried to login a realised the logs had used up all the disk space.  I had to delete some logs to make enough space and then reboot expedition, when I logged into the gui after the reboot I noticed the PanOrders agent had stopped.  I followed some instructions on the forum about reinstalling rabbitmq or something. 

sudo apt-get remove rabbitmq-server && apt-get purge rabbitmq-server
sudo apt-get install rabbitmq-server

This resolved the PanOrders issue but the rsyslog service would not restart, the rsyslog log said something about no socket available (or something like that) I think google suggested to run the command "systemctl enable rsyslog.service" which resolved the error and the rsyslog service is now running but I am still getting syslog dropped according to panorama.  I can see the sylsog hitting the expedition server using tcpdump but the files are not showing in the PALogs folder.

The only other change I have made since it worked was move the /PALogs folder to /data/PALogs, I have checked the permissions and updated both the rsyslog.conf and the expedition config regarding this change but I assume none of this would effect why the server is now dropping the syslog traffic?  I have restarted the rsyslog service multiple times and rebooted expedition.

Any help?  I am sure this is related to the hard disk filling up.

[ayazdani]

Hi,

Have you looked into the /etc/rsyslog.conf file to verify the config?

[ChrisHammock]

Turns out the issue was changing /PALogs folder to /data/PALogs.  Although this was done correctly in the /etc/rsyslog.conf and the permissions of the new folder was correct (and I rebooted to ensure any services were restarted) it was suggested there was some sort of bug with changing the directory.  I ended up creating a symlink and changing the directory back in /etc/rsyslog.conf so the syslog files are stored in /data/PALogs but the etc/rsyslog.conf still points to /PALogs.

thanks for the reply