Thanks for the response. Unfortunately, no, permissions have been consistent and what appears to be correct since beginning. In Expedition GUI / Settings /M.Learning, there is a blurb at the bottom for CSV and file rights:
Scheduled Log Export from FW devices may export your log files as expedition owned files. In case you want to activate delete after processing the CSV logs, make sure that www-data has write rights over the files. You can achieve it by adding the following into your root cron (the following example will verify the file rights every day at 00:05 am): 00 05 * * * * php /var/www/html/OS/spark/scripts/changeCSVLogRights.php
I entered this in my crontab (syntax shown above looks incorrect though; has an extra *) and it fixes permissions for 660 / chown www-data:expedition for any logs in a project that are set to be autoprocessed but haven't been processed yet. This does work. This step helped in my lab environment to get autoprocessing+deleteafter to succeed.
For what it's worth, here are two log files: one with 770 was set immediately after it was uploaded to expedition prior to processing. The other was set using the built-in script that runs at 00:05 nightly via cron. Both files have since been "auto processed" with "delete after processing" checked, but no deletion occurred after processing. Just showing that 770/660 seem to have no impact on problem.
expedition@Expedition:~/logs$ ls -lah total 317G drwxrwxr-x 2 expedition expedition 4.0K Sep 28 15:56 . drwxr-xr-x 5 expedition expedition 4.0K Sep 10 08:41 .. -rwxrwx--- 1 expedition www-data 157G Sep 27 17:39 PA5250-PRI_traffic_2020_09_27_last_calendar_day.csv -rw-rw---- 1 expedition www-data 161G Sep 28 17:50 PA5250-PRI_traffic_2020_09_28_last_calendar_day.csv
My lab environment has a FW sending logs via SCP to expedition, and the device in expedition is for the firewall itself w/no panorama. "process enabled files" button in device tab works for manual processing (have another thread on this board about this part). Logs weren't getting deleted until I put the php script in crontab, now it works good. My prod environment has FW sending logs via SCP to expedition, but the device is panorama with managed devices/config imported to it. "process enabled files" is greyed out and has never been usable, for what its worth, and deleting logs after auto processing has never worked.
... View more