After upgrading to Expedition 1.1.50 Devices show CSV Pending after log processing completes successfully. Additionally, Apache Spark URL does not appear on the M. Learning tab Processing status bar when Spark is processing the CSV. Also the grey message bar always displays: The '/PALogs/NTXIRT00-IG0?-H_traffic_*' cannot be accessed. '
Finally, Once log processing completes the Traffic tab displays traffic for the log that was imported however Devices shows the CSV is still pending
More detail for this issue. A fresh Expedition VM was built using New Expedition Installation Procedure.
Looking in the pandbRBAC.device_logs table the logs are listed as processed with the HA Serial. It seems the logic is not working correctly to flag logs sent from the HA Serial as processed.
Could you click on the Summary tab within the device?
If the CSV logs were processed, you should see entries for the dates that are compressed in those CSV files.
That could help us identifying where the issue could be. It is strange that the first time your file was processed without issues, but the second time, having deleted the connections.parquet, it would report as not processed when you actually processed it.
The difference between the first time and second time is which firewall in the HA pair is active vs configuration in Expedition. If the firewall configured as primary in Expedition generated the logs then everything works as expected. However, if the logs were generated by the device in HA Serial # field, then logs are process but listed as unprocessed.
Thanks a lot for the clarification.
I will take a look at it with this new understanding of the issue
An additional confirmation. We failed our HA pair back where Primary IP & Serial match Expedition primary and logs are processing correctly.
Click Accept as Solution to acknowledge that the answer to your question has been provided.
The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!
These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!
The Live Community thanks you for your participation!