Expedition Version 1.1.50 Log Analysis: CSV Still Pending after processing completes successfully
cancel
Showing results for 
Search instead for 
Did you mean: 

Expedition Version 1.1.50 Log Analysis: CSV Still Pending after processing completes successfully

L1 Bithead

After upgrading to Expedition 1.1.50 Devices show CSV Pending after log processing completes successfully.  Additionally, Apache Spark URL does not appear on the M. Learning tab Processing status bar when Spark is processing the CSV. Also the grey message bar always displays: The '/PALogs/NTXIRT00-IG0?-H_traffic_*' cannot be accessed. '

 

Finally, Once log processing completes the Traffic tab displays traffic for the log that was imported however Devices shows the CSV is still pending

 

PAN1.png.

 

PAN2.png

5 REPLIES 5

L1 Bithead

More detail for this issue. A fresh Expedition VM was built using New Expedition Installation Procedure.

 

  1. Added Device to Expedition using Active IP & Serial. CSV's processed correctly and shows as processed.
  2. Deleted Device in Expedition and Deleted Connections.parquet FILES.
  3. Added Device to Expedition using Passive IP & Serial + HA Serial (currently active). CSV's are processed but still show as pending in UI

Looking in the pandbRBAC.device_logs table the logs are listed as processed with the HA Serial. It seems the logic is not working correctly to flag logs sent from the HA Serial as processed.

Could you click on the Summary tab within the device?

If the CSV logs were processed, you should see entries for the dates that are compressed in those CSV files.

 

That could help us identifying where the issue could be. It is strange that the first time your file was processed without issues, but the second time, having deleted the connections.parquet, it would report as not processed when you actually processed it.

The difference between the first time and second time is which firewall in the HA pair is active vs configuration in Expedition. If the firewall configured as primary in Expedition generated the logs then everything works as expected. However, if the logs were generated by the device in HA Serial # field, then logs are process but listed as unprocessed.

 

Logs already processed but still listed as newLogs already processed but still listed as new

Thanks a lot for the clarification.

I will take a look at it with this new understanding of the issue

An additional confirmation. We failed our HA pair back where Primary IP & Serial match Expedition primary and logs are processing correctly.

Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!