Spark Temporary Data Structure - Path not provided

Showing results for 
Show  only  | Search instead for 
Did you mean: 

Spark Temporary Data Structure - Path not provided

L0 Member

I have configured a new instance of Expedition as our previous VM was not able to contact the internet to obtain updates (and there is no offline updater that works for ML).


After upgrading to the latest release (1.1.108), I am trying to complete the Spark Temp Data Structure path in the settings section, however the information is never saved. I have confirmed the permissions on the folder I'm looking to use. I have also followed the following advice:


>sudo apt-get update

>sudo apt-get --reinstall install expedition-beta


I have also rebooted several times. Why am I unable to change the path (I am intending to use /PADataStruct as the folder??




L1 Bithead

we have the same issue. We have created a directory "/datastore" with rights "drwxrwxrwx", owner "www-data" and group "expedition" (exactly like /PALogs). When we enter "/datastore" as "Temp. Data Structs:" the system responds with "Data saved", but it isn't saved. When we visit the page again the filed is empty and in the dashboard we still have the message " The ML Temporary Data Structures Path is incorrectly defined."

What are we doing wrong? Any ideas?

After you changed the temp data structures to the new path , please click on the red button "Delete Connections.parquet files" and then click on "Save". It should save the new path.  I verified it's working on v1.1.109.


Screen Shot 2021-09-13 at 8.39.16 AM.png

This doesn't work for me. When I click "Delete Connection.parquet files" it shows a message "/PALogs/connections.parquet does not exist. Nothing to delete". I then click "Save", but the issue is still the same - when I come back to the page the data struct field is blank and the check fails on the home page. Can you help further?

L5 Sessionator

Deleting the Connections.parquet should not be necessary, and it should be done only if we want to remove all the already processed data.


Looking at the figure we have in Expedition M.Learning settings, our CSV traffic logs would be stored in a path (individually defined for each firewall) and converter into Connections.parquet to allow performing the data analytics processes in ML and RE.

During the preprocessing, a folder can be defined to store temporary structures.


If you have admin rights within Expedition, the tool should allow you to modify these settings. If you cannot, please contact me to fwmigrate at and we can do a session to see what may be preventing the storage of the changes.

L2 Linker

I'm still having this issue. I have created a folder and applied the same ownership properties as /PALogs. When I enter the new folder name and click the "save" button it appears to be fine. But if I navigate away from the setting page and come back there is no setting. Is there a configuration file where I can set the manually under the hood?


Was there ever a resolution?

Get out there and do great things!

Please, check if the latest 1.1.113 version corrects this issue for you.


Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!