Forward logs from Firewall to Expedition for ML but storage always full

cancel
Showing results for 
Search instead for 
Did you mean: 

Forward logs from Firewall to Expedition for ML but storage always full

L1 Bithead

Hi All,

I tried to forward traffic logs from firewalls to the expedition for machine learning purposes, however, those traffic logs always occupied the storage and cause my expedition always unable to login with the WebUI. After I free up some storage and restart the MariaDB then I will only able to login.

 

However, when I tried to retrieve the Machine Learning results, it show no logs and no results. Ideally, I got one Allow-Any rules inside the firewall and I want to analyze those traffic which hitted the policy through machine learning and generate the results in csv. Previously I able to do once successfully when the firewalls and expedition just setup, after 6 month later I found even one logs will occupied the whole storage on expedition VM.

 

My question is :

1. Can I just forward the last 7 days logs of those traffic that hitted the Allow-Any rules to expedition for Machine Learning purpose only?

2. How can I get the Machine Learning analyze results without the logs on expedition? Can I directly pull traffic logs on firewalls?

 

Thank you very much for the helps!

1 REPLY 1

L4 Transporter

Hi @EdwinKhng ,

 

To answer your questions:

1. Yes, you could, there are two ways to do it.  1. you can specify query like (rule eq Allow-Any) in the traffic log window and download the log manually from the firewall 2. you could make Expedition as Syslog server and use log forwarding profile in the "Allow-Any"'' rule. 

 

2. For ML, You will need the logs stored in Expedition, however, if you use Rule Enrichment, you can add firewall as log connector , then you don't need to store logs in Expedition. 

 

 

Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!