Forward logs from Firewall to Expedition for ML but storage always full

Announcements

ATTENTION Customers, All Partners and Employees: The Customer Support Portal (CSP) will be undergoing maintenance and unavailable on Saturday, November 7, 2020, from 11 am to 11 pm PST. Please read our blog for more information.

Reply
Highlighted
L1 Bithead

Forward logs from Firewall to Expedition for ML but storage always full

Hi All,

I tried to forward traffic logs from firewalls to the expedition for machine learning purposes, however, those traffic logs always occupied the storage and cause my expedition always unable to login with the WebUI. After I free up some storage and restart the MariaDB then I will only able to login.

 

However, when I tried to retrieve the Machine Learning results, it show no logs and no results. Ideally, I got one Allow-Any rules inside the firewall and I want to analyze those traffic which hitted the policy through machine learning and generate the results in csv. Previously I able to do once successfully when the firewalls and expedition just setup, after 6 month later I found even one logs will occupied the whole storage on expedition VM.

 

My question is :

1. Can I just forward the last 7 days logs of those traffic that hitted the Allow-Any rules to expedition for Machine Learning purpose only?

2. How can I get the Machine Learning analyze results without the logs on expedition? Can I directly pull traffic logs on firewalls?

 

Thank you very much for the helps!

Highlighted
L4 Transporter

Hi @EdwinKhng ,

 

To answer your questions:

1. Yes, you could, there are two ways to do it.  1. you can specify query like (rule eq Allow-Any) in the traffic log window and download the log manually from the firewall 2. you could make Expedition as Syslog server and use log forwarding profile in the "Allow-Any"'' rule. 

 

2. For ML, You will need the logs stored in Expedition, however, if you use Rule Enrichment, you can add firewall as log connector , then you don't need to store logs in Expedition. 

 

 

Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!

The Live Community thanks you for your participation!