AutoFocus Miner Thought

Reply
Highlighted
L2 Linker

AutoFocus Miner Thought

Getting the minemeld engine up running the AutoFocus/TAXII redesign today, one thing I noticed while trying to create better and more targeted AutoFocus miners is that I cant create a list that maintains a specific “age”

 

Where many lists will allow an operator to call out the last 7 days or last 60 days of events, or being no older than that, AutoFocus only has the ability to code it for a range via JSON. Perhaps it would be a good feature to have them allow the database to process a JSON that’s a constantly no older than x? Would cut down on over producing data just to get cut back down inside minemeld.

 

I know it will let me find a range, but the range wont automatically follow along at a specific "Time to Live" of sorts. Might be a good idea?

L7 Applicator

Re: AutoFocus Miner Thought

Hi @hallerr,

I think you can achieve that by defining a miner based on an AF query with a specific start point and use the age_out parameter inside the Miner.

 

Example, with this query you can retrieve indicators of malware found in the Finance industry starting 30 days ago:

{
"children": [
{
"field": "sample.malware",
"operator": "is",
"value": 1
},
{
"field": "session.device_industry",
"operator": "is",
"value": "Finance"
},
{
"field": "sample.create_date",
"operator": "is after",
"value": [
"2017-02-23T00:00:00",
"2017-02-23T00:00:00"
]
}
],
"operator": "all"
}

When you create a Miner based on this query, it will first retrieve all the indicators matching this query (last 30 days worth of indicators) and then it start polling for new samples every 60 minutes. You can then use the age_out parameters inside the prototype to decide the maximum age of indicators to be stored in MineMeld. Example: after an indicator has been retrieved, you can decide to keep it in the feed for 60 days.

L2 Linker

Re: AutoFocus Miner Thought

Similar to what I was thinking but the dates would then be static correct? Im more thinking of a sliding value that is always 30 days old, and maybe thats what it does but it asks for a specific date?

L7 Applicator

Re: AutoFocus Miner Thought

Hi @hallerr,

only the date in the query is static and set the initial start of the sliding window. After that, every hour, the sliding window is pushed ahead:

1 - At time 0 a query is performed to retrieve the last 30 days worth of indicators using a query with a static date (time 0 - 30 days)

2 - After that every hour, a new query is performed to retrieve the new indicators that have been added after the last query

3 - and every hour, indicators older than 30 days are aged out and removed from the feed/graph

 

You can achieve this today by tuning the config settings of the AF Miner. 30 days could also be an arbitrary time range.

 

 

Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!

The Live Community thanks you for your participation!