About last 30days query.

[KiCheonLee]

Hello

Here is a qurery about last 30days.

I just would like to change started date/time automatically when Minemeld do the query.

{"operator":"all","children":[{"field":"sample.malware","operator":"is","value":1},{"operator":"all","children":[{"field":"sample.create_date","operator":"is","value":["2017-12-31T00:00:00","2018-01-29T23:59:59"]}]}]}

Please let me know it

[xhoms]

@KiCheonLee,

just forget about the time attribute and let MineMeld take care of it. Do not add time-based attributes to the query.

Any search you provide to a samples.miner will be performed in the first poll with a time-window attribute of "last 24h" (you can override this with the "initial_interval" prototype attribute).

The same query will be perfomed again each "interval" seconds (defaults to a min value of 3600) adapting the time-window attribute based on the last successful poll.

Xavi

[Kichoen_Lee]

@xhoms

Thanks a lot for your answer.

I was glad that you answered my question.

May I ask you further questions?

Here are my configurations of prototype in a miner.

age_out  default : 1d

                    interval : 1800

                    sudden_death : false

attributes confidence : 100

                       share_level : red

initial_interval 1d

Please make sure my understanding of the above configurations as below,

the miner performed the first poll for "the last 24h" because the innitial_interval of the attribute is 1d.

And then the miner will perform the next poll after 1800 second because the interval is 1800.

Is my explanation right? and What does "age_out default 1d" mean?

I would like to keep that the FW has indicators for "last 30 days" by EDL from minemeld in autofocus .

I was confused on between your answer and the following link.

https://live.paloaltonetworks.com/t5/MineMeld-Discussions/AutoFocus-Miner-Thought/td-p/148935

Please let me know it. Thanks you.

[xhoms · Accepted solution]

@Kichoen_Lee

Your explanation is almost right.

In the age_out section:

• default = 1d -> It means the aging out engine will remove all indicators from the node that are older than one day
• interval = 1800 -> It means how often the aging out engine will be triggered to evaluate the aging out policy for indicators in this node.
• sudden_death = false -> It means indicators will remain in the node as long as they are not aged out. Regardless if they show up or not on following polls

You could add another "interval" prototype configuration attribute but at the root level (not under the age_out section). This root-level interval attribute would drive the polling engine (how often a a new AF search is performed). The AF samples miner has a hardcoded minumum interval of 3600 seconds.

• You must use the "initial_interval" root level prototype configuration attribute to control how long in the past does the first poll goes (1d, 30d, etc.)
• You must use the "default" age-out level protottype configuration attribute to control how long the indicators are kept in the node (1d, 30d, etc.)

[Kichoen_Lee]

 @xhoms

Thanks a lot for your answer.

Please let me know why you recommend letting MineMeld take care of it better than adding time-based attributes to the query.

I am considering what the best way is on both.

[xhoms]

@Kichoen_Lee,

typical use case for MineMeld in AF is to generate "live feeds" of IOC's. For instance to have a feed with the malware hashes of the discovered ransomware samples in the last 30 days.

In this case your query would be as basic as:

{"operator":"all","children":[{"field":"sample.tag_group","operator":"is","value":"Ransomware"}]}

MineMeld would perform a first query of "initial_interval" days and then keep polling once an hour. With an age_out policy of "30d" you'll get your feed ready.

[Kichoen_Lee]

@xhoms

Sorry, I don't understand it correctly.

I am just wondering which cases have advantage and disadvantage and What the better is either of the two.

Case 1 : using last 30 days in the query as below,

 {"operator":"all","children":[{"field":"sample.tag_group","operator":"is","value":"Ransomware"},{"field":"sample.update_date","operator":"is after","value":["2018-01-03T00:00:00","2018-02-01T23:59:59"]}]}

It is important "is after" above the query.

Case 2 : using miner's config

the setting age-out default will be 30 days.

I would like to keep going on that the EDL would have indicators in the last 30days and then it has to refresh.

[xhoms]

@Kichoen_Lee,

MineMeld's AF Samples Miner node always attaches the sample.update_date and sample.create_date to the query ("any" operator). And it keeps a satateful pointer remembering the last_polling time. The idea is not to poll the same indicators from AF again and again because that wastes AF resources. That's the reason why AF Samples Miner "remembers" the timestamp of the last extracted sample and keeps a pointer to that for the next poll.

You want to have a list of indicators for the last 30 days, right? You must avoid, in any case, using time-based attributes in your query (let that to MineMeld)

Option 1:

• You poll every 3600 seconds (interval = 3600) and define an age_out policy with default = 30d / interval = 3600
• initial_interval = 2592000 (30 days)

Option 2:

• You poll every 2592000 seconds and define an age_out policy with default = null / interval = 2592000 / sudden_death = true
• initial_interval = 2592000

Option 1 will refresh your feed once per hour while Option 2 will refresh it just once a month. Obviously Option 1 is much better.

[KiCheonLee]

@xhoms

Thanks a lot for your assistance.

I was glad to discuss with you and it was so helpful.

Have a great day.