Minemeld Ageout Policys and Withdraw

Reply
Highlighted
L0 Member

Minemeld Ageout Policys and Withdraw

Hello,

 

Im having several issues and questions about what the best practices would be for surronding ageout policys.

 

  • Is it better to add an ageout policy to the Miners, Aggregators, or Outputs?
  • If I use the following Ageout policy, if a feed sends an IP right after the age-out occurs, will the first_seen time start over?  
    • age_out:
          default: first_seen+20d
          interval: 600
          sudden_death: true
  • I have a TAXII feed that is currently ignoring all withdrawal requests, but I cannot figure out why it is doing so.  Its using the base minemeld.ft.taxii.Datafeed class, and all the miners prior to it have the same ageout policy as the one above.  
L1 Bithead

Re: Minemeld Ageout Policys and Withdraw

@lmori 

 

Having issues with age-out policy on miners.

 

Any answers for the questions above? Anything new? 

 

Please, advise and thank you very much for your time!

L1 Bithead

Re: Minemeld Ageout Policys and Withdraw

Hello,

 

Still going through configurations via trial-and-error approach as the indicator numbers are not looking correct at all. 

 

Reviewing configuration for Output of class minemeld.ft.taxii.DataFeed

Added the following config:

infilters:
- actions:
- accept
conditions:
- __method == 'withdraw'
name: accept withdraws
- actions:
- accept
conditions:
- type == 'IPv4'
name: accept IPv4
- actions:
- drop
name: drop all

Would it be helpful to add "store_value: true" and how?

 

Is there anything that may be helpful to add to an OUTPUT configuration to make sure the node (somehow) does not keep accumulating indicators after every update and ignores indicators that have already been seen/aged-out ?

 

Any and all assistance or feedback is very much appreciated. 

L1 Bithead

Re: Minemeld Ageout Policys and Withdraw

@lmori  
@xhoms 
Hello,

There are still unanswered questions regarding aging of indicators, as well as minemeld working output configuration.

  • Can Minemeld function with “first_seen+2d” even if the indicator is still present in the feed?
  • If “first_seen+2d” is accepted, and the miner pulls the feed again with the indicator still present what happens to the indicator?

We are also trying to understand behaviors showing in our Minemeld instance such as:

                Miner node #1 has 7413 indicators

                Miner node #2 has 783 indicators

                Processor, with Miner node #1 and Miner node #2 as input, has 8196 indicators

                Output (minemeld.ft.redis.RedisSet) has 7413 indicators

                Same processor with different Output (minemeld.ft.taxii.DataFeed) and it currently has 587479 indicators – this number keeps on growing as indicators just keep getting added on until the next service restart.

Any advice, guides or hints are much appreciated and thank you very much for your time and assistance.

Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!

The Live Community thanks you for your participation!