TaxiiDataFeed - Aging out of Feed

[rafy92]

Hi Guys,

using as prototype the "stdlib.taxiiDataFeed" I've exposed through Minemeld a TAXII Feed. 

Now i've observed that this prototype is the only that can't be aged out, in fact the IoCs collected from the sources comes in addition to those already present in the Feed.

Is there a functionality to enable the aging out of the Output (stdlib.taxiiDataFeed)? 

This question is asked me by more customers.

Waiting for your feedback.

Regards,

R.

[lmori]

Hi @rafy92,

due to its semantic different from others feed formats, TAXII DataFeed has its own internal age out. By default it ages updates older than 24 hours. One thing to remember is that the TAXII DataFeed records all the updates of the indicators, that means that if an indicators has been updated 1K times in the last 24 hours there will be 1K entries for that indicators with different timestamps in the TAXII DataFeed. This is based on TAXII 1.1 standard.

You can change the age out by modifying the *age_out_interval* value in the prototype.

[rafy92]

Hi @lmori,

many thanks for your support but I don't understand why the field "removing" is always equal to 0 if the aging out related to TaxiiDataFeed is by default set to 24h.

In addition to the previous point, as you can see in the attached image, the miner do aging out correctly after a timeframe setting by me while the output (stdlib.taxiiDataFeed) not seems to remove the IoCs from feed after 24h.

Waiting for your feedback

Regards,

R.

[lmori]

Hi @rafy92,

thanks for you feedback, I found the issue. This should be fixed in the next release.

Luigi

[rafy92]

Hi @lmori,

thank you!

Please, let me know when the new release will be released.

Regards,

R.

[rafy92]

Hi @lmori,

in Minemeld version 0.9.50 i've the same issue.

Could you please support me?

 

Thanks in advance!

Regards

[lmori]

Hi @rafy92,

please could you check the stats page of the output node and see the historical data of the number of indicators over the past 7 days?

[rafy92]

Hi @lmori,

sorry for the delay. Today i've doing some test and the results are these:

1. I add to my node some IoCs and the node and the output (Taxii Feed) perform the update correctly.

2. I remove some IoCs from node source and the node perform the update correctly but the output show this result --> Case two image

3. Sometimes the output aging out all IoCs inside even if the node is not empty --> Case three.

Let me know pls.

Thank you for your support!