Could you tell me why taxii output doesn't do data deduplication?
Is it normal behaviour or bag?
This problem is very important for us because we have huge amount of IOCs (about 450K).
TAXII output just multiply this list.
Additionally after the output toked 1000000 IOCs it just stop to accept new data until deletion of some old IOCs.
The screanshot in attachment.
Solved! Go to Solution.
TAXII DataFeed is not "classical" feed where you can find only the active list of indicators. A TAXII DataFeed stores all the updates to the indicators. So if indicator A is updated 3 times, the TAXII DataFeed output will contain 3 indicators A with different attributes with a timestamp of when the update happened.
By default the entries in the TAXII DataFeed are removed when older than 24 hours, to store more updates you can:
- decrease the age out interval using the age_out_interval config knob in the prototype. Example:
- increase the number of entries that can be store in the feed (watch the memory usage !):
So, I think it's normal.
Before I save the config it looks like this:
But when it was saved, it look different.
Click Accept as Solution to acknowledge that the answer to your question has been provided.
The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!
These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!
The Live Community thanks you for your participation!