Differences between stdlib.feedHCGreen and stdlib.taxiiDataFeed output miners

L2 Linker

Differences between stdlib.feedHCGreen and stdlib.taxiiDataFeed output miners




I have each of my processors set up to send to two different output miners - one for the firewalls to ingest (stdlib.feedHCGreen) and one for my SIEM to ingest (stdlib.taxiiDataFeed). I'm seeing differences between what is ending up in my SIEM and what is on the list for the firewalls. I believe feedHCGreen outputs are correct while TAXII outputs are off.


Looking at the NODES screen, the number of indicators is different between the output miners. This is consistent with 6 different processors (total of 12 miners, 6 of feedHCGreen and 6 of taxiiDataFeed). Screenshots are below showing differences in counts of indicators. 


It is quite possible that I am not understanding how TAXII works or I am doing something wrong. Is it part of the protocol that indicators automatically age out after a period of time? I didn't see anything in the prototype configs that specify an age out time. The result I am trying to accomplish is both sets of outputs having the exact same data at all times.







L7 Applicator

Re: Differences between stdlib.feedHCGreen and stdlib.taxiiDataFeed output miners

Hi Dan,

yes, there is a difference between "traditional" feeds and TAXII DataFeeds:

  • in a traditional feed indicators are removed when they are aged out by Miners. These feeds basically are static, they keep indicators until they are aged out.
  • a TAXII DataFeed instead is incremental, it is an orderd feed of indicators. The feed records all the updated indicators on a rolling window of 24 hours (configurable). It becomes then a responsability of the consumer to decide how to build the list of current live indicators. Example: in QRadar you can define a TTL to all the elements of the reference set.

There is a second type TAXII Data Collection called Data Set that is more similar to a traditional static feed. This is not supported yet, but we plan to add support in the short term.


Ref page 51 of http://taxiiproject.github.io/releases/1.1/TAXII_Services_Specification.pdf.

L2 Linker

Re: Differences between stdlib.feedHCGreen and stdlib.taxiiDataFeed output miners

Luigi, this is very helpful, thank you!

Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!

The Live Community thanks you for your participation!