What is in a MineMeld Node?

[lmori]

In MineMeld, you can configure how indicators are retrieved, processed and presented by configuring the MineMeld engine graph. The graph is composed by one or more interconnected nodes.

The definition of a node in the graph is composed by:

- Name. A unique node name.

- Inputs. A list of nodes the node should receive messages from.

- Output. A boolean value that enables/disables messages to downstream nodes.

- Class. Defines which kind of processing is applied to indicators.

- Config. Configuration of the node class.

Node Inputs

A node can have zero or more input nodes. The node will listen to and process messages from the input nodes. Messages between nodes can be used to publish/update indicators, or to withdraw indicators.

In the following graph, the node inboundaggregator has spamhaus_DROP, spamhaus_EDROP and dshield_blocklist as input nodes. If a node has no input nodes it is considered a Miner. In the following graph spamhaus_DROP, spamhaus_EDROP and dshield_blocklist have no input nodes because they are Miners.

Node Output

A node can be configured to send messages to downstream connected nodes. In the previous picture spamhaus_DROP, spamhaus_EDROP, dshield_blocklist and inboundaggregator have their Ouput enabled.

Output nodes have their Output disabled - ok, this is a bit misleading - because they don't have downstream nodes. inboundfeedhc, inboundfeedlc, inboundfeedmc are Output nodes and their output is disabled.

Node Class

The node class defines what is actually done by the node. In the previous graph spamhaus_DROP, spamhaus_EDROP and dshield_blocklist are all nodes of class minemeld.ft.http.HttpFT, this class of nodes manage feeds of indicators retrievable via URLs and send updates to downstream nodes. The feed URL and how the indicators should be extracted are defined in each node config. inboundaggregator instead is a node of class minemeld.ft.ipop.AggregateIPv4FT. This class of nodes aggregate IPv4 indicators from multiple input nodes and generate a set of non-overlapping IPv4 indicators.

Node Config

The node config defines the parameters that should be used by the node class to perform its job. As an example, for the node dshield_blocklist the config defines the URL that should be used to retrieve the list of indicators, the patterns to extract the indicators and the age out policies to apply.

Node Prototype

MineMeld comes with a library of node prototypes. A prototype is a semi-pre-canned node definition, and it is composed by a node class and a node config. Prototypes are available for the most commonly used feeds, processors and output nodes.

As an example, this is the zeustracker.badips prototype:

 badips:
     development_status: STABLE
     node_type: miner
     config:
         source_name: zeustracker.badips
         attributes:
             type: IPv4
             direction: outbound
             confidence: 100
             share_level: green
         ignore_regex: '#'
         url: https://zeustracker.abuse.ch/blocklist.php?download=badips
         interval: 1800
     class: minemeld.ft.http.HttpFT