The set of config parameters supported by a node depends on the node class. Node configs are stored inside prototypes.

Base class

All nodes support these parameters.

Parameters

Filter set

Each filter set is a list of filters. Filters are checked from top to bottom, the first matching filter is applied and following filters are not checked. Default action is accept. Each filter is a dictionary with 3 keys:

In addition to the atttributes in the indicator value, filters can match on 3 special attributes:

Condition

A condition in the filter is a boolean expression composed by: a JMESPath expression, an operator (<, <=, ==, >=, >, !=) and a value.

Example

Example config in YAML:

infilters:
    - name: accept withdraws
      conditions:
        - __method == 'withdraw'
      actions:
        - accept
    - name: accept URL
      conditions:
        - type == 'URL'
      actions:
        - accept
    - name: drop all
      actions:
        - drop
outfilters:
    - name: accept all (default)
      actions:
        - accept

Base poller class

In addition to Base class config parameters, the base poller class supports the following parameters.

Config parameters

Age out policy

Age out policy is described by a dictionary with at least 3 keys:

Additional keys can be used to specify age out interval per indicator type.

Age out interval

Age out intervals have the following format:

<base attribute>+<interval>

base attribute can be last_seen, if the age out interval should be calculated based on the last time the indicator was found in the feed, or first_seen, if instead the age out interval should be based on the time the indicator was first seen in the feed. If not specified first_seen is used.

interval is the length of the interval expressed in seconds. Suffixes d, h and m can be used to specify days, hours or minutes.

Example

Example config in YAML for a feed where indicators should be aged out only when they are removed from the feed:

source_name: example.persistent_feed
interval: 600
age_out:
    default: null
    sudden_death: true
    interval: 300
attributes:
    type: IPv4
    confidence: 100
    share_level: green
    direction: inbound

Example config in YAML for a feed where indicators are aged out when they disappear from the feed and 30 days after they have seen for the first time in the feed:

source_name: example.long_running_feed
interval: 3600
age_out:
    default: first_seen+30d
    sudden_death: true
    interval: 1800
attributes:
    type: URL
    confidence: 50
    share_level: green

Example config in YAML for a feed where indicators are aged 30 days after they have seen for the last time in the feed:

source_name: example.delta_feed
interval: 3600
age_out:
    default: last_seen+30d
    sudden_death: false
    interval: 1800
attributes:
    type: URL
    confidence: 50
    share_level: green

minemeld.ft.http.HttpFT class

In addition to Base poller class config parameters, the HttpFT class supports the following parameters:

Parameters

Extraction dictionary

Extraction dictionaries contain the following keys:

See Python re module for details about Python regular expressions and templates.

Example

Example config in YAML where extraction dictionaries are used to extract the indicator and additional fields:

url: https://www.dshield.org/block.txt
ignore_regex: "[#S].*"
indicator:
    regex: '^([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})\t([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})'
    transform: '\1-\2'
fields:
    dshield_nattacks:
        regex: '^.*\t.*\t[0-9]+\t([0-9]+)'
        transform: '\1'
    dshield_name:
        regex: '^.*\t.*\t[0-9]+\t[0-9]+\t([^\t]+)'
        transform: '\1'
    dshield_country:
        regex: '^.*\t.*\t[0-9]+\t[0-9]+\t[^\t]+\t([A-Z]+)'
        transform: '\1'
    dshield_email:
        regex: '^.*\t.*\t[0-9]+\t[0-9]+\t[^\t]+\t[A-Z]+\t(\S+)'
        transform: '\1'

Example config in YAML where the text in each line until the first whitespace is used as indicator:

url: https://ransomwaretracker.abuse.ch/downloads/CW_C2_URLBL.txt
ignore_regex: '^#'

For a complete config example check dshield.block prototype.

minemeld.ft.csv.CSVFT class

In addition to Base poller class config parameters, the CSVFT class supports the following parameters:

Parameters

Example

Example config in YAML:

url: https://sslbl.abuse.ch/blacklist/sslipblacklist.csv
ignore_regex: '^#'
fieldnames:
    - indicator
    - port
    - sslblabusech_type

For a complete config example check sslabusech.ipblacklist prototype.

minemeld.ft.json.SimpleJSON class

In addition to Base poller class config parameters, the SimpleJSON class support the following parameters:

Parameters

Example

Example config in YAML:

url: https://ip-ranges.amazonaws.com/ip-ranges.json
extractor: "prefixes[?service=='AMAZON']"
prefix: aws
indicator: ip_prefix
fields:
    - region
    - service

For a complete config example check aws.AMAZON prototype.