MineMeld Speedtest.net host mining

L4 Transporter

MineMeld Speedtest.net host mining

Hi all,

 

I managed to install MineMeld on-prem and are playing around with it now. As a first task I'd like to setup a domain feed delivering SpeedTest.net hosts from countries we deployed Palo Alto firewalls.

 

Setting up the miner, a domain aggregator and an output worked, no problem there. We download the full hosts list from http://c.speedtest.net/speedtest-servers-static.php and pass it through to an output feed. However, I don't need all the 7k+ hosts mined. Only those with specific two-digit country codes, such as "CH" or "US". That information is also present in the mined XML file and I tried to fetch this information using "fields" in the input config (field 'countrycode'):

 

age_out:
  default: null
  interval: 631
  sudden_death: true
attributes:
  confidence: 100
  direction: inbound
  share_level: green
  type: domain
fields:
  country: 

    regex: country="([\w\s]*)"
    transform: \1
  countrycode:
    regex: cc="(\w){2}"
    transform: \1
ignore_regex: ^\<settings\>|^\<servers\>
indicator:
  regex: host="(.*):8080"
  transform: \1
interval: 3307
source_name: speedtest.hosts
url: http://c.speedtest.net/speedtest-servers-static.php

 

Now I want to filter for this 'countrycode' field in the output node and created a new prototype for this (condition #2):

 

infilters:
- actions:
- accept
conditions:
- __method == 'withdraw'
name: accept withdraws
- actions:
- accept
conditions:
- countrycode == 'CH'
- share_level == 'green'
name: accept share level green
- actions:
- drop
name: drop all

 

Result: 0 indicators in that output feed. Thanks for any hints on this. I'd like to avoid creating a miner for every country.

 

Thanks,

Oliver

L4 Transporter

Re: MineMeld Speedtest.net host mining

Implemented the following solution now. Filtering all the countries we need directly in the input node. However, it could be desirable to filter the output instead of input.

 

age_out:
default: null
interval: last_seen+900
sudden_death: true
attributes:
confidence: 100
direction: inbound
share_level: green
type: domain
ignore_regex: ^\<settings\>|^\<servers\>
indicator:
regex: cc="(CH|US|<some more countries>)".*host="(.*):8080"
transform: \2
interval: 86400
source_name: speedtest.hosts
url: http://c.speedtest.net/speedtest-servers-static.php

Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!

The Live Community thanks you for your participation!