Miner for host file format.

Reply
L0 Member

Miner for host file format.

Is there a miner + documentation on how to get it working for a host file list?

i.e.

https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts

 

This tool was recommended by Palo Alto for a project we are working on, howver the documentation on how to actually use it is hard to understand.

 

L5 Sessionator

Re: Miner for host file format.

Hi @DatacomNetadmin,

 

you can use the generic HttpFT class miner for such a lists published through HTTP/S. The following are the steps to mine the list at https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts

 

A- Create a new prototype for StevenBlack's list

  1. Locate in the configuration any prototype using the HttpFT class. For instance the "auscert.1day_dumpsites" one.
  2. Click on "new" to create a new prototype and name it "StevenBlack" (or anything else that suits you)
    2018-01-29_08-44-55.png

     

  3. Replace the configuration of the new prototype with this one:
    age_out:
        default: null
        interval: 3600
        sudden_death: true
    attributes:
        confidence: 100
        direction: inbound
        interval: 3600
        share_level: green
        type: domain
    ignore_regex: ^#
    indicator:
        regex: ^0\.0\.0\.0[\s\t](.*\.[a-z]{2,})$
        transform: \1
    source_name: StevenBlack
    url: https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts

B- Clone the just created prototype into a working node.

Just locate the new prototype in the configuration and use the "Clone" option.2018-01-29_08-49-36.png

 

 

 

L0 Member

Re: Miner for host file format.

Hi xhoms,

 

Thank you,

 

I can  see where I got my miner wrong.

I had the wrong indicator type (URL), I changed it to domain and changed the aggergator to suit.

 

I have it working now using the stdlib.aggregatorDomain aggregator and the stdlib.feedHCGreen protype for the output.


Again, much appreciated.

 

DaveC

Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!

The Live Community thanks you for your participation!