Custom url feeds

Reply
L3 Networker

Custom url feeds

Is there any sort of documentation surrounding things like adding a custom url in? I'm thinking I'll have to dig into the file system but was wondering if there is anything documented as to what to do.

 

Say I have an ip list at http://somefancywebsite.com/directory/badiplist.txt I wanted to throw into the mix here. Is there an easy way to do this in the gui or do I need to go hit the file system?

L7 Applicator

Re: Custom url feeds

That is usually possible by defining a new prototype using the Web UI.

 

Do you have a specific example ?

 

 

Highlighted
L3 Networker

Re: Custom url feeds

Nothing specific. More trying to understand how the system works.

L7 Applicator

Re: Custom url feeds

To add a new feed you should start from the following details:

  1. what are the format and protocol used by the feed ?
  2. what expiration policy should I apply to the indicators ?
  3. what confidence level should I use for the indicators ?

Question 1) defines the class of Miner you want to use. Currently there are classes supporting plain text feeds over HTTP/HTTPS, JSON over HTTP/HTTPS, CSV over HTTP/HTTPS, STIX/TAXII, and a number of other classes for specific public or commercial API. If the protocol and format used by the feed are not covered by one of the existing classes you should write your own Python class. Most of the times it's pretty easy, details here: https://github.com/PaloAltoNetworks/minemeld/wiki/How-To-Write-a-Simple-Miner

 

If instead the protocol and format are already covered, you don't need to write a single line of code. You can just write a prototype, i.e. a config for the Miner. In the Web UI go under CONFIG and click the browse button (the 3 stackd lines). Select a prototype for a feed similar to the one you want to add and click on the NEW button in the top right corner. This will create a private copy of the prototype you can modify. Now you can change the config of the Miner and specify new parameters, like URL, age out policy, confidence level, new attributes, ...

Additional details about prototypes here: 

https://github.com/PaloAltoNetworks/minemeld-core/blob/master/docs/nodeconfig.rst

https://live.paloaltonetworks.com/t5/MineMeld-Articles/What-is-in-a-MineMeld-node/ta-p/72046

Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!

The Live Community thanks you for your participation!