Customizing Prototypes

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements
L7 Applicator
100% helpful (3/3)

As explained in this article, you can think of a prototype as a node template that can be instantiated inside the MineMeld engine config to define a new node. 

 

Prototype Libraries

In the default MineMeld installation, prototypes are stored inside prototypes libraries located in 2 different directories:

- /opt/minemeld/prototypes/current contains the standard prototypes libraries. These are automatically updated by the MineMeld auto update mechanism.

- /opt/minemeld/local/prototypes contains local prototypes libraries. This is the directory you want to put your own prototypes. 

 

A prototype library is a YAML file with the following structure (from dshield.yml library):

# library author, optional
author: lmori
# URL with more details, optional
url: https://www.dshield.org/xml.html
# library description, recommended
description: >
The ISC uses the DShield distributed intrusion detection system for
data collection and analysis. DShield collects data about malicious
activity from across the Internet. This data is cataloged and summarized
and can be used to discover trends in activity, confirm widespread
attacks, or assist in preparing better firewall rules.

# list of prototypes, mandatory
prototypes:
# prototype name, mandatory. Should be unique inside the library
block:
# development status, recommended
development_status: STABLE
# node type, recommended
node_type: miner
# description of the prototype, recommended
description: suggested block list
# node class, mandatory
class: minemeld.ft.http.HttpFT
# node config, recommended
config:
[...]

 

Inside the MineMeld Engine Config

Inside the MineMeld engine config file prototypes are used inside the node definition. You can check the running config file /opt/minemeld/local/config/running-config.yml for an example:

nodes:
# ... more nodes here
dshield_blocklist:
output: true
prototype: dshield.block
# ... nore nodes here

 

A prototype is referenced as <library name>.<prototype name>.

 

Customizing a Prototype

The easiest way to customize a prototype is creating your own local version of the prototype and then use it inside the config.

 

As an example we will create a local version of the malwaredomainlist.ip prototype to raise the confidence of the indicators.

 

1. Copy the library to the local prototype directory

Copy the original library to a new library with a new unique global name in the local prototype directory

$ sudo -u minemeld cp /opt/minemeld/prototypes/current/malwaredomainlist.yml /opt/minemeld/local/prototypes/myownmalwaredomainlist.yml

 

2. Edit the new library

And override the confidence attribute (and optionally description):

author: foobar
description: Local version of Malware Domain List library.
url: https://www.malwaredomainlist.com/

prototypes:
ip:
development_status: EXPERIMENTAL
node_type: miner
description: list of active ip addresses
class: minemeld.ft.http.HttpFT
config:
source_name: malwaredomainlist.ip
attributes:
type: IPv4
direction: outbound
share_level: green
confidence: 75
url: http://www.malwaredomainlist.com/hostslist/ip.txt

 

3. Use the new prototype inside the UI

You may need to reload the UI in the browser to load the new prototype, there is no need to restart the engine or the web frontend.

Screen Shot 2016-01-06 at 22.24.14.png

Rate this article:
Comments
L0 Member

Hello,

 

We are trying to integrate badips.com lists, but the recipe on this page does not work for us. Here is our UC3M.yml file.

We had restarted mimemeld and even the system. But no luck, it should be a simple mistake, but we are unable to find it.

 

Could anyone help us?

 

 

author: certuc3m
description: >
       A comunity driven repository
url: https://www.badips.com

prototypes:
CommunityBadIPs:
development_status: STABLE
node_type: miner
description: badips.com is a community based IP blacklist service.
class: minemeld.ft.http.HttpFT
config:
source_name: UC3M.2w
url: https://www.badips.com/get/list/any/3?age=2w
attributes:
type: IPv4
direction: inbound
share_level: green
confidence: 50


 

L7 Applicator

Hi netmaster_UC3M,

please, could you open a discussion with your prototype under MineMeld discussions ? I'll be happy to help.

L1 Bithead

I'm currently in the process of creating a new prototype to pull a feed from a trusted source.

The feed uses basicauth so i tried to do as following url: https://username:password@domain.tld

But no luck.

Is there a trick i need to know about when it comes to password protected feeds with BasicAuth or other types of authentication ?

L7 Applicator

Hi @Kimwii,

currently (0.9.30) plain Http Miner does not support Basic Auth, you would need to extend it to support Basic Auth.

L1 Bithead

Now minemeld is integrated with Autofocus. How can we update custom prototype from UI?

 

Is there any option in UI to make changes to existing prototype?

L7 Applicator

Hi @hsolis,

yes, just click on an existing prototype and then click NEW in the top right corner. This will create a copy of the current prototype you can customize and save.

L2 Linker

Hi,

 

Do you know something sample about integration with MISP (Malware Information share platform)???

 

So another question is about scripts, can I launch a script into conifg a new prototype? If I've created a new prototype I set a url option...can I set the url option for script option????

 

Thanks a lot

L1 Bithead

I think misp has it's own feed tools.
But if you want to integrate the feed into misp i recommend using pymisp.

https://github.com/CIRCL/PyMISP

https://media.readthedocs.org/pdf/pymisp/master/pymisp.pdf

 

USE the api call below 

add_url(event, url, category=’Network activity’, to_ids=True, comment=None, distribution=None, proposal=False)

 

Remembner to change your key file and add your api key for misp

 

L2 Linker

Hi Kimwii

 

Do you refer about file config??? such us:

 

attributes:
confidence: 80
share_level: green
type: IPv4
delimiter: '#'
fieldnames:
- indicator
interval: 3600
source_name: misp.reputation.ip-dst
url: http://ip_myMisp/attributes/text/download/ip-dst
add_url(event, url, category=’Network activity’, to_ids=True, comment=None, distribution=None, proposal=False)

 

minemeld.PNG

 

 

is it so??????

 

Thx

L2 Linker

So I want to integrate MISP into Minemeld, I don't want to integrate Minemeld into MISP.

 

How I can integrate MISP into Minemeld??????

 

Regards!

L1 Bithead

@lmori hi again.
We talked about feeds an basicauth some timeago and i wondered how do i get minemeld to "support" basic auth feeds ?.

Is there som magic trick or is it an rfc

L7 Applicator

Hi @Kimwii,

this should be addressed and supported in the next release.

L1 Bithead

@lmori still having hazzle with creating a json prototype with basic auth for this format

 

{
  "something1": [
    "somedomain1.tld",
    "somedomain2.tld"
  ],
  "something2": [
    "somedomain3.tld",
    "somedomain4.tld"
  ],
  "something3": [
    "somedomain5.tld",
 "somedomain6.tld",
  ]
}

 

Any tips or ticks ? :).

 

Also  want to recomend minemeld anyone who'd like a easy way to manage their IOCs.

@lmori is super awsome and always helpfull.

 

Also is there a place i can write a review of minemeld here at paloaltos website ?.
That's the least i can do for everything you guys have done to make my day easier

L7 Applicator

HI @Kimwii,

you should create a prototype based on minemeld.ft.json.SimpleJSON (you can start from aws.AMAZON prototype), and configure it like this (note that the following prototype won't work because S3 does not accept Basic Auth headers):

 

age_out:
    default: null
    sudden_death: true
attributes:
    confidence: 100
    share_level: green
    type: domain
extractor: '*[].{ indicator: @ }'
indicator: indicator
prefix: test
source_name: test.kim
url: https://s3-eu-west-1.amazonaws.com/minemeld-test/test.json
username: guest
password: guest

extractor and indicator are JMESPath expressions, extract should be used to extract a list of objects from the JSON document while indicator can be used to identify the indicator inside each object.

 

@Kimwii, I am really glad you like MineMeld. I would be happy to see an article from yours about all the cool things you are doing with Splunk & MineMeld 🙂

L1 Bithead

I understand that Soltra is part of the existing 3rd party intelligence feed, just wondering has anyone created a prototype from FS-ISAC? THe portal address is https://portal.fsisac.com/

 

Understand from FS-ISAC, they uses Soltra as part of their intel too, is FS-ISAC intelligence pool as subset of Soltra?

L7 Applicator

Hi @c_cong,

please, could you open a discussion under MineMeld Discussions about this ? Will be happy to help you.

L1 Bithead

Hi @lmori,

 

I had opened a discussion, please find the link https://live.paloaltonetworks.com/t5/MineMeld-Discussions/Prototype-for-FS-ISAC/m-p/159162#M1132, really appreciate your help. 

 

Regards

L1 Bithead

so, everytime i try adding ibm xfe node gitvrepository to minemeld extensions, i get timeout errors, can someone help?

  • 46564 Views
  • 18 comments
  • 1 Likes
Register or Sign-in
Contributors
Labels
Article Dashboard
Version history
Last Updated:
‎05-22-2020 08:33 AM
Updated by:
Retired Member