- Access exclusive content
- Connect with peers
- Share your expertise
- Find support resources
Enhanced Security Measures in Place: To ensure a safer experience, we’ve implemented additional, temporary security measures for all users.
on 02-03-2016 01:59 AM - edited on 05-22-2020 08:33 AM by Retired Member
As explained in this article, you can think of a prototype as a node template that can be instantiated inside the MineMeld engine config to define a new node.
In the default MineMeld installation, prototypes are stored inside prototypes libraries located in 2 different directories:
- /opt/minemeld/prototypes/current contains the standard prototypes libraries. These are automatically updated by the MineMeld auto update mechanism.
- /opt/minemeld/local/prototypes contains local prototypes libraries. This is the directory you want to put your own prototypes.
A prototype library is a YAML file with the following structure (from dshield.yml library):
# library author, optional
author: lmori
# URL with more details, optional
url: https://www.dshield.org/xml.html
# library description, recommended
description: >
The ISC uses the DShield distributed intrusion detection system for
data collection and analysis. DShield collects data about malicious
activity from across the Internet. This data is cataloged and summarized
and can be used to discover trends in activity, confirm widespread
attacks, or assist in preparing better firewall rules.
# list of prototypes, mandatory
prototypes:
# prototype name, mandatory. Should be unique inside the library
block:
# development status, recommended
development_status: STABLE
# node type, recommended
node_type: miner
# description of the prototype, recommended
description: suggested block list
# node class, mandatory
class: minemeld.ft.http.HttpFT
# node config, recommended
config:
[...]
Inside the MineMeld engine config file prototypes are used inside the node definition. You can check the running config file /opt/minemeld/local/config/running-config.yml for an example:
nodes:
# ... more nodes here
dshield_blocklist:
output: true
prototype: dshield.block
# ... nore nodes here
A prototype is referenced as <library name>.<prototype name>.
The easiest way to customize a prototype is creating your own local version of the prototype and then use it inside the config.
As an example we will create a local version of the malwaredomainlist.ip prototype to raise the confidence of the indicators.
Copy the original library to a new library with a new unique global name in the local prototype directory
$ sudo -u minemeld cp /opt/minemeld/prototypes/current/malwaredomainlist.yml /opt/minemeld/local/prototypes/myownmalwaredomainlist.yml
And override the confidence attribute (and optionally description):
author: foobar
description: Local version of Malware Domain List library.
url: https://www.malwaredomainlist.com/
prototypes:
ip:
development_status: EXPERIMENTAL
node_type: miner
description: list of active ip addresses
class: minemeld.ft.http.HttpFT
config:
source_name: malwaredomainlist.ip
attributes:
type: IPv4
direction: outbound
share_level: green
confidence: 75
url: http://www.malwaredomainlist.com/hostslist/ip.txt
You may need to reload the UI in the browser to load the new prototype, there is no need to restart the engine or the web frontend.
Hello,
We are trying to integrate badips.com lists, but the recipe on this page does not work for us. Here is our UC3M.yml file.
We had restarted mimemeld and even the system. But no luck, it should be a simple mistake, but we are unable to find it.
Could anyone help us?
author: certuc3m description: > A comunity driven repository url: https://www.badips.com prototypes: CommunityBadIPs: development_status: STABLE node_type: miner description: badips.com is a community based IP blacklist service. class: minemeld.ft.http.HttpFT config: source_name: UC3M.2w url: https://www.badips.com/get/list/any/3?age=2w attributes: type: IPv4 direction: inbound share_level: green confidence: 50
I'm currently in the process of creating a new prototype to pull a feed from a trusted source.
The feed uses basicauth so i tried to do as following url: https://username:password@domain.tld
But no luck.
Is there a trick i need to know about when it comes to password protected feeds with BasicAuth or other types of authentication ?
Hi @Kimwii,
currently (0.9.30) plain Http Miner does not support Basic Auth, you would need to extend it to support Basic Auth.
Hi,
Do you know something sample about integration with MISP (Malware Information share platform)???
So another question is about scripts, can I launch a script into conifg a new prototype? If I've created a new prototype I set a url option...can I set the url option for script option????
Thanks a lot
I think misp has it's own feed tools.
But if you want to integrate the feed into misp i recommend using pymisp.
https://github.com/CIRCL/PyMISP
https://media.readthedocs.org/pdf/pymisp/master/pymisp.pdf
USE the api call below
add_url(event, url, category=’Network activity’, to_ids=True, comment=None, distribution=None, proposal=False)
Remembner to change your key file and add your api key for misp
Hi Kimwii
Do you refer about file config??? such us:
attributes:
confidence: 80
share_level: green
type: IPv4
delimiter: '#'
fieldnames:
- indicator
interval: 3600
source_name: misp.reputation.ip-dst
url: http://ip_myMisp/attributes/text/download/ip-dst
add_url(event, url, category=’Network activity’, to_ids=True, comment=None, distribution=None, proposal=False)
is it so??????
Thx
@lmori still having hazzle with creating a json prototype with basic auth for this format
{
"something1": [
"somedomain1.tld",
"somedomain2.tld"
],
"something2": [
"somedomain3.tld",
"somedomain4.tld"
],
"something3": [
"somedomain5.tld",
"somedomain6.tld",
]
}
Any tips or ticks ? :).
Also want to recomend minemeld anyone who'd like a easy way to manage their IOCs.
@lmori is super awsome and always helpfull.
Also is there a place i can write a review of minemeld here at paloaltos website ?.
That's the least i can do for everything you guys have done to make my day easier
HI @Kimwii,
you should create a prototype based on minemeld.ft.json.SimpleJSON (you can start from aws.AMAZON prototype), and configure it like this (note that the following prototype won't work because S3 does not accept Basic Auth headers):
age_out: default: null sudden_death: true attributes: confidence: 100 share_level: green type: domain extractor: '*[].{ indicator: @ }' indicator: indicator prefix: test source_name: test.kim url: https://s3-eu-west-1.amazonaws.com/minemeld-test/test.json username: guest password: guest
extractor and indicator are JMESPath expressions, extract should be used to extract a list of objects from the JSON document while indicator can be used to identify the indicator inside each object.
@Kimwii, I am really glad you like MineMeld. I would be happy to see an article from yours about all the cool things you are doing with Splunk & MineMeld 🙂
I understand that Soltra is part of the existing 3rd party intelligence feed, just wondering has anyone created a prototype from FS-ISAC? THe portal address is https://portal.fsisac.com/
Understand from FS-ISAC, they uses Soltra as part of their intel too, is FS-ISAC intelligence pool as subset of Soltra?
Hi @lmori,
I had opened a discussion, please find the link https://live.paloaltonetworks.com/t5/MineMeld-Discussions/Prototype-for-FS-ISAC/m-p/159162#M1132, really appreciate your help.
Regards