Note: Palo Alto Networks made an end-of-life announcement about the MineMeld™ application in AutoFocus™ on August 1, 2021. Please read this article to learn about our recommended migration options.
MineMeld was conceived as a Threat Indicator Sharing platform. Its main goal is as a tool to help users consume all sorts of threat indicators from diverse sources, aggregate them, age them, then finally make them actionable by featuring very flexible output nodes. Over time, MineMeld has incorporated Data API's and local storage for indicators, which has extended its range of use cases. In this article in the Using MineMeld... series, we'll cover the use case of MineMeld as a Incident Response Platform.
The following bullets describe the need for Incident Response we'll solve in this article:
First, visit https://live.paloaltonetworks.com/t5/MineMeld/ct-p/MineMeld and select the article (from the top right) about installing and running MineMeld appropriate to your environment. Note, if using the VMWare desktop instructions (https://live.paloaltonetworks.com/t5/MineMeld-Articles/Running-MineMeld-on-VMWare-desktop/ta-p/72038) you can go ahead with the "Super fast setup" but please download the cloud-init ISO and mount it on first boot. Assuming an IP comes via DHCP and you have internet access, your VM will automatically be updated to the latest version of Minemeld.
Make note of MineMelds IP address (from an ifconfig) and login from your browser (defaults to username: admin / password: minemeld)
At the end of this article you'll find copy&paste ready snipets. But we highly encourage you to follow the steps to get used to MineMeld's WEB UI features and procedures.
Let's start with a blank configuration. The following is the summary of steps we'll take:
To start we must reach the prototype library that is available by navigating to small icon in the down left part of the Config Tab.
In this example we'll trust all indicators coming from VirusTotal and WildFire. This means that any SHA256 hash delivered by these miners will have a value of 100 in the confidence attribute. So we can just clone the stdlib.feedHCGreenWithValue prototype that will accept all indicators with confidence > 75 shared in the green level.
We can, as well, clone the stdlib.aggregatorSHA256 from the prototype library into our graph.
The VirusTotal miner prototype in the library is still tagged as experimental which means that all indicators used by it will be placed in the red share level. As our output node only accepts indicators in the green share level, we must create a new prototype using the one in the library as a base but chaging the share level attribute value.
Note that the prototype in the library shares in the red channel. Click on "new" to create a new prototype that shares in the green level instead.
Now it is time to clone this recently created prototype into our graph.
We, finally, need a miner node that could receive async events (WildFire logs from a PANOS device) and store them into a local database. This miner must also provide aging for the indicators so they are removed from the list at a given expiration time. This is what the class minemeld.ft.localdb.Miner exposed through the stdlib.localDB prototype is all about.
As in the case of the VirusTotal prototype, this node is tagged as experimental which means we must first create a new prototype to share indicators in the green level before we can clone it into our graph.
At this moment our configuration should look like the one in the following picture. Note that the nodes are not linked. We can bind them together by clicking on the input cell of both the aggregator and the output node.
Once all nodes are conneted we just need to commit the configuration to get our working graph.
The graph can be shown by clicking on any node in the Nodes Tab and selecting the graph view.
A good practice is to use authentication in the output nodes so only approved clients can grab indicators from it. MineMeld features a very flexible feed authentication mechanism: The administrator can create "feed users" that can latter be attached as "tags" to the output nodes. A given output node can host multiple tags which means it will accepts connections from more that one customer.
First of all let's access the feed user panel available in the Admin main tab. Once there, a new feed user can be added by clicking on the plus icon.
The feed user must then be attached to the output node. That can be achieved by clicking on the output node from the Nodes main Tab. In the node's status pannel you must click on the TAG row to attach the feed user authentication to that node.
We are all set for now. We just need the miners to start gathering indicators for us.
You might have noticed the small error icon besides the VirusTotal miner node. That error indicates that the node is not fully configured. A mandatory attribute is not yet provided: The VirusTotal Intelligece API Key.
The key can be obained from the VirusTotal page under the profile settings. Note, please, that the API Key must come from a Virus Total Account with Intelligence access permissions. If you're not familiar with VirusTotal malware hunting features please visit https://www.virustotal.com/intelligence/help/malware-hunting/ before using the VT miner.
Once you have copied the key to your clipboard just paste it into the API KEY field available in the status tab of the virustotal node panel.
No commit is needed. Just force a polling by clicking on the icon at the end right of the LAST RUN field. If everything is right the indicators field will be updated with a non zero value. If you want to take a look to these indicators just navigate to the output node an click on the FEED BASE URL to get a plain text version of the list.
The last step in our mission today is to capture WildFire events from our PANOS device to extract SHA256 indicators out of them. To achieve this we'll leverage the HTTP Forwarding feature introduced with PANOS 8.0.
The following is our theory of operations:
First of all we need a new HTTP Server Profile for the MineMeld. We must provide:
The localDB miners's API accepts payloads either in JSON or PlainText format. In any case the type and indicator fields are the only mandatory ones. Optionals fields are:
A couple of comments in the payload format shown in the picture above:
The next step is to create a new log forwarding object to feature this collection of HTTP server and payload format we've just defined.
Finally we must attach this log forwarding object to the security policy rules that features WildFire inspection.
If you do not want to wait for WildFire incidents just trigger a new one by downloading the WildFire test PE file. Once you get your incident just confirm that it is available in the MineMeld's output feed.
MineMeld's feed output node, by default, provides the content in a plain text format. But you can modify this behavior with additional attributes as explained in Parameters for the output node to easy the consumption by third parties. And, why not, contribute to the MineMeld OpenSource project with new output nodes like, McAfee DXL (TIE) or Tanium.
Local Prototypes used in this use case (/opt/minemeld/local/prototypes/minemeldlocal.yml)
author: minemeld-web
description: Local prototype library managed via MineMeld WebUI
prototypes:
localDBGreen:
class: minemeld.ft.localdb.Miner
config:
attributes:
confidence: 100
share_level: green
interval: 3600
description: 'list of indicators. Use a name starting with "wl" to create
a whitelist for an aggregator
'
development_status: EXPERIMENTAL
indicator_types:
- any
node_type: miner
tags:
- ConfidenceAny
- ShareLevelAny
virusTotalShareGreen:
class: minemeld.ft.vt.Notifications
config:
age_out:
default: 365d
interval: 7200
sudden_death: false
attributes:
confidence: 100
share_level: green
description: 'Miner for VirusTotal Intelligence Notifications feed.
'
development_status: EXPERIMENTAL
indicator_types:
- md5
- sha256
- sha1
node_type: miner
tags:
- ConfidenceHigh
- ShareLevelRed
Graph configuration (Config -> Import)
nodes:
VirusTotal:
inputs: []
output: true
prototype: minemeldlocal.virusTotalShareGreen
WildFireEvent:
inputs: []
output: true
prototype: minemeldlocal.localDBGreen
sha256agg:
indicator_types:
- sha256
inputs:
- VirusTotal
- WildFireEvent
node_type: processor
output: true
prototype: stdlib.aggregatorSHA256
sha256out:
indicator_types:
- any
inputs:
- sha256agg
node_type: output
output: false
prototype: stdlib.feedHCGreenWithValue
In MineMeld 0.9.42 we have introduced a new type of Miner (stdlib.localDB) exposing the HTTP-based API used in this article. In the following examples we assume that you have created a localDB Miner called FooBar.
You can push new indicators using JSON. The ttl attribute is the expiration time in seconds of the indicator. If omitted the default config value of the Miner will be used. Only the indicator and type attributes are mandatory.
curl -XPOST -H "Content-Type: application/json" -u admin:minemeld "https://192.168.55.150/config/data/FooBar_indicators/append?h=FooBar&t=localdb" -d '
{
"indicator": "8.8.8.8",
"type": "IPv4",
"comment": "usual Google DNS Public IP",
"share_level": "green",
"confidence": 100,
"ttl": 3600
}'
You can use the same API call to delete an indicator, by setting the ttl attribute to 0:
curl -XPOST -H "Content-Type: application/json" -u admin:minemeld "https://192.168.55.150/config/data/FooBar_indicators/append?h=FooBar&t=localdb" -d '
{
"indicator": "8.8.8.8",
"type": "IPv4",
"ttl": 0
}'
You can also use a simple text format to push a list of indicators:
Example:
curl -XPOST -H "Content-Type: application/text" -u admin:minemeld "https://192.168.55.150/config/data/FooBar_indicators/append?h=FooBar&t=localdb" -d '
URL
https://www.google.com
comment
test comment
confidence
80
share_level
green
ttl
1800
IPv4
8.8.8.8
comment
usual Google DNS Public IP
share_level
green
confidence
100
ttl
86400
'
You can use the same API call to delete an indicator, by setting the ttl attribute to 0:
curl -XPOST -H "Content-Type: application/text" -u admin:minemeld "https://192.168.55.150/config/data/FooBar_indicators/append?h=FooBar&t=localdb" -d '
URL
https://www.google.com
ttl
0
IPv4
8.8.8.8
ttl
0
'
