Note: Palo Alto Networks made an end-of-life announcement about the MineMeld™ application in AutoFocus™ on August 1, 2021. Please read this article to learn about our recommended migration options. MineMeld was conceived as a Threat Indicator Sharing platform. Its main goal is as  a tool to help users consume all sorts of threat indicators from diverse sources, aggregate them, age them, then finally make them actionable by featuring very flexible output nodes. Over time, MineMeld has incorporated Data API's and local storage for indicators, which has extended its range of use cases. In this article in the Using MineMeld... series, we'll cover the use case of MineMeld as a Incident Response Platform.   The following bullets describe the need for Incident Response we'll solve in this article: You own a VirusTotal Intelligence account that you use to perform hunting for malware samples with YARA rules. The corresponding VirusTotal notification service provides you with fresh malware incidents (SHA256 hashes) seen on its community of users that matches the YARA rules you've provided. You have deployed PA-Series Next Generation Firewalls for segmentation (either Internet Gateway or Datacenter) with WildFire subscription that unmask unknown malware crossing your network (incident) in less than 5 minutes. A set of endpoints is secured with TRAPS (effectively blocking zero day malware). But you also take care of a large base of endpoints protected by legacy AV solutions that do not provide protections against zero day malware. But that same AV solution can block execution of binaries provided someone could keep a list of SHA256 hashes belonging to malicious files that should be blacklisted. So you want MineMeld to provide a response keeping an updated output node with malware hashes coming both from VT and WildFire incidents that will be consumed by the legagy AV solution. This way you buy time to execute the migration of the large installed base to TRAPS. Step 1. Deploy MineMeld First, visit https://live.paloaltonetworks.com/t5/MineMeld/ct-p/MineMeld and select the article (from the top right) about installing and running MineMeld appropriate to your environment. Note, if using the VMWare desktop instructions (https://live.paloaltonetworks.com/t5/MineMeld-Articles/Running-MineMeld-on-VMWare-desktop/ta-p/72038) you can go ahead with the "Super fast setup" but please download the cloud-init ISO and mount it on first boot. Assuming an IP comes via DHCP and you have internet access, your VM will automatically be updated  to the latest version of Minemeld.   Make note of MineMelds IP address (from an ifconfig) and login from your browser (defaults to username: admin / password: minemeld)   Step 2. Create the MineMeld Graph At the end of this article you'll find copy&paste ready snipets. But we highly encourage you to follow the steps to get used to MineMeld's WEB UI features and procedures.   Let's start with a blank configuration. The following is the summary of steps we'll take: Clone the out-of-the-box stdlib.feedHCGreenWithValue prototype into our graph as output node. Clone the out-of-the-box stdlib.aggregatorSHA256 prototype into our graph as aggregator node. Create a new prototype based on the virustotal.notifications one to be able to share VT intelligence into the green share level. Create a new prototype based on the stdlib.localDB one to be able to store WildFire events into the green share level. Link the nodes to form a connected graph.  To start we must reach the prototype library that is available by navigating to small icon in the down left part of the Config Tab.     In this example we'll trust all indicators coming from VirusTotal and WildFire. This means that any SHA256 hash delivered by these miners will have a value of 100 in the confidence attribute. So we can just clone the stdlib.feedHCGreenWithValue prototype that will accept all indicators with confidence > 75 shared in the green level.         We can, as well, clone the stdlib.aggregatorSHA256 from the prototype library into our graph.         The VirusTotal miner prototype in the library is still tagged as experimental which means that all indicators used by it will be placed in the red share level. As our output node only accepts indicators in the green share level, we must create a new prototype using the one in the library as a base but chaging the share level attribute value.       Note that the prototype in the library shares in the red channel. Click on "new" to create a new prototype that shares in the green level instead.     Now it is time to clone this recently created prototype into our graph.         We, finally, need a miner node that could receive async events (WildFire logs from a PANOS device) and store them into a local database. This miner must also provide aging for the indicators so they are removed from the list at a given expiration time. This is what the class minemeld.ft.localdb.Miner exposed through the stdlib.localDB prototype is all about.   As in the case of the VirusTotal prototype, this node is tagged as experimental which means we must first create a new prototype to share indicators in the green level before we can clone it into our graph.               At this moment our configuration should look like the one in the following picture. Note that the nodes are not linked. We can bind them together by clicking on the input cell of both the aggregator and the output node.     Once all nodes are conneted we just need to commit the configuration to get our working graph.     The graph can be shown by clicking on any node in the Nodes Tab and selecting the graph view.      Step 3. Configure authentication in the output node A good practice is to use authentication in the output nodes so only approved clients can grab indicators from it. MineMeld features a very flexible feed authentication mechanism: The administrator can create "feed users" that can latter be attached as "tags" to the output nodes. A given output node can host multiple tags which means it will accepts connections from more that one customer.   First of all let's access the feed user panel available in the Admin main tab. Once there, a new feed user can be added by clicking on the plus icon.        The feed user must then be attached to the output node. That can be achieved by clicking on the output node from the Nodes main Tab. In the node's status pannel you must click on the TAG row to attach the feed user authentication to that node.               We are all set for now. We just need the miners to start gathering indicators for us.   Step 4. Connect MineMeld with VirusTotal You might have noticed the small error icon besides the VirusTotal miner node. That error indicates that the node is not fully configured. A mandatory attribute is not yet provided: The VirusTotal Intelligece API Key.   The key can be obained from the VirusTotal page under the profile settings. Note, please, that the API Key must come from a Virus Total Account with Intelligence access permissions. If you're not familiar with VirusTotal malware hunting features please visit https://www.virustotal.com/intelligence/help/malware-hunting/ before using the VT miner.       Once you have copied the key to your clipboard just paste it into the API KEY field available in the status tab of the virustotal node panel.       No commit is needed. Just force a polling by clicking on the icon at the end right of the LAST RUN field. If everything is right the indicators field will be updated with a non zero value. If you want to take a look to these indicators just navigate to the output node an click on the FEED BASE URL to get a plain text version of the list.     Step 5. Configure WildFire incident forwarding The last step in our mission today is to capture WildFire events from our PANOS device to extract SHA256 indicators out of them. To achieve this we'll leverage the HTTP Forwarding feature introduced with PANOS 8.0.   The following is our theory of operations: PANOS generates a WildFire event log with multiple fields. One of them is the "file_digest" that contains the SHA256 of the recently detected malware. A log forwarding object is attached to the security rule that features WildFire inspection The log forwarding object accepts all events (no filter needed) in the WildFire category, formats them in a way that the miner node can consume and forwards it using HTTPS to MineMeld. MineMeld's default distribution includes a self signed certificate. For the HTTPS protocol to work you must: Issue a certificate for MineMeld's web server from a PKI that is in the trust list of the PANOS Device. Alternatively you can use the PANOS Device as a PKI and generate the MineMeld certificate from it. Follow the instructions in https://live.paloaltonetworks.com/t5/MineMeld-Discussions/Minemeld-SSL-Certificates/m-p/112938 to import that certificate into your MineMeld instance. First of all we need a new HTTP Server Profile for the MineMeld. We must provide: A name An address or FQDN Protocol and Port Method Credentials (we can use any MineMeld admin user credential. Feed user credentials won't work)     The localDB miners's API accepts payloads either in JSON or PlainText format. In any case the type and indicator fields are the only mandatory ones. Optionals fields are: share_level: to override miner's share_level value. A given threat intel source might want to specify the share level of a given indicator. ttl: the amount of seconds the indicators should be kept into the store. A null ttl value means the indicator won't ever expire. In this example we'll use a 1h ttl so you can verify how the indicators are aged out by the miner. any other attribute will just be attached to the indicator to enrich it.   A couple of comments in the payload format shown in the picture above: The url format must be that exact literal just changing the text in red to match the miner's node name we want to forward the indicators to. In our example /config/data/WildFireEvent_indicators/append A Content-Type header with the value application/json is needed for the MM API to accept json data. The parameter t must be equal to localdb to forward the received data to a localdb type of miner. The optional parameter h can contain a node name. If it exists the MineMeld engine will trigger a node "sync request" to the named node after the data has been processed. In this case we want the indicator captured to be automatically flushed down the graph path so we request a sync of the WildFireEvent node. Note the $filedigest variable in the payload field. This tells the PANOS forwarding engine to replace that space in the payload with the corresponding field value in the event log. The next step is to create a new log forwarding object to feature this collection of HTTP server and payload format we've just defined.     Finally we must attach this log forwarding object to the security policy rules that features WildFire inspection.     If you do not want to wait for WildFire incidents just trigger a new one by downloading the WildFire test PE file. Once you get your incident just confirm that it is available in the MineMeld's output feed.               MineMeld's feed output node, by default, provides the content in a plain text format. But you can modify this behavior with additional attributes as explained in Parameters for the output node to easy the consumption by third parties. And, why not, contribute to the MineMeld OpenSource project with new output nodes like, McAfee DXL (TIE) or Tanium.   Annex 1: the prototypes Local Prototypes used in this use case (/opt/minemeld/local/prototypes/minemeldlocal.yml)   author: minemeld-web description: Local prototype library managed via MineMeld WebUI prototypes: localDBGreen: class: minemeld.ft.localdb.Miner config: attributes: confidence: 100 share_level: green interval: 3600 description: 'list of indicators. Use a name starting with "wl" to create a whitelist for an aggregator ' development_status: EXPERIMENTAL indicator_types: - any node_type: miner tags: - ConfidenceAny - ShareLevelAny virusTotalShareGreen: class: minemeld.ft.vt.Notifications config: age_out: default: 365d interval: 7200 sudden_death: false attributes: confidence: 100 share_level: green description: 'Miner for VirusTotal Intelligence Notifications feed. ' development_status: EXPERIMENTAL indicator_types: - md5 - sha256 - sha1 node_type: miner tags: - ConfidenceHigh - ShareLevelRed   Graph configuration (Config -> Import)   nodes: VirusTotal: inputs: [] output: true prototype: minemeldlocal.virusTotalShareGreen WildFireEvent: inputs: [] output: true prototype: minemeldlocal.localDBGreen sha256agg: indicator_types: - sha256 inputs: - VirusTotal - WildFireEvent node_type: processor output: true prototype: stdlib.aggregatorSHA256 sha256out: indicator_types: - any inputs: - sha256agg node_type: output output: false prototype: stdlib.feedHCGreenWithValue Annex 2: the API In MineMeld 0.9.42 we have introduced a new type of Miner (stdlib.localDB) exposing the HTTP-based API used in this article. In the following examples we assume that you have created a localDB Miner called FooBar.   A2.1: Push new indicators via JSON You can push new indicators using JSON. The ttl attribute is the expiration time in seconds of the indicator. If omitted the default config value of the Miner will be used. Only the indicator and type attributes are mandatory. curl -XPOST -H "Content-Type: application/json" -u admin:minemeld "https://192.168.55.150/config/data/FooBar_indicators/append?h=FooBar&t=localdb" -d ' { "indicator": "8.8.8.8", "type": "IPv4", "comment": "usual Google DNS Public IP", "share_level": "green", "confidence": 100, "ttl": 3600 }'  A2.2: Delete indicators via JSON You can use the same API call to delete an indicator, by setting the ttl attribute to 0: curl -XPOST -H "Content-Type: application/json" -u admin:minemeld "https://192.168.55.150/config/data/FooBar_indicators/append?h=FooBar&t=localdb" -d ' { "indicator": "8.8.8.8", "type": "IPv4", "ttl": 0 }' A2.3: Push new indicators via text format You can also use a simple text format to push a list of indicators: Indicators are divided by a blank line The first line of an indicator is the type of the indicator The second line is the indicator itself After you can specify the attributes, where a line is the attribute name and the line after is the attribute value Example: curl -XPOST -H "Content-Type: application/text" -u admin:minemeld "https://192.168.55.150/config/data/FooBar_indicators/append?h=FooBar&t=localdb" -d ' URL https://www.google.com comment test comment confidence 80 share_level green ttl 1800 IPv4 8.8.8.8 comment usual Google DNS Public IP share_level green confidence 100 ttl 86400 ' A2.4: Delete indicators via text format You can use the same API call to delete an indicator, by setting the ttl attribute to 0: curl -XPOST -H "Content-Type: application/text" -u admin:minemeld "https://192.168.55.150/config/data/FooBar_indicators/append?h=FooBar&t=localdb" -d ' URL https://www.google.com ttl 0 IPv4 8.8.8.8 ttl 0 '