Using MineMeld to generate IP lists from wildcards

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements
Please sign in to see details of an important advisory in our Customer Advisories area.
L5 Sessionator
100% helpful (1/1)

Overview

In some cases you might face the need to create a policy rule in a Palo Alto Networks next generation firewall that targets a large list of IP addresses that shares a common schema.

 

For example:

  • All printers in a set of branch office networks that happens to be the ".7" in a collection of subnets where the third byte is a variable: "192.168.x.0/24"
  • All odd networks in the collection "10.10.x.0/24"

Using a set of wildcard masks could ease the description of these lists. For example, the first case fits within the wildcard mask 192.168.0.7/255.255.0.255 and the second example in the wildcard mask "10.10.0.0/255.255.1.0"

 

A miner extension available at https://github.com/PaloAltoNetworks/wildcardip-miner can be used to generate these lists of IP's from user provided wildcard masks.

 

In this article you'll find all the steps needed to deploy MineMeld and configure a Palo Alto Networks NGFW using the wildcard extension miner.

 

Step 1. Deploy MineMeld

First, visit https://live.paloaltonetworks.com/t5/MineMeld/ct-p/MineMeld and select the article (from the top right) about installing and running MineMeld appropriate to your environment. Note, if using the VMWare desktop instructions (https://live.paloaltonetworks.com/t5/MineMeld-Articles/Running-MineMeld-on-VMWare-desktop/ta-p/72038) you can go ahead with the "Super fast setup" but please download the cloud-init ISO and mount it on first boot. Assuming an IP comes via DHCP and you have internet access, your VM will automatically be updated  to the latest version of Minemeld.

 

Make note of MineMelds IP address (from an ifconfig) and login from your browser (defaults to username: admin / password: minemeld)

 

Step 2. Add the wildcardip extension

MineMeld introduced support for external extensions in the version 0.9.32. You can install them either localy or from a GitHub repository. To use the GitHub (recommended) option you must access to the repo URL available in its main page ("Clone or Download" button)

 01_GitHubRepo_1.png

 

The URL for the repo is https://github.com/PaloAltoNetworks/wildcardip-miner.git. Use this URL in the "Install Extension From Git" option in MineMeld's WebUI (accessible through the "System" menu)

 02_Extension_1.png

 

Once the extension is successfully downloaded, the next step is to activate it.

 03_Activation_1.png

 

Step 3. Creating a wildcardip node

The wildcard extension provides an example prototype that must be customized for our needs. The following is the sequence of tasks needed to add a node:

  1. Create a "New" prototype using the example provided one. Configure this new prototype with the needed list of wildcard masks
  2. "Clone" the just created prototype to convert it into a working node.

All these tasks are performed under the "Configuration" main menu in MineMeld.

 04_PrototypeLibrary_1.png

 

 

Once we reach the prototype library, use the "search" field to find the wildcard prototype provided by the extension.

 05_WildcardPrototype_1.png

 

 

Click over the prototype to display its contents and to be able to access the "new" and "clone" operations. In this case we need to create a "new" prototype that suits our needs.

 06_NewPrototype_1.png

 

 

Clicking on "new" will show the YAML prototype editor.

 07_BranchPrototype_1.png

 

 

This example creates a wildcard that will generate IP address objects that will match addresses .1 to .15 in a list of networks 192.168.[0-255].0. The meaning of the rest of the attributes in the prototype is:

  • age_out: It describes the "age out policy" of indicators for this node. In this case:
    • default = null : indicators are not aged out
    • interval = 86400 : check for aged out indicators once a day
    • sudden_death = true : remove indicators if they're not longer in the input list
  • interval: the amount of seconds between polls on this miner. We used 86400 to regenerate the list once a day. But it is safe to use larger values.
  • max_entries: a wildcard-class specific attribute that allows the user define a safety limit on the entries that should be generated by this node. The miner will fail to generate entries if the provided wildcard masks extend over this limit.
  • wildcard_list: a list of wildcard masks to be expanded. Verify, please, the YAML syntax (using TAB of each entry in the list)

Once the new prototype is added to the library, we can generate a node out of it using the "Clone" procedure.

 08_preclone_1.png

 

09_clone_1.png

 

10_addNode_1.png

 

Next task is to attach an output node to our recently created wildcard miner. Take note of the name provided to the miner because it will be used in the output creation.

 11_nodeAdded_1.png

 

Clone the stdlib.feedHCGreen prototype from the list as a node and configure the willdcard miner as its input.

 12_feedHCGreen_1.png13_addfeednode_1.png

 

At this moment, configuration should look like the following screen capture. Commit the new configuration.

 14_commit_1.png

 

Step 4: Enable feed authentication

Once the configuration is commited a new graph will be available in the "Nodes" main menu.

 15_graph_1.png

 

The number of indicators (256) corresponds is the result of the expansion of the wildcard mask provided in the prototype (192.168.0.0/255.255.0.240)

 

The details panel of the output contains the URL link to the feed. Clicking in it will provide the resulting list of the wildcard expansion.

 

16_feed_1.png

 

17_feedList_1.png

 

At this moment in time is important to note that you have been able to see the feed contents because the browser have a session cookie with the MineMeld WebUI. You can verify it by opening a new "incognito mode" window in your browser and attempting to access the URL from there. You should receive an authentication error in this case.

 

Before we keep going on, you must replace the original MineMeld SSL self-signed certificates with a valid ones issued by your local CA. The article How to Generate New MineMeld HTTPS Cert explains how to use a NGFW or Panorama as a local CA.

 

MineMeld feed authentication framework contains the following elements:

  • Feed User: A user/password pair with a list of "Access Tags" it belongs to.
  • Access Tag: A label that can be attached to a feed to limit its exposure to only the users belonging to that tag.

Feed Users are created from the "Admin" main menu.

 18_useradd_1.png

 

And access tags created clicking on the user's "Access" cell.

 19_accesstag_1.png

 

Next task is to attach this recently created access tag to the feed. The way to do this is by navigating to the feed node and clicking in its access cell.

 20_nodeaccesstag_1.png

 

All these changes apply at runtime. No commit is needed. We're ready for the next step.

 

Step 5: PANOS Configuration

NGFW from Palo Alto Networks feature a dynamic object called "External Dynamic List" (EDL) that can be used to consume the feed generated by MineMeld.

 

The following components are needed to successfuly complete the EDL configuration:

  1. The URL of the feed.
  2. A valid user/password pair that belongs to the access tag of that given feed.
  3. Certificate Profile including the CA that issued MineMeld's SSL certificate (for PANOS 8.0)
  4. NGFW Policy referencing the EDL (PANOS won't try to download the EDL until it is consumed somewhere)

 21_fwedl_1.png

 

User and password fields will not appear until you select one certificate profile. For PANOS 7.1 or before you must use the https://<username>:<password>@url syntax instead.

 22_certificateprofile_1.png

 

Final EDL configuration should look like the following screenshot.

 23_edlauthentication_1.png

 

Once configuration is commited, list entries should appear (provided that you are consuming this EDL anywhere in your NGFW configuration)

 24_edlresults.png

 

Annex A: Advanced MineMeld wildcard use case

Imagine that you're asked to extend the previous case removing the ".5" nodes from the list. This could be described like substract the list created by "192.168.0.5/255.255.0.255" from the list created by "192.168.0.0/255.255.0.240".

 

This can be achieved easily with the "whitelist" feature of the stdlib.aggregatorIPv4Generic processor. We just need to add a new wildcard miner with the mask "192.168.0.5/255.255.0.255", give it a name starting with "wl_" (whitelist) and combine it with the miner from the previous exercice.

 90_prototype_1.png91_graph_1.png

 

If you take a look to the feed output, you'll notice the "aggregator magic". It now describes the address spectrum as a list of ranges. For each network you have two ranges:

  • 192.168.x.0-192.168.x.4 and
  • 192.168.x.6-192.168.x.15

 92_list_1.png

 

Rate this article:
Comments
L3 Networker

@xhoms 

This is good guideline.

 

Could you capture example for step 5 - 3. Certificate Profile including the CA that issued MineMeld's SSL certificate (for PANOS 8.0) or Step for generate MineMeld's SSL certificate with CA?

 

28-11-2017 15-38-51.jpgI try to both certificate but it is not working.

L5 Sessionator

@iThreatHunt

I use a Panorama as a PKI to issue all my LAB certificates.

 

This is  how it looks like

ca_cert.png

 

 

 

L3 Networker

Hi @xhoms

 

I found a issue because I hardening MM web server to support TLSv1.1/1.2 only but PAN Firewall connect to MM via TLSv1 as default.


L1 Bithead

the second example in the wildcard mask "10.10.0.0/255.255.1.0"

 

Should this be "10.10.0.0/255.255.255.0"?

L5 Sessionator

Thanks @JamesRen for rising this error.

 

The example mask was valid for all "odd" (not even) networks of a specific C-class CIDR. Already amended the example text description

L1 Bithead

Does the new XSOAR CE support this as well?

  • 23321 Views
  • 6 comments
  • 1 Likes
Register or Sign-in
Contributors
Labels
Article Dashboard
Version history
Last Updated:
‎06-22-2021 03:59 AM
Updated by: