Connecting PAN-OS to MineMeld using External Dynamic Lists

[lmori]

One of the most common use cases for MineMeld is generating feeds to be used on PAN-OS as External Dynamic Lists. Using the MineMeld powerful engine, you can create External Dynamic Lists to track on AutoFocus the IP addresses, URLs and domains used by ransomware, known APT groups and active malware campaigns. You can also create External Dynamic Lists to track the IPs and URLs used by Microsoft Office365, or used as tor exit nodes, or used by CDNs and cloud services.

In this article we provide a step-by-step guide on how to configure authentication on AutoFocus/MineMeld generated feeds. We will also cover how to configure External Dynamic Lists objects on PAN-OS 7.1 and later. In this long article there are 3 main sections:

1. How to configure authentication on MineMeld feeds
2. How to configure EDLs on PAN-OS 8.0+
3. How to configure EDLs on PAN-OS 7.1

Note for community MineMeld

Even if this guide has been written for MineMeld running on AutoFocus, the same steps can be applied to the community version of MineMeld.

Note that on the community version of MineMeld feeds authentication is disabled by default. You have 2 options:

• enable feeds authentication and install a SSL certificate signed by a public or internal CA (strongly preferred for security reasons)
• leave the feeds authentication disabled

1. Configuring authentication on MineMeld feeds

Configuring the authentication on MineMeld generated feeds is a simple, 3 steps process:

1. Create a new feed user
2. Associate at least one access tag to the new feed user
3. Associate the same access tag to the feed you want to control

Let's start with creating a new feed user. On the MineMeld user interface click on the Admin tab:

In the Admin tab, click on the circle icon on the left to select the Feeds Users tab:

Then click on the plus icon in the bottom right corner to add a new user to the list:

In the Add User dialog, specify the username of the new feed user and the password (1). These are the credentials that will be used by PAN-OS to access the feed. Once done, click on the Ok button (2😞

Click on the Access field of the new user to specify the access tags associated with the user. The user will have access to all the feeds generated by MineMeld outputs tagged with these access tags:

Type in the Tags box (1) to associate one or more tags to the user. You can also create new tags, just type the new tag in the Tags box and press space. Click on Ok button (2) when done:

Now we have created a new feed user and associate one or more access tags to it. We should now associate at least one of these access tags to a MineMeld output to let the feed user actually have access to the feed generated by the output. Click on the Nodes tab:

Click on the output you want PAN-OS to connect to.

Note

The output should be based on one of the stdlib.feed* prototypes to be able to generate the feed in EDL format.

Click on the Tags field in the Status tab of the output to bring up the Tags dialog:

Add the access tag we created earlier to the Tags list (1) and click Ok (2). From now on, all the feeds users associated to this tag will be able to access the EDL generated by this output. There are 2 special tags you can associate with an output:

• any - any autheticated user will be able to access the generated feed
• anonymous - non-authenticated users will be able to access the generated feed

Take note of the URL in the Feed Base Url field of the output. This is the URL that should be configured inside the PAN-OS EDL object.

2. PAN-OS 8.0 and later - Configuring External Dynamic Lists

Now that you have configured authentication on the MineMeld generated feeds, it's time to create a PAN-OS External Dynamic List to connect to the MineMeld output. The process on PAN-OS 8.0 and later has the following steps:

1. Upload on PAN-OS the certificate of the CA of AutoFocus/MineMeld SSL certificate
2. Create a Certificate Profile with the uploaded CA certificate for remote SSL server verification
3. Create an EDL object using the Certificate Profile

First thing, download the certificate of the CA of the AutoFocus/MineMeld SSL certificate from the following link: https://certs.godaddy.com/repository/gd-class2-root.crt

Note for community MineMeld

If you have enabled authentication on feeds, you should provide and install on MineMeld an SSL certificate signed by a valid CA. The CA can be internal or public. Refer to the article How to Generate New MineMeld HTTPS Cert or to this thread (link) for the instructions.

On PAN-OS, click on the Device tab (1), select Cerificates (2) in the left bar and then click on Import (3):

Specify the Certificate Name (1), in Certificate File (2) select the CA certificate file you just downloaded (check the beginning of this section for the URL if you missed it) and click OK (3):

Now that we have uploaded the certificate, we can proceed to the next step that is creating a Certificate Profile to verify the AutoFocus/MineMeld SSL server certificate. Click on the Device tab (1), click on Certificate Profile (2) in the left bar and click on Add (3):

Specify the Name (1). We should now add the CA certificate to the list of CA certificates trusted by this Certificate Profile, click Add (2):

Select the CA certificate (1) and press OK (2):

Click OK to save the Certificate Profile:

Now we can finally create the External Dynamic List Object. Click on the Objects tab (1), select External Dynamic Lists (2) on the left bar and click Add (3😞

In the External Dynamic Lists dialog, specify the name of the new External Dynamic List (1), select the type of indicators contained in the new External Dynamic List (2) and copy the Feed Base URL of the MineMeld output we noted down at the end of the previous section (3😞

Now, select the Certificate Profile we created before (1). As soon as you specify the Certificate Profile, the Client Authentication section appears (2) and you will be able to specify the username and password of the feed user we created on MineMeld in the previous section. Press OK (3) to create the External Dynamic List.

Notes

• Test Source URL could be unreliable at times, commit the config to have a reliable check if the connectivity with MineMeld works
• If the Eexternal Dynamic List object is not being used in at least one policy, PAN-OS won't pull and refresh the contents

Done!! You just created a new External Dynamic List Object to point to one of the feeds generated by MineMeld. To add a second, third, ... External Dynamic Lists you don't need to do all the steps again and again but instead you can just reuse the Certificate Profile we have created.

3. PAN-OS 7.1 - Configuring External Dynamic Lists

PAN-OS 7.1 doesn't support configuration of Basic Authentication for External Dynamic Lists from the Web User Interface. We should instead embed the credentials inside the URL.

Click on the Objects tab (1), select External Dynamic Lists on the left bar (2) and click Add (3):

Type the name of the new External Dynamic List (1), select the type of indicators (2) and specify the URL (3) embedding the credentials inside the URL. Example: if the username is edluser and password test123 the URL would be https://edluser:test123@<minemeld hostname>/feeds/<feed name>. Press Ok (4😞

Done!! You just created a new External Dynamic List Object to point to one of the feeds generated by MineMeld.

[Mike.ship]

Can the certificate profile be created as shared allowing the EDL Objects to be shared or are they individual on to specific DGs and then have to be created multiple times?

[xhoms]

@Mike.ship: I don't see any reason why the Certificate Profiles could not be shared between DG's.

[MichaelMelone]

@Mike.ship I also have the problem with being able to have an overall shared Cert Profile object tied to EDL objects.  When the shared box is checkec on a shared EDL object, I cannot reference the Cert Profile, even with the cert profile pushed to the device template. This is using panorama.  The shared cert profiles can only be used within a template. For example, a Cert Profile created as shared in Tempalte "ABC" cannot be referenced by a shared EDL object. In order for the EDL object to reference the Cert Profile, the EDL must be in ABC location.  A "shared" EDL in a device group related to the template where the the shared Cert profile lives cannot see the cert profile even though the Cert Profile is shared. Only a non shared EDL can see the Cert Profile.   This makes me believe shared template objects dont necessarily mean they are able to be seen by shared Device Group Objects.  It makes me believe that EDL objects can only reference a cert profile that already lives on the device via it's template.  It's been slightly confusing when configuring EDLs on panos8 using panorama and getting those lists/cert profiles to work accross all devices. I'm ending up having to make 3 seperate EDLs of the same list for my 3 differeny vSYSs of one device. Those vSYSs device groups cannot see my Shared cert profile in the template of the devices where the 3 vSYSs are.  Maybe I'm configuring things wrong or that is just how it's designed. 

[Mike.ship]

@MichaelMelone Yes, you are correct. After spending a lot of time with TAC on the phone EDL objects are not able to be shared across device groups. I, as yourself, will have to create new EDLs for each device group. 

[nbilal]

@MichaelMelone and @Mike.ship:

It's kind of a hack, but we have been able to use Shared EDL's with a cert profile that's present in all of our template stacks by updating the cert profile via the command line. You can use this syntax:

set shared external-list {EDL_NAME} type {TYPE} certificate-profile {CERT_PROFILE_NAME}

It will commit without a problem to Panorama, though you need to make sure that that cert profile exists on the firewalls (either via template stack push or by configuring it locally on them) or your device-group pushes to the devices will fail.

HTH,

Nasir

[MichaelMelone]

 @Mike.ship and @nbilal

Thanks for the good information sharing and sorry for my pervious long-winded response. 

[MichaelMelone]

I was informed by some project managers that this is a limiatation of the current implementation of panorama and objects vs template settings. 

[tstrietelmeier]

@nbilal's slightly more manual solution worked perfectly!  It even displays properly in the GUI afterward.

[jasonrakers]

Another workaround for the Shared EDL cert profile is to create the EDL is a Device Group, then clone that EDL.  Edit the cloned EDL and mark it as Shared.  The cert profile will remain.  Just be sure all your firewalls will have the same cert profile.  We use a stack template with the base containing the cert profile path.

[IvanBarkic]

Hi,

I'm trying to setup EDL using HTTP access to MM server without any authentication (no feed users configured on MM server) and I'm getting:

Error:  pan_ebl_system_ebl_refresh_handler(pan_cfg_ebl.c:6602): EDL URL access error

Even in the Internet browser, I'm getting "Unauthorized".

But, when I click on the URL from MM WebUI for the specific output, I can see a list of IPs.

Is it possible to have it set up as http without any authentication?

Thanks

[mivaldi]

@IvanBarkic  yes, use the anonymous tag in the Output node. It is covered in this guide. The access will be unauthenticated HTTPS.