Showing results for 
Show  only  | Search instead for 
Did you mean: 

Who rated this article

L5 Sessionator
90% helpful (9/10)

Note: Palo Alto Networks made an end-of-life announcement about the MineMeld™ application in AutoFocus™ on August 1, 2021. Please read this article to learn about our recommended migration options.

This article describes a procedure that requires MineMeld version 0.9.50 or newer.




As customers migrate to Office 365, they find themselves whitelisting a range of App-IDs for the various workloads they might use the Office 365 product sets, such as Skype for Business, OneNote, Exchange Online and so on. Because Microsoft publishes Office 365 over a huge range of URLs, and IP addresses, a security admin would be tempted to simply allow access in policies to a destination of any, and this gets complicated when the Office 365 App-IDs tend to have dependencies on explicitly allowing web-browsing and SSL. It would be preferable to configure external dynamic lists and reference that in our security policies, and as it happens, Microsoft dynamically publishes a fully up-to-date list of all IPs, URLs and ports used by each of the 17 components of Office 365 every hour that we can use! This article will take you through setting up the open source MineMeld utility to parse this data into EDLs for PAN-OS to consume, and creation of a couple of example security policies for your environment


Step 1. Deploy MineMeld

First, visit the MineMeld Resource Page and select the article (from the top right) about installing and running MineMeld appropriate to your environment. NOTE: if using the VMWare desktop instructions (read Running MineMeld On VMWare Desktop), you can go ahead with the "Super fast setup," but please download the cloud-init ISO and mount it on first boot. Assuming an IP comes via DHCP and you have internet access, your VM will automatically be updated to the latest version of MineMeld.


Make note of MineMeld's IP address (from an ifconfig) as you’ll need it for the web interface in the next step.


Step 2. Obtain & Import Configuration

MineMeld does already come with Prototypes for each of the O365 services but you would normally need to create a miner for each of these from those Prototypes, along with 3 processors and 3 outputs (one each for IPv4 addresses, IPv6 addresses and URLs respectfully). To save you the hassle we've created a set of configurations you can import. Unzip the attached file to get the following collection of configurations.

  • o365-api-any-any.txt
  • o365-api-skype-usgovdod.txt
  • o365-api-sharepoint-usgovdod.txt
  • o365-api-exchange-usgovdod.txt
  • o365-api-skype-usgovgcchigh.txt
  • o365-api-sharepoint-usgovgcchigh.txt
  • o365-api-exchange-usgovgcchigh.txt
  • o365-api-skype-china.txt
  • o365-api-sharepoint-china.txt
  • o365-api-exchange-china.txt
  • o365-api-skype-germany.txt
  • o365-api-sharepoint-germany.txt
  • o365-api-exchange-germany.txt
  • o365-api-skype-ww.txt
  • o365-api-sharepoint-ww.txt
  • o365-api-exchange-ww.txt
  • o365-api-any-usgovdod.txt
  • o365-api-any-usgovgcchigh.txt
  • o365-api-any-china.txt
  • o365-api-any-germany.txt
  • o365-api-any-ww.txt

In this document we'll use the configuration named "o365-api-any-any.txt" that will set a graph to mine all ServiceAreas in all O365 Instances. But you might choose to use the any configuration that better suits your needs.


Browse to https://Your-MM-IP-address/ (obtained above) and sign in with the username admin and password minemeld. Next click CONFIG at the top followed by IMPORT.


MineMeld web interface.png


MineMeld config replace.png

Take into account that this procedure will replace any configuration you might have with this new collection of nodes. Your old configuration will be lost.

Accept to replace the candidate configuration, followed by clicking the COMMIT button and waiting some time for the engine to restart.


Step 3. Review Connection Graph and retrieve Feed Base URLs

After giving the MineMeld engine a few minutes to restart, click “Nodes” in the banner at the top of the interface and then, click any of the nodes in the list.


MineMeld node view.png


Then click the Graph tab (asterisk sign) to bring up the Connection Graph which should look like this:


MineMeld node show.png



Here you see each of the miner nodes on the left scraping Microsoft’s dynamically updated endpoints (as described in Managing Office 365 endpoints), the processor nodes that receive URLs, IPv4 and IPv6 addresses, and finally the 3 output nodes that publish a URL that your firewall can poll for an External Dynamic List (EDL).


Click each of the output notes and make a note of the Feed Base URL.


MineMeld feed link.png



Step 4. Consume MineMeld’s output

Follow the instructions in the article Connecting PAN-OS to MineMeld using External Dynamic Lists to connect your PAN-OS Device with the lists provided by MineMeld


Step 5. Create a URL Filtering Profile

This will allow you to limit your access onto to the URLs in the O365-URLs dynamic list, which you’ll apply to your security polic(ies) allowing O365 later. Add a URL filtering profile, and block all categories (hint: Click the top checkbox to select all items, then click the Action banner in the list, and then click “Set Selected Actions,” then block to block all categories at once).  Scroll to the bottom and allow only the external dynamic list of O365 URLs.


URL Filtering Profile.png


Step 6: Create Security Policies

Now that we have EDLs and a URL profile in place it’s time to modify/create our security policies. In the example below, we are allowing our Office 365 apps for all known users in the trust zone. The destination zone has been set to untrust zone but with the IPv4/6 lists as destination addresses.


The following is the set of screenshots that ilustrate how the security policy should be created.
















What applications should I list in the policy?

App-IDs that you may find detected during use of Office 365 (depending on the clients and product sets being used)

  • activesync
  • mapi-over-http
  • ms-exchange
  • ms-office365
  • ms-onedrive
  • rpc-over-http
  • soap
  • ssl
  • stun
  • web-browsing
  • webdav
  • ms-office365
  • office-live
  • office-on-demand
  • outlook-web-online
  • ms-lync-online
  • ms-lync-online-apps-sharing
  • sharepoint-online
  • ms-lync-online-file-transfer


What if there's still some O365 activity that is NOT hitting my new security policy?

You may find (from using a catch-all rule with logging) that some sessions are not hitting this O365 rule when they should be. The reason is because Microsoft use CDN networks, which are outside of the IPv4/v6 ranges Microsoft use, like CloudFront for some applications in O365. To allow access to the CDNs that do not match the security policy above, simply create a second security policy that allows from trust to untrust, from the same set of applications in the previous rule, and a destination address of any. In the Service/URL category tab, insert the custom URL category from Step 5. The FQDNs will be present in that URL category and thus match this second rule


How do I filter 3rd party URLs from the endpoint list?

There are 2 ways of doing this.

Use a local Miner. This works on any Miner version:

  • create a new Miner based on stdlib.localDB to be used a whitelist. The name of the Miner should start with "wl", example: wlSneaky3rdPartyURLs
  • connect the Miner to the URL aggregator for the O365 URL and commit
  • on the Web UI, under Nodes click on the new Miner and click on the table icon on the left

image (10).png

  • add the undesired URL in the indicator list ( in our case) and disable expiration. Click OKimage (11).png
  • The undesired is now removed from the URL list

Select only required endpoints. This requires MineMeld version 0.9.62+. An easy way to automatically remove all the 3rd party URLs is selecting only the O365 URLs marked as required by MSFT. MineMeld translates this attribute in the internal o365_required_list attribute (a list) and we can create a filter based on that:

  • go to the prototypes, search feedHCWithValue and click NEW (not CLONE)
  • give to the new prototype a meaningful name
  • paste this config:







-   actions:
    - accept
    - __method == 'withdraw'
    name: accept withdraws
-   actions:
    - accept
    - contains(o365_required_list, 'true') == true
    name: accept o365 required indicators
-   actions:
    - drop
    name: drop all
store_value: true







  • use the new prototype to build a new output node and connect it to the URL aggregator
  • only required URLs will be placed in the new output node


Rate this article:
Who rated this article