As customers migrate to Office 365 they find themselves whitelisting a range of App-IDs for the various workloads they might use in the Office 365 product sets, such as Skype for Business, OneNote, Exchange Online and so on. Because Microsoft publishes Office 365 over a huge range of URLs, and IP addresses, a security admin would be tempted to simply allow access in policies to a destination of ‘any’, and this gets complicated when the Office 365 App-IDs tend to have dependencies on explicitly allowing web-browsing and SSL. It would be preferable to configure external dynamic lists and reference that in our security policies, and as it happens, Microsoft dynamically publishes a fully up-to-date list of all IPs, URLs and ports used by each of the 17 components of Office 365 every hour that we can use! This article will take you through setting up the open source MineMeld utility to parse this data into EDLs for PAN-OS to consume, and creation of a couple of example security policies for your environment.
Step 1. Deploy MineMeld
First, visit https://live.paloaltonetworks.com/t5/MineMeld/ct-p/MineMeld and select the article (from the top right) about installing and running MineMeld appropriate to your environment. Note, if using the VMWare desktop instructions (https://live.paloaltonetworks.com/t5/MineMeld-Articles/Running-MineMeld-on-VMWare-desktop/ta-p/72038) you can go ahead with the "Super fast setup" but please download the cloud-init ISO and mount it on first boot. Assuming an IP comes via DHCP and you have internet access, your VM will automatically be updated to the latest version of Minemeld.
Make note of MineMelds IP address (from an ifconfig) as you’ll need it for the Web UI in the next step.
Step 2. Obtain & Import Configuration
MineMeld does already come with Prototypes for each of the O365 services but you would normally need to create a miner for each of these from those Prototypes, along with 3 processors and 3 outputs (one each for IPv4 addresses, IPv6 addresses and URLs respectfully). To save you the hassle we've created a configuration you can import, simply download it from https://paloaltonetworks.app.box.com/s/4ubmkgrq72a8mdd24j733ddqdgbkyvv4 and open it in a text editor.
NOTE: for a minimal config collecting all the IPv4s, IPv6 and URLs of all the O365 products download this instead: https://paloaltonetworks.box.com/s/gndwe5rzheg1ekwplxb4m3mrpcf5k41f
Browse to https://Your-MM-IP-address/ (obtained above) and sign in with the username admin and password minemeld. Next click CONFIG at the top followed by IMPORT.
This will bring up the IMPORT CONFIGURATION window. Copy and paste all the text from the .yml file you downloaded in step 2 into here and click Replace (or Append, if you have already configured this instance of Minemeld for another purpose.
Accept to replace the candidate configuration, followed by clicking the COMMIT button and waiting some time for the engine to restart.
Can't see an IMPORT button? This is simply because you are using an older version of MineMeld. If you cannot upgrade for whatever reason, follow step 2a below instead. If not, carry on to step 3.
Step 2a. Importing configuration for an OLDER version of MineMeld only
nb: Skip this step if you were able to import using the web interface as above!
$ scp ./office365-config.yml ubuntu@10.193.23.98:
To replace the configuration of a fresh install, SSH into your MineMeld instance (again as the ubuntu user) and run the following command:
$ sudo -u minemeld cp office365-config.yml /opt/minemeld/local/config/committed-config.yml
$ sudo -u minemeld cat office365-config.yml >> /opt/minemeld/local/config/committed-config.yml
Now run the command to restart MineMeld:
sudo service minemeld restart
Step 3. Review Connection Graph and retrieve Feed Base URLs
After giving the MineMeld engine a few minutes to restart, click “Nodes” in the banner at the top of the interface and then, click any of the nodes in the list.
Then click the Graph tab (asterisk sign) to bring up the Connection Graph which should look like this:
Here you see each of the miner nodes on the left scraping Microsoft’s dynamically updated XML File (direct link for your reference: https://support.content.office.net/en-us/static/O365IPAddresses.xml), the processor nodes that receive URLs, IPv4 and IPv6 addresses, and finally the 3 output nodes that publish a URL that your firewall can poll for an External Dynamic List (EDL).
Click each of the Output notes and make a note of the Feed Base URL.
Step 4. Consume MineMeld’s output
Log into your firewall (or Panorama) and go to Objects > External Dynamic Lists (or Objects > Dynamic Block Lists if using PAN-OS prior to v7.1). Click Add and create Dynamic IP address lists and URL lists to ‘subscribe’ to each of outputs created in the previous step. In my example below, I have created three dynamic lists matching the three Minemeld outputs above.
Step 5. Create a URL Filtering Profile
This will allow you to limit your access onto to the URLs in the O365-URLs dynamic list, which you’ll apply to your security polic(ies) allowing O365 later. Add a URL filtering profile, and block all categories (hint: Click the top checkbox to select all items, then click the Action banner in the list, and then click “Set Selected Actions”, then block to block all categories at once). Scroll to the bottom and allow only the external dynamic list of O365 URLs.
Step 6: Create Security Policies
Now that we have EDLs and a URL profile in place it’s time to modify/create our security policies. In the example below, we are allowing our Office 365 apps for all known users in the trust zone. The destination zone has been set to untrust zone but with the IPv4/6 lists as destination addresses.
App-IDs that you may find detected during use of Office 365 (depending on the clients and product sets being used)
What if there's still some O365 activity that is NOT hitting my new security policy?
You may find from using a catch-all rule with logging, that some sessions are not hitting this O365 rule when they should be. The reason is because Microsoft use CDN networks, which are outside of the IPv4/v6 ranges Microsoft use, like CloudFront for some applications in O365. The following URL will allow you to confirm if this is the case and whether you need to widen your whitelist to allow for these CDNs.
To allow access to the CDNs that do not match the security policy above, simply create a second security policy that allows from trust to untrust, from the same set of applications in the previous rule, and a destination of any. In the Service/URL category tab, insert the custom URL category from Step 5. The FQDNs will be present in that URL category and thus match this second rule.
i,
I have installed Minemild and I applied article and create dbl for Offcie365:
I have cretate 2 rules for Office 365 to test if feed sources Office 365 works.
I have a rule to match with feed list office 365 servers and application Office365 and all traffic should match with this rule. And I have below this rule other rule with destination "any" and application Office 365.
Log shows traffic match with 2 rule.
That means there are IPs that are not in the sources provided by Minemeld.
On the other hand, System Log Pa-500 shows this message:
'EBL(O365-IPV4) Refresh Aborted. No changes to list file
what's mean this message? Is an error ?
Hi,
Following this instructions blocks traffic to office365 in my environement. It looks like many of the url:s are being categorized as not 365 url:s (the Url filter I created from minemeld) but others. Here you can see that traffic for application ms teams url is catgegorized as internet-communications-and-telephony. And since the url filters is set to block all categories but the EDL from minemeld traffic is denied. What am I doing wrong?
Thanks, Mikael
Hello,
Looks like i do not have access to these links. Cannot seem to access the box.
https://paloaltonetworks.app.box.com/s/4ubmkgrq72a8mdd24j733ddqdgbkyvv4
https://paloaltonetworks.box.com/s/gndwe5rzheg1ekwplxb4m3mrpcf5k41f
How can i fix this?
Thanks,
