About sperich

Discontinuation Notice Microsoft announced a new WEB Service that will deprecate the dynamic XML document used by the miners listed in this document. A new class and corresponding set of MineMeld prototypes was introduced in version 0.9.50 to deal with the new WEB Service. To to safely enable access to Office 365 please follow the instructions in the updated document at: Enable Access to Office 365 with MineMeld   Overview As customers migrate to Office 365 they find themselves whitelisting a range of App-IDs for the various workloads they might use in the Office 365 product sets, such as Skype for Business, OneNote, Exchange Online and so on. Because Microsoft publishes Office 365 over a huge range of URLs, and IP addresses, a security admin would be tempted to simply allow access in policies to a destination of ‘any’, and this gets complicated when the Office 365 App-IDs tend to have dependencies on explicitly allowing web-browsing and SSL. It would be preferable to configure external dynamic lists and reference that in our security policies, and as it happens, Microsoft dynamically publishes a fully up-to-date list of all IPs, URLs and ports used by each of the 17 components of Office 365 every hour that we can use! This article will take you through setting up the open source MineMeld utility to parse this data into EDLs for PAN-OS to consume, and creation of a couple of example security policies for your environment.   Step 1. Deploy MineMeld First, visit https://live.paloaltonetworks.com/t5/MineMeld/ct-p/MineMeld and select the article (from the top right) about installing and running MineMeld appropriate to your environment. Note, if using the VMWare desktop instructions ( https://live.paloaltonetworks.com/t5/MineMeld-Articles/Running-MineMeld-on-VMWare-desktop/ta-p/72038 ) you can go ahead with the "Super fast setup" but please download the cloud-init ISO and mount it on first boot. Assuming an IP comes via DHCP and you have internet access, your VM will automatically be updated  to the latest version of Minemeld.   Make note of MineMelds IP address (from an ifconfig) as you’ll need it for the Web UI in the next step.   Step 2. Obtain & Import Configuration MineMeld does already come with Prototypes for each of the O365 services but you would normally need to create a miner for each of these from those Prototypes, along with 3 processors and 3 outputs (one each for IPv4 addresses, IPv6 addresses and URLs respectfully). To save you the hassle we've created a configuration you can import, simply download it from https://paloaltonetworks.app.box.com/s/4ubmkgrq72a8mdd24j733ddqdgbkyvv4 and open it in a text editor.   NOTE: for a minimal config collecting all the IPv4s, IPv6 and URLs of all the O365 products download this instead: https://paloaltonetworks.box.com/s/gndwe5rzheg1ekwplxb4m3mrpcf5k41f   Browse to https://Your-MM-IP-address/ (obtained above) and sign in with the username admin and password minemeld. Next click CONFIG at the top followed by IMPORT.     This will bring up the IMPORT CONFIGURATION window. Copy and paste all the text from the .yml file you downloaded in step 2 into here and click Replace (or Append, if you have already configured this instance of Minemeld for another purpose.      Accept to replace the candidate configuration, followed by clicking the COMMIT button and waiting some time for the engine to restart.   Can't see an IMPORT button ?   This is simply because you are using an older version of MineMeld.  If you cannot upgrade for whatever reason, follow step 2a below instead.  If not, carry on to step 3.     Step 2a.  Importing configuration for an OLDER version of MineMeld only nb: Skip this step if you were able to import using the web interface as above!   SCP  the .yml file you downloaded in Step 2 to your MineMeld instance.  For example, on a Mac, run the following with the default password rsplizardspock $ scp ./office365-config.yml ubuntu@10.193.23.98: To replace the configuration of a fresh install, SSH into your MineMeld instance (again as the ubuntu user) and run the following command: $ sudo -u minemeld cp office365-config.yml /opt/minemeld/local/config/committed-config.yml Or, to append an existing configuration (ie. you have other configuration you would like to keep such as the default Spamhaus polling), run the following command or manually append the contents of office365-config.yml to the end of committed-config.yml yourself in a text editor: $ sudo -u minemeld cat office365-config.yml >> /opt/minemeld/local/config/committed-config.yml Now run the command to restart MineMeld: sudo service minemeld restart   Step 3.  Review Connection Graph and retrieve Feed Base URLs   After giving the MineMeld engine a few minutes to restart, click “Nodes” in the banner at the top of the interface and then, click any of the nodes in the list.     Then click the Graph tab (asterisk sign) to bring up the Connection Graph which should look like this:     Here you see each of the miner nodes on the left scraping Microsoft’s dynamically updated XML File (direct link for your reference: https://support.content.office.net/en-us/static/O365IPAddresses.xml ), the processor nodes that receive URLs, IPv4 and IPv6 addresses, and finally the 3 output nodes that publish a URL that your firewall can poll for an External Dynamic List (EDL).   Click each of the Output notes and make a note of the Feed Base URL.       Step 4. Consume MineMeld’s output Log into your firewall (or Panorama) and go to Objects > External Dynamic Lists (or Objects > Dynamic Block Lists if using PAN-OS prior to v7.1). Click Add and create Dynamic IP address lists and URL lists to ‘subscribe’ to each of outputs created in the previous step.  In my example below, I have created three dynamic lists matching the three Minemeld outputs above.     Step 5. Create a URL Filtering Profile   This will allow you to limit your access onto to the URLs in the O365-URLs dynamic list, which you’ll apply to your security polic(ies) allowing O365 later.  Add a URL filtering profile, and block all categories (hint: Click the top checkbox to select all items, then click the Action banner in the list, and then click “Set Selected Actions”, then block to block all categories at once).  Scroll to the bottom and allow only the external dynamic list of O365 URLs.       Step 6: Create Security Policies   Now that we have EDLs and a URL profile in place it’s time to modify/create our security policies. In the example below, we are allowing our Office 365 apps for all known users in the trust zone. The destination zone has been set to untrust zone but with the IPv4/6 lists as destination addresses.     App-IDs that you may find detected during use of Office 365 (depending on the clients and product sets being used)   activesync mapi-over-http ms-exchange ms-office365 ms-onedrive rpc-over-http soap ssl stun web-browsing webdav ms-office365 office-live office-on-demand outlook-web-online ms-lync-online ms-lync-online-apps-sharing sharepoint-online ms-lync-online-file-transfer What if there's still some O365 activity that is NOT hitting my new security policy? You may find from using a catch-all rule with logging, that some sessions are not hitting this O365 rule when they should be. The reason is because Microsoft use CDN networks, which are outside of the IPv4/v6 ranges Microsoft use, like CloudFront for some applications in O365. The following URL will allow you to confirm if this is the case and whether you need to widen your whitelist to allow for these CDNs. https://support.office.com/en-us/article/Office-365-URLs-and-IP-address-ranges-8548a211-3fe7-47cb-abb1-355ea5aa88a2   To allow access to the CDNs that do not match the security policy above, simply create a second security policy that allows from trust to untrust, from the same set of applications in the previous rule, and a destination of any. In the Service/URL category tab, insert the custom URL category from Step 5.  The FQDNs will be present in that URL category and thus match this second rule.   ... View more