Just started going through the new miners and looking over docs, we are not in production on our deployment for O365 yet. I understand there are app-Id's that would catch most of these, but I noticed that the URL Minemeld feed for the "any-any" version includes quite a few URLS....specifically including:
I understand that the best way to police this type of access is to use app-id and decryption in a single ruleset that would include these lists. I just don't understand why items like the above are in this list getting mined directly from microsoft.....
I'm guessing that SSL decryption is absolutely essential before allowing anything out to these lists, and that you would only use these lists in your O365 app-id rulesets?- not the dependencies?
It appears that in the ruleset setup in the below article banking on the dependencies for O365-SSL and Web-browsing, as the firewall would read-down the ruleset, it would see your O365 enterprise-access custom app- and hit that rule, and then proceed to your dependencies. The catch all dependencies rule, just makes me a bit nervous,
Long story short- are people creating exclude lists, to Omit some of these IP/URL's that they don't want to mine?
you can generate a feed with 3rd party removed by using the output o365-api.feed-no-3rdparty prototype: https://github.com/PaloAltoNetworks/minemeld-node-prototypes/blob/master/prototypes/o365-api.yml#L50...
It applies some dump heuristics to detect if the IP Address/URLs belongs to MSFT or to 3rd parties.
If I'm reading this right, that miner is keying off of the keyword "integration" in the link that Microsoft provides, right? Is this marked experimental due to that- or that they could change that keyword at any time?
PS- Thank you Lmori for the quick replies. Your a wizard on this.
That's correct. I am checking with MSFT if there is a better way. Also note there is a bug in that prototype, I will fix in the next release.
We've run into the same 3rd party issue. Therefore, per this thread, we implemented the "...no-3rdparty" output but it is not removing the 3rd parties like Dropbox.com. The thread also mentioned an issue with this output, is there an ETA on when the issue will be resolved? Thanks
We have added a new feature that will be shipped in the next release (0.9.52 - by the end of the week). The O365 Miners now have an "Integrations" flag on the WebUI. By disabling the Integrations, 3rd party URLs will be removed.
After I toggle "integrations" the # of indicators drops to 618 from 654, but still includes youtube.com and itunes.com etc. What is the logic used to exclude 3rd party integrations?
I think I'm going to create some new nodes to match the "required" note in the category field.
Category: Shows whether the endpoint set is categorized as “Optimize”, “Allow”, or “Default”. You can read about these categories and guidance for management of them at http://aka.ms/pnc. This column also lists which endpoint sets are required to have network connectivity. For endpoint sets which are not required to have network connectivity, we provide notes in this field to indicate what functionality would be missing if the endpoint set is blocked. If you are excluding an entire service area, the endpoint sets listed as required do not require connectivity.
the nodes look for the word "integration" in the notes attribute of the URLs/IPs sections. This is far from optimal, but today there is no way to identify programmatically if a URLs/IPs in that list belongs to MSFT or not. The "required" field has a different meaning, not only includes 3rd party accessory sites but also optional MSFT sites.
If you are ok with dropping *all* the optional sites, you can change the prototype of the Miner to drop all the indicators with o365_required attribute set to false. The O365 API Miners already expose that attribute in the indicators.
Click Accept as Solution to acknowledge that the answer to your question has been provided.
The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!
These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!
The Live Community thanks you for your participation!