Post process Minemeld URL list

[Zach]

Hello - 

I am loving the product so far, but ran into an issue recently.  The miners are pulling the URL's correctly from Microsoft, but some of the URL's are not formatted correctly for use.  A few examples:

*-my.sharepoint.com

*word-view.officeapps.live.com

autodiscover-*.outlook.com

http://sr.symcd.com

There is also the issue of the way the PA processes the URL's.  For example, users need to access "outlook.com".  But the miners are pulling "www.outlook.com" and " *.outlook.com".  Therefore the access is denied.  

It seems like I need to do some sort of post-processing of the info.  Is anyone else already doing this?  I can try to muddle my way through a script, but I figured I would throw out the question first.  

[lmori]

Hi @Zach,

you can add the "v=panousrl" parameter to the feed URL to tell MineMeld to translate the URLs in PAN-OS format. Example:

https://<minemeld>/feeds/O365URLs?v=panosurl

Details here:

https://live.paloaltonetworks.com/t5/MineMeld-Articles/Parameters-for-the-output-feeds/ta-p/146170

[Zach]

That is great info!  I did not know that existed.  Thank you.  

This takes care of 95% of the problem.  I still am looking for a way around this though:

I show the following URL's in my feed: 

www.outlook.com

*.outlook.com

Is there a way to process that so that I also get an entry for "outlook.com"?  The firewall is very specific on the format, so if a user/service does not append www. to the URL, then the service is not reachable.

[lmori]

Hi @Zach,

have you seen other cases where the session was dropped because PAN-OS wasn't matching ?

A quick solution would be to add "outlook.com" to a static list Miner and add it to the URL aggregator.