Dear MM comunity,
I am trying to use MM for parsing a URL list to populate a PA NGFW which lacks Url filtering license.
I have found that predefined miner urlhaus.URL which seems very well done. It is based on https://urlhaus.abuse.ch/ , which is free of charge.
I have cloned it, then cloned a URL aggregator and a URL Output.
I used the following aggregator
and the following URL output
So, I obtained an output, but seems it is not useful for NGFW (running 8.1 version) , probably because of http:// in front of every URL
that is the output (BE CAREFUL DON'T CLICK THEM)
I think I need to strip the http:// in order to be used by Panos..
For reference the queue reference the complete output is that:
Any tips is appreciated.
Solved! Go to Solution.
just add the "?v=panosurl" at the end of the output node url to get all these anonying prefixes being removed by MineMeld.
I would like to use the urlhaus list as well, but it currently has over 90,000 entries, while the PA-5000 and PA-7000 support a maximum of 50,000 URLs. Is there a smarter way to trim this list other than just blindly dropping the oldest entries using the "?n=50000" parameter?
The predefined miner urlhaus.yml has a url of https://urlhaus.abuse.ch/downloads/text/, which is just a listing of malware URLs with no other values. There is a different url at https://urlhaus.abuse.ch/downloads/csv/ that has several fields (ID, Dateadded, URL, URL status, Threat, Associated tags, and Link to URLhaus entry), but none with a confidence value.
I suppose one could re-write the miner to use the other URL and generate their own level of confidence from the "Dateadded" and "URL status" (excluding the oldest entries that have an "offline" status), but that's a little beyond my current level of proficiency.
Click Accept as Solution to acknowledge that the answer to your question has been provided.
The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!
These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!
The Live Community thanks you for your participation!