Google Safebrowsing miner

[chirss]

Google has a threat list api, has anyone created a miner for it?

https://developers.google.com/safe-browsing/v4/lists

[xhoms]

@chirss Google Safe Browsing lists are not really "lists". It is an API that will give you information about a given URL. I mean: you have a URL and you're wondering what Google's Safe Browsing thinks about that URL. You can use the API for such a case.

I'm planning an "Enrichement Framework" for MineMeld that will be able to attach additional attributes to indicators. A Google Safe Browsing node for the Enrichement Framework would be awesome.

[chirss]

Ya that's what I want as well. If I can compare url information from a feed with what safebrowsing thinks of it and then come up with a ranking to be used by different outputs that would be ideal. Is this what you are thinking? I haven't played enough with miner creation to build anything like this out.

[chirss]

Also maybe a miner isn't the right thing so much as a processor. If an ioc hits the processor it then queries the api (within limits of the api). 

There are an awful lot of reputation type things which could possibly be used in a similar manner.

[xhoms]

You're following my same path.

• I started thinking on miners calling enrichement API's to attach additional attributes to the indicators.
• Then I though it would be better implemented as an aggregator node (once for many nodes)
• Then I realized a same indicator (i.e. a URL) could be enriched by many sources (i.e. Safe Browsing, PAN-DB, etc.) and that a cache should be put in place to avoid continous calls for the same indicatos.

This is why I reached to the point that a "Enrichement Framework" for MineMeld would be welcome by the community. So I have it in my current plan of intentions.

[chirss]

Ya exactly. 

The problem I'm finding is a lot of the miners likely have duplicate entries of some kind. So I'm sending them all to the same processor for similar types of feeds (phishing type miners to phishing processor for example). However I have to validate everything coming in before being able to trust it, i.e. verify before trusting.

The scenario you're talking about would be very beneficial in at least this scenario.