Whitelisting Load-Balanced Sites - Fetch DNS records as JSON

[Hugh.Kelley]

For sites (to be whitelisted) that are behind ever-changing IP ranges (e.g. Amazon load balancer), has anybody used a services like these?   

https://dns.google.com/resolve?name=www.netflix.com 

https://dns-api.org/A/www.netflix.com

Is there an existing miner that does DNS lookups?

I saw this recommend in another thread but I'm hoping to avoid another VM:

https://github.com/PaloAltoNetworks/fqdn-service

This is the prototype I'm using

prototypes:
    dnsLookup:www_netflix_com:
        class: minemeld.ft.json.SimpleJSON
        config:
            age_out:
                default: null
                interval: 3600
                sudden_death: true
            attributes:
                confidence: 100
                share_level: green
                type: IPv4
            extractor: Answer[?type == `1`]
            fields:
            - name
            indicator: data
            prefix: dns
            source_name: dns.netflix
            url: https://dns.google.com/resolve?name=www.netflix.com
        development_status: STABLE
        indicator_types:
        - IPv4
        node_type: miner
        tags:
        - ConfidenceHigh
        - ShareLevelGreen

[xhoms · Accepted solution]

Hi @Hugh.Kelley,

can't comment on dns.google.com or dns-api.org but would like to provide some comments on https://github.com/PaloAltoNetworks/fqdn-service

• It is a "serverless implementation" (no VM needed). It is likely to cost you 0$ a month unless you share it with a large community of users
• It can resolve many FQDN's at once which means that a single miner is needed
• It can store a history of responses

Take into account, though, that if you're using PANOS then you better create custom L7 apps (SSL Decrypt + matching the HTTP Host Header or SSL Response Certificate in case you're not decryting) instead of matching based on FQDN. FQDN matching is performed at "sample intervals" (i.e. once an hour) and these FQDN entries behind AWS rotate tipically at 1 minute intervals. That means that you will, probably, fail to match many sessions between sample intervals no matter which DNS service you end up using