Tips & Tricks: Using DNS Sinkhole to find Malicious Clients

by on ‎04-22-2015 08:23 AM - last edited on ‎05-09-2017 05:45 PM by (18,206 Views)

fy16-tipstricks-lato.png

Hello again, Joe Delio from the Community team here, Today I will be introducing another new Community Feature:

Tips & Tricks

 

Inside Tips & Tricks, we will be talking about a specific Palo Alto Networks Firewall or Panorama feature, explaining what it is, how to configure it and how to use that feature. It is intended to teach you new features easily.

Today I will be talking about DNS Sinkhole.

 

What is DNS Sinkhole? Why do I want to enable this feature?

Starting with Pan-OS 6.0, a new feature called DNS Sinkhole was added to help battle malicious software (malware and spyware) in networks. This feature works by monitoring DNS requests for malicious DNS requests passing through the Firewall. If one is found, then the Palo Alto Networks device can forge a response causing the malicious domain name to resolve to a customer defined IP address (bogus IP). This will prevent the Malicious DNS request from working, stopping the Malware or Spyware from communicating to the Internet, as well as recording this information in the logs.

 

How do I configure DNS Sinkhole?

Please refer to the following document for instructions on configuring DNS Sinkhole:

https://live.paloaltonetworks.com/docs/DOC-6220

 

Tips & Tricks using DNS Sinkhole to find malicious clients

The following is the sequence of events that will occur when the sinkhole feature is enabled with bogus IP 1.1.1.1:

  1. Malicious software on an infected host sends a DNS query to resolve a malicious DNS request on the Internet.
  2. If the infected internal hosts's DNS query is sent to an internal DNS server(the firewall will never see the host make this request, and is essentially hidden), that internal DNS server then queries a public DNS server on the other side of the firewall.
  3. If the DNS query matches a DNS entry in the DNS signatures database, the sinkhole action will be performed on the query, and the forged IP (1.1.1.1) is given in response, and a log in the threat logs for "Suspicious DNS request" will be recorded.
    T&T-dns-s1.png
  4. The infected client then attempts to start a session with the host, but uses the forged IP (1.1.1.1) address instead.  If a policy blocks access to that forged (bogus) IP, then that will show up in the traffic logs.
    dns-sinkhole-block-log.png
  5. All the administrator needs to do is look for "malicious DNS" query in the threat log, and can then search the traffic logs for the forged sinkhole IP address 1.1.1.1 and can easily locate the client IP address that is trying to start a malicious session with the sinkhole IP address. Since all traffic is stopped, it will ensure a secure network.

 

NOTE: If the request is made from an internal device making the DNS request directly, through the firewall, you will see the same IP address make the Initial DNS request as well as seeing the HTTP(S) request to the bogus IP (1.1.1.1). Most of the time this is NOT the case.

 

Previous to the sinkhole action it was impossible to tell who is the end user (the infected machine) and in the logs you could only see the internal DNS server as a part of the communication that is triggering this signature, but now, the Palo Alto Networks device can forge the response for the infected domains to a locally significant address we is able to capture the traffic afterwards and get the real infected machine.

 

Thanks for reading.

Stay secure,

Joe Delio

 

Tips & Tricks 4-21-2015

Comments
by HITSSEC
on ‎04-25-2015 02:02 PM

For those of you who use a vpn client (other than Global Protect) quite often you are delivering DNS services to the vpn client device. If you allow the sinkhole traffic through the vpn connection (set as a default or global rule) then you will be able to identify vpn endpoints with spyware / malware present.  Using the IP addresses identified as going to the sinkhole you can go to your VPN console and look at the logs to find the user / endpoint involved.

by jlinkowsky
on ‎05-05-2015 02:42 PM

I'm a bit confused.  In the line above it states: "The infected client then attempts to start a session with the host, but uses the forged IP (1.1.1.1) address instead.  "  However, the screen shot shows the traffic being ALLOWED.  Is this a typo or an incorrect screen shot?

Thanks.

by HITSSEC
on ‎05-05-2015 03:55 PM

jlinkwsky,

Since the sinkhole IP address has nothing physically associated with it the action is irrelevant. The important thing is that the malware / spyware infected host is attempting to go to the destination and the traffic attempt is being logged.  An action of "allowed" does not introduce any risk as there is normally nothing assigned to the sinkhole IP address.  In our case it is a 10.x.x.x IP address that gets routed out to the internet in our case.  I hope this explaination adds sone clarity.

Phil

by mivaldi
on ‎05-05-2015 04:00 PM

It actually says:

" If a policy blocks access to that forged (bogus) IP, then that will show up in the traffic logs. "

So it looks like in the screenshot example, a policy does *not* block the access to the sinkhole IP.


In reality, it wouldn't really matter if the traffic was allowed or blocked, since it's going to the sinkhole IP, and there's nothing listening there. The main thing is having a security policy that will "log" the event at session end, so that you can track which client computers are infected. Whether its a block or an allow, -not very important-. (Though yes, we can argue a block makes sessions age out faster, so it would result in a slightly more efficient memory utilization on the Firewall).

by jlinkowsky
on ‎05-06-2015 07:21 AM

Thank you both HITSSEC & mivaldi for your comments.  They totally make sense.  It was just confusing as the screenshot is showing "allow" and the statement was saying "If a policy blocks access to that forged (bogus) IP, then that will show up in the traffic logs."

I agree, the action really doesn't matter, as the IP is bogus.  In our case we DO want to see who is attempting to go out, and address them.

Thanks again!

John L.

by
on ‎05-06-2015 08:54 AM

Jlinkowsky,

I am sorry for the confusion, the screenshot should have been showing the access to those sites being blocked.  And I will fix that.

But mivaldi is correct, because this is a bogus/fake IP, the site will NOT work. Access just needs to be recorded, allow or block.  But it would be easier to just block this traffic.

by
on ‎05-08-2015 10:54 AM

The image is fixed now, and it should make more sense.

by john.berry
on ‎03-01-2018 09:34 AM

The behavior of intercepting DNS requests and modifying DNS responses seems to be shared with the unsupported feature of DNS doctoring/rewriting/translating described here:

 

https://live.paloaltonetworks.com/t5/Management-Articles/DNS-rewrite-on-a-Palo-Alto-Networks-firewal...

 

Can this functionality be used to accomplish this?  If not, can Palo Alto expand this underlying functionality to support DNS doctoring/rewriting/translating in a future update?

by Sergey.Ivashkin-PL
on ‎03-16-2018 03:48 AM

good day,

what is difference between action on DNS Query - block or sinkhole? 

by BPry
on ‎03-16-2018 06:21 AM

@Sergey.Ivashkin-PL,

Block is going to simply block the DNS query from getting a response; sinkhole is going to direct to supply a specific sinkhole IP instead of the requested domain. 

Generally you would utilize sinkhole if you are utilizing an internal DNS server that is in the same zone as your clients. If you weren't to utilize a sinkhole you wouldn't be able to see in the logs the internal client making the request, you would simply see the internal DNS server sending the response. If you setup the sinkholing correctly then you would actually be able to direct these internal clients to an IP in another zone, and alert on any of this traffic. 

 

If your DNS server is in another zone from your internal clients, then it really doesn't matter if you sinkhole or block. You'll be able to see the actual client making these requests regardless of utilizing a block or sinkhole action. 

by Sergey.Ivashkin-PL
on ‎03-16-2018 06:37 AM

@BPry,

 tahnk you for quick answer.

 

still can't get it. My DNS, for instance, located in different zone. palo intercepts the request to malicious server and - if block, don't send any responce to client (and C&C sleeps well till next attempt). If sinkhole, then send sinkhole address to client, so client do following request to that 'fake' address and we see alerts in threat and traffic logs, right? 

 

"If your DNS server is in another zone from your internal clients, then it really doesn't matter if you sinkhole or block. You'll be able to see the actual client making these requests regardless of utilizing a block or sinkhole action. " - I will see only alert in threat log for that suspicious dns request. Difference that client in same or different zone is only source of that request (client itself or his DNS server), 

 

Thank you!

 

 

 

by BPry
on ‎03-16-2018 09:45 AM

@Sergey.Ivashkin-PL,

If your DNS is already located in a different zone I wouldn't worry about setting up a DNS sinkhole. You are already seeing the client that is making the malicious response to the DNS server.

In most enviroments where the DNS server is located in the same zone as the clients you wouldn't see which client is making the request, you would only get an alert that the DNS server itself is making a DNS lookup attempt to the 'malicious' DNS server. In this situation you would want to setup a DNS sinkhole to some other address that you could track through the firewall, say 1.1.1.1. This way if any logs were generated from say 10.191.0.5 to 1.1.1.1, you would know that the client at 10.191.0.5 was the client that actually made the malicious DNS request. 

by Ryan Kearney
3 weeks ago

Why would you suggest using a non-private IP for this? 1.1.1.1 is now operated by Cloudflare as a public DNS resolver service.

by BPry
3 weeks ago

@Ryan Kearney,

Likely because APNIC has until today held that address and it wasn't actually assigned to anyone. For whatever reason APNIC decided to assign the address and allow CloudFlare to use it, regardless of the fact that they knew that it was in use as a dummy address in a large amount of documentation around the internet. To the point where they have been unable to actually measure how much traffic actually attempts to hit 1.1.1.1.

by Ryan Kearney
3 weeks ago - last edited 3 weeks ago

@BPry

 

1.1.1.1 was allocated to APNIC. Just because APNIC didn't assign it doesn't mean companies like Palo Alto should use it as a dumping ground for "features" such as this.

by BPry
3 weeks ago

@Ryan Kearney,

Cisco used it as the default for Captive Portals with WLC configurations up until a couple versions ago. Palo Alto's example isn't nearly as bad as other uses of 1.1.1.1 in documentation, although it now no longer functions as a proper sinkhole example as the DNS request will be answered. Not saying it's the proper thing to due, but since everybody found out about this going live yesterday it'll obviously take everyone a little bit of time to modify all of the documentation and find where its been used. 

by Ryan Kearney
3 weeks ago

@BPry

I'm aware, it was one of the first settings we changed on our WLC. Hopefully PaloAlto updates their documentation to stop encouraging users to use other companies IP space as dummy addresses. The whole "Well Cisco does it" shouldn't really apply here as PaloAlto should know better.

Ask Questions Get Answers Join the Live Community
Announcements
Status information for Palo Alto Networks Cloud Services are available at status.paloaltonetworks.com.
Click the Subscribe button on the page to receive email notifications.