A discussion in a IRC-channel this evening was regarding the ongoing DDoS against wordpress installations all around the world and what to do in order to protect your webservers from the known bad ip addresses.
Using ACLs in for example a modern Cisco router seems to only be able to handle something like 1-10k ace's depending on masks being used etc.
Using iptables locally on the webservers seems to bail out after approx 30-40k lines with 10% cpu usage in kernel space for the linux kernel.
So what other countermeasures can be used?
A BGP based blackholing should be doable - however that only seems to block outgoing (returning) traffic from your network, it wont stop the incoming bad request (at least the syn will reach the server and then become hanging - while udp-traffic will be able to reach your server anyway). Unless im missing something here?
Another possible approach could be to enable HostnameLookups in your apache-server and use something like:
<Limit GET POST>
<LimitExcept GET POST>
and then in your DNS server make it authoritive for PTR records regarding the ip addresses you wish to block. When bad ip shows up your apache will ask the DNS for PTR-record of this host and if the answers from the DNS is "blacklisted.example.com" (or whatever you wish to call it) the apache will just drop the connection (perhaps with a http error 403 or such in return).
Which boils down to why I created this thread... what can be used if you have a PA device in place?
According to the datasheet the PA-5060 can use up to 40.000 security policies - but this is of course unique security policies.
A single security policy could perhaps be something like:
option: log on session end
or for that matter:
option: log on session end
So the question is really is it possible and in which way can one setup a custom country (or a custom address group) to hold 200k+ members (mostly /32 ip addresses)?
And as a sidenote - any efficient ways to keep this list easily up2date? There is for example methods to setup dynamic objects where the PA device will load a textfile from a webserver containing the ip addresses to act on as srcip (in this case) - or for that matter using the REST API to push (or withdraw) ip-addresses to blacklist. Will any of these methods work for a list that contains 200k+ ip addresses?
Or does any of you have other suggestions mainly from own experience? :-)
Solved! Go to Solution.
What if you use BGP blackholing in addition to uRPF (IP spoof protection)? I'm not sure if this is any more efficient than using an ACL on your router, but assuming your router has enough memory to hold all the /32 entries in its routing table, it may be worth a try. Since the router's core function is routing packets, hopefully a route lookup for uRPF enforcement would be faster than an ACL check. In theory, if your blackhole route doesn't point towards the internet, then the router should discard any packet from a blackholed IP when it arrives on your internet interface.
With the dynamic address objects introduced in PAN-OS 5.0, it should be easier to accomplish. The hard part would be getting these bad IP addresses into a flat file for the XML-API. I was thinking about this the other day and trying to integrate with fail2ban. I just haven't had time to look into it. :smileylaugh:
Even if dynamic objects would solve the administrative hazzle they still doesnt seem to be the solution.
According to the linked technote: "Each dynamic address object can have 256 unique IP addresses associated with it."
This can of course be workedaround with some script-fu to create 782 dynamic address objects and put them into a single address group and put that into a single security policy... but then on the next page:
Each dynamic address object counts as one object towards the platform’s maximum objects threshold regardless of how
many IP addresses are registered to that object. The maximum objects per platform and the maximum registered IP address
per platform (for 5.0.0) are detailed below:
Platform / Maximum Objects / Maximum Registered IP addresses
PA-5060 80,000 25,000
PA-5050 40,000 25,000
PA-5020 10,000 25,000
PA-4060 and PA-4050 40,000 5,000
PA-4020 10,000 5,000
PA-3050 10,000 5,000
PA-3020 5,000 5,000
PA-2050 10,000 1,000
PA-2020 5,000 1,000
PA-500 2,500 1,000
PA-200 2,500 1,000
VM-300 5,000 1,000
VM-200 2,500 1,000
VM-100 2,500 1,000
So back to square 1 :-(
I think you should be looking at the Dynamic Block list, vs the Dynamic Address object. The Block list object allows you to put to a dynamically created block list, hosted on a webserver. Look up Spamhaus and see if theirs will work for you.
Isnt that what umphmharding already suggested?
And the limit regarding maximum number of ip addresses one can handle in a PA-5000 seems to be 25.000 which is far less than the +200.000 needed.
Utilizing a list of +200,000 ip address may impracticle for a lot of us. A couple of approaches may be helpful. External block lists can be used. Entries in that list can be a single ip or up to a /16 if needed (utilizing subnet masks may cut down on the numbers). The use of resource protection under QOS protection can be used to limit the number of connections you will allow to your wordpress servers from regions or the internet as a whole. We have also used custom vulnerability signatures to stop external users from accessing specific directories and or files on our web servers. The use of any or all of these techniques may reduce the risk to a more manageable level.
Thanks for the suggestions but the main problem remains - how to efficiently block shitloads of ip-addresses?
The maintenance of the address-list is fairly simple - create a sql-table and use the export function to dump the addresses in sorted order into a txt-file on your webserver when needed (and if you got some time create a web-gui to make it easier to search and handle the db).
And with the help of the dynamic block list will make the life easy for the admin - with the downside that not even the PA-5000 series can hold more than 25.000 addresses...
The problem is that there doesnt seem to exist any hardware today that can load these +200k addresses and filter them without any noticable performance-drop (for example iptables works up to approx 30-40k).
A fugly workaround might be to use a couple of 48 int cisco switches (given that it can do 1k ACEs per interface), setup 24 VLANs on each and then just a bunch of networkcables to connect the VLANs in serial with each other. You would need about 400 interfaces (200 VLANs) but you would solve the problem (or about 10x boxes running iptables)... which seems odd to me that in year 2013 the network hardware is still very limited in this area.
I mean the TCAMs used 1k ACE's in the 90s and they still seem to have the same limit today while computing overall is way faster and have way more memory today than back in the 90s :S
How is the geoip stuff handled within a PA device? Could that be used to put all these 200k addresses into a single country named "blacklist islands" and then just create a single security policy where you drop any traffic where srcip = country(blacklist islands)?
Or for that matter setup a security police which use FQDN (srcip = blacklist.example.com) with roundrobin (and the roundrobin is a list of 200k ip-adresses)? :smileysilly:
Mikand, to answer your question... Dynamic Objects and Dynamic Block list are NOT the same thing. Dynamic Block list is https://live.paloaltonetworks.com/message/23654#23654 Dynamic objects (virtualization) is https://live.paloaltonetworks.com/docs/DOC-4121 Your answer/post is correct in referencing the limitations of the Block list functionality, but I wanted everyone on the post to know there is a difference, because the 2 were being intermingled. Thanks.
TippingPoint IPS can do this using custom reputation entries. Identify the attackers by Wordpress Admin login attempts (TP filter 12373) exceeding whatever threshold you set, use the sms responder to quarantine and add the offender to a custom reputation list (actions effective across all IPS enterprise wide).
As far as I recall, the N and NX series have no practical upper limit in IPv4. (old E series, was capped at 10k)
Maybe we're all thinking about this the wrong way.
There's an HTTP 401 Brute Force signature in the Threat-DB (https://threatvault.paloaltonetworks.com/Home/ThreatDetail/40031) that the threshold can be changed. ()
Just a thought.
Click Accept as Solution to acknowledge that the answer to your question has been provided.
The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!
These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!
The Live Community thanks you for your participation!