Pushback against DDOS Attacks

Huusker writes "Steven Bellovin and others at ATT Research Labs and ICIR have come up with mechanism to stop DDOS attacks. The idea is called Pushback. When the routers get flooded they consult a Unix daemon (/etc/pushbackd) to determine if they are being DDOS'ed. The routers propagate the quench packets back to the sources. The policy and propagation are separate, allowing hardware vendors to concentrate on the quench protocol while the white hats invent ever more clever DDOS detection filters for /etc/pushbackd. The authors of the paper have an initial implementation on FreeBSD."

Comments

Couldnt pushback be a Dos tool in itslf?

[Anonymous Coward]

If pushback is subverted, couldnt it function like an inverse DOD tool?

Re:hey

[autocracy]

That's exactly what this would do. The DDOS'd routers tell their upstream routers to cut back the flow of traffic - basically cutting out the source of the traffic. This of course requires that the upstream routers agree to do this...

Re:hey

[autocracy]

I'd like to withdraw/modify that statement. I read the top post too fast :)

It would shut off the source of the flood, not the destination as the original poster implied...

Re:Manual RegEx?

[Bill Wong]

DDoS is usually bandwitdh consumption...
Even if you drop 100% of the evil packets...
Your pipe is still filled...

And for the amount of traffic needed to actually DDoS a large-enough site like Yahoo (4 gbps last time around?), RegExs wouldn't be helpful
since, the sheer amount of cpu required to process *every*single*packet*that*passes*through* is wayy too much...

My take

[bobetov]

Sounds like a pretty v1.0 idea at this stage, but I'm psyched people are spending brain cycles working on DDoS and flash-flood solutions, since they're both problems that are only going to get worse.

(Gotta love the Slashdot effect getting named explicitly, eh? Nice to be part of the problem for a change... hehe.)

Seems to me the tricky part here is defining the aggregates. After reading the article, it isn't *really* a way to save your site from going down due to overload, more a way to prevent others sharing your pipe/routers from going down with you. ;-)

Which is a good goal in itself. It seems like a real tough thing to determine which of the millions of hits to www.yahoo.com (for ex.) are valid users, and which are DDoS bots. So both get restricted (net result: bots win), but the guy in the cage next to yahoo stays up.

Re:My take

[Subcarrier]

Sounds like a pretty v1.0 idea at this stage

I have to agree. They leave a lot of issues for further study. One big problem seems to be that gigabit backbone routers don't really have time to do any of this stuff. It's not much use if the back plate packet rate drops to one quarter because of having to detect and deal with flow aggregates.

Re:My take

[Cato]

That's not true of all such routers - as long as the number of aggregates to be filtered is fairly low, it shouldn't have too much impact. Most of the filtering should be on the routers at the edge of a given provider's network, which have less work to do than the true core routers - this is similar to the DiffServ QoS model except that the core routers don't need to do anything at all, since traffic is limited on the edges.

Juniper routers do this sort of filtering and policing in hardware, and can also generate traffic stats efficiently. Other vendors have similar features - Cisco 7500 routers can have multiple VIP processors, distributing the work down to the interface cards.

The main constraint is that you need new software written, installed and debugged in these routers, which will take time and require an agreed standard across router vendors. In the short term, it's easier to use existing features such as NetFlow/cflowd for traffic stats, feeding into an existing DDoS analysis tool (e.g. the Arbor Networks ones), which then tells a router provisioning tool to reconfigure the routers. This would not be as slick or dynamic as the proposed scheme, but could be done today. It would also make it possible to have a human in the loop initially, reviewing suggested changes. This would work OK as long as management and routing traffic are assigned a separate queue on each router interface, guaranteeing enough bandwidth to make these changes in the face of a DDoS attack (something that the ACC approach would also need).

Re:sure

[garcia]

educate people who are getting infected? Come on. Your not serious...

These people think that when they install virus scan software they are safe. I recently re-installed Windows on my gf's computer. She had V-Shield on there from 1999. She had no idea that she would need to update it.

At least my roommate, my parents, and my gf know (from me) not to open attachments. But educate a WIDE group of people? That's just not going to happen and you know it.

not all DDoS attacks..

[Anonymous Coward]

Not all DDoS attacks are bandwidth based, they could be application level and targeted at all sorts of other resources.

Some examples:

SYN floods can exhaust incoming connection queues.

DNS floods (asking a recursive nameserver a million questions, or even asking an authoritative nameserver a million questions).

Too many HTTP requests to processor intensive dynamic content pages could deny service well before you are serving at your bw limit.

The paper kept referring to the aggregate detection algorithm only coming into effect when the bandwidth limit is being exceeded .. it would be nice if these actions could be initiated in other situations also.

Never the less, this is a promising initiative.

--Iain

Re:sure

[Anonymous Coward]

that has to be one of the least constructive, head in the sand arguments I've ever read. Did you read the article ?

The technique is about making the internet move the point of dropping the flood packets, BACK closer to the source. That is, remove the flood from the internet itself, and contain it into the localised areas.

Instead of expecting the impossible as you suggest, (which is joe-average running a secure system), finally someone is thinking about securing the internet in general from unsecured systems, which is a pragmatic approach which may well protect the internet in general from many unforeseen DDOS attacks, as well as the ones we know about.

Re:sure

[Shishak]

Not exactly...

If every network provider ran this type of a system on their edge routers. Have all the edge routers communicate to distributed servers. Then, when you are being attacked you simply announce the offending IPs involved in the attack. That announcement gets propogated around all the servers which tell the edge devices to filter the traffic. It isn't a reverse flood. It is a way of telling the router closest to the source to start dropping packets.

Forged source IP's should be dropped at the edge already.

What we need is a protocol for sending dynamic filters to cisco routers. I would like to have input/output lists put on an interface that I can later build dynamically. I do it now with my Linux firewalls but it would be nice if I could drop the packets on the far side of my expensive link.

Re:sure

[Doug Neal]

How is this design proposing to saturate the Internet?

It involves sending a short message back to the routers that are routing the packets to you asking them to "quench" - i.e. filter out and don't route - the offending upstream sources.

The message could propagate as far back as the individual ISPs from which the packets are originating from so that each participant in the attack is cut off.

At least that's what I'm getting from the summary of the story, I could be completely wrong.

This is worse

[greenrom]

What the paper suggests is that if a router is getting way too many packets to a specific destination address, it will tell the routers upstream to throttle packets to that destination address (drop a certain percentage of them).

How does this really help a DOS attack? The idea behind a DOS attack is to flood a server with so many packets that the server can't keep up and ends up dropping most of the packets. This paper does not provide a solution to this problem. It simply shifts where the packets are being dropped... at a router upstream instead of at the server or router at the edge of the network. The only advantage here is that other servers hanging off the router that aren't being DOSed will be unaffected.

The suggested solution also opens up a potential security hole. If you gained access to a server, it might be possible to send a packet to routers upstream and tell them to throttle bandwidth. This could be a much more effecient way of doing a DOS attack. Now instead of multiple machines on fast connections, all you really need to DOS your favorite website is a 268 and a 300 baud modem.

Re:This is worse

[Anonymous Coward]

No, it throttles packets based on whatever is common to a majority of the packets. So, if a website suddenly gets a huge number of requests for /index.html, it can throttle those and let requests for another page through unhindered. If a web server gets a huge number of identically formed packets, it can throttle those and let differently formed packets through unhindered.

You are correct when you say it shifts the site where the packets are dropped. However, you miss the whole point. The site's router determines a pattern common to an attack, and tells the routers upstream the pattern. Those routers tell their upstream routers the pattern, etc. Alone, the site's router might be overloaded. The routers two levels upstream might all be just about overloaded, but still able to let through all non-attacking traffic. If these routers all begin throttling, the site's router will no longer be overloaded. All nonattacking packets will be let through unhindered. All attacking packets will be throttled severely. If the attack picks up and the second-level-upstream routers can't handle it, they will pushback to the third-level-upstream routers, etc.

At least, that's how I understood it.

Can this be right?

[rocjoe71]

This sounds like innovation and that just can't happen on non-M$ operating systems, can it?

Back down to earth, it's mega-wicked when good ideas are developed in FreeBSD (or Linux). Developments like these come the closest to the original intents and purposes of open sourced OSes.

blocking packets with forged return addresses

[wfmcwalter]

Perhaps someone more network-literate than I can answer this DDoS question, which has bothered me for some time.

I believe most DDoS attacks have the following in common:

1. DDoS zombies generally send packets with forged return addresses, as doing so greatly complicates attempts both to block packets and to track down individual zombies.
2. Machines used for DDoS attacks are almost always either corporate PCs or home PCs connected by DSL/cable. These nodes are single-homed, and as such packets emanating from them have only one initial route to the internet.

My question is this - why can't corporate IT people or their counterparts at ISPs reprogram their front-line routers (those that directly connect to individual end-user PCs) to block packets with forged return addresses? Forged addresses typically are either totally illegal or indicate a totally different net or subnet from the actual sender.

I can't see any reason why this wouldn't be a good idea - there really isn't any reason for the type of machines mentioned to ever act as true IP routers (as opposed to NATs), and it doesn't seem like this would be either hard or burdensome for the first-line routers to do.

Employing this would mean that DDoSers would be confined to forging return addresses within the zombies' own subnet, which would make both blocking and back-tracking much easier.

It's plain that this isn't done, so there must be a good reason why people much more network savvy than I haven't implemented it - what is it?

Re:blocking packets with forged return addresses

[swb]

"Good" networks prevent forged packets by doing what you suggest, dropping packets with bogus source addresses at the edge of the network or at appropriate ingress points.

I think the argument that is made for not doing this at a lot of ISPs is that with most Cisco routers its expensive as a lot of their routers can't fast switch with ACLs applied, they process switch, turning an adequate router into an inadequate packet-dropper.

It can also be a PITA to maintain -- if you put it at the very edge, like on an ISPs peering router with their upstream, it doesn't prevent in-block spoofing (eg, spoofing packets within the ISPs block). If you try to beat that on all the aggregation routers, you have a lot of ACLs to maintain; customer churn could put address blocks all over the place.

I'd argue that ISPs should make it a term of service that *their* customers ACL their edge routers; we-catch-spoofing-we-cut-you-off language.

Re:blocking packets with forged return addresses

[Cato]

This is mainly laziness - there are tools to help you do this, from Expect-based scripts up to commercial router provisioning tools (which can also be used to activate IP VPNs and QoS).

As for router capacity - Junipers don't have this problem, and if the ISP manages the CPE router on the customer site they can just push it down to that device. On a Cisco, where you have symmetric routing (probably the case for most smaller customers i.e. not dual-homed), you can just set the IP reverse-path forwarding option, which is very efficient - on each packet, the router does a routing lookup on the *source* address, as if it was trying to send a packet back to its origin. If the routing table doesn't have an entry for that source address that points out via the interface the packet was received on, the source address has been forged. This is not much overhead at all - just one more routing lookup.

For dual-homed customers, the provider has to use ACLs or perhaps a managed CPE, but ideally this would be a selling point for the ISP helping to generate cash to pay for router upgrades if needed - it safeguards the customer's network from being used to generate DDoS attacks with forged source addresses, which could save the customer from a lawsuit.

Old Idea

[Brew Bird]

This idea has been hashed to death for years.
The basic implementation has already been done.

What is novel and new about this paper is the suggestion that upstream routers are going to allow any tom, dick and mary to tell them what packets to throttle.

Always ass-uming that the larger switches can actually do this on the scale that is hinted at in the paper.

While issue 1 is specificly a political issue between carriers and customers, one could always point to the ease of which BGP routes are exchanged as an example of how easy this would be to do. Unfortunatly, since we are now talking about something that could effectivly put a transit provider out of business, there is no way issue number 1 will be overcome, unless the router manufactures give me the same kind of filter and ruleset technology I have for BGP. This would allow me to ignore anything I want from anyone, and would have the net affect of the feature being disabled!

as for 2, I'm sure some router manufacture has been touting this type of 'feature' on thier new multi-gig-a-bit MPLS/IP-does-everything-at-once switch. Don't believe it until it's out of the lab, guys. As many times as carriers have been screwed over by these new startups and their 'awsume powerful technology', I'm supprised anyone believes thier line of crap anymore.

It's too bad DDOS attacks don't go on for weeks, then we could use something like RBL to deal with it. Since they are so transitory, blackholing on the fly, (which is basicly what this paper is advocating) would require a lot more thinking about than has been put into this work.

Perhaps, instead of trying to complicate our lives with Yet Another New Protocol, you could simply come up with and IDS concatonation system, that puts together 'lists' of known DDOS sources at the current moment, and put it into a BGP feed... What a concept! Taking 2 technolgies that are known to work, and available to ANYONE that does BGP on the internet, and making it work!

Thank You, Come Again.

Re:Old Idea

[Cato]

A BGP feed will only help if you want to drop ALL traffic to a given IP prefix - the ACC proposal actually lets you limit traffic by port number as well.

Also, a BGP-only solution would only let you drop traffic, so it's not very useful for flash crowds, where the traffic is legitimate but excessive. It's also not useful where the port / prefix etc can't precisely identify only DDoS traffic - rate limiting allows some good traffic to get through while also limiting the DDoS. Blackholing != limiting (did you read the paper at all?)

I agree that this can be prototyped using existing technology (see my post elsewhere), but if this approach proves useful, a dedicated protocol would be helpful - though this could perhaps be piggybacked onto BGP using additional attributes to carry the filter and rate limit information.

Um, this isn't new.

[Mordant]

Bellovin came out with this a while ago. It's an interesting concept, but has the following practical drawbacks:

1. All the various vendors would have to implement it.

2. False positives. A new form of DoS would be to generate enough spoofed traffic to trigger this sort of thing -aimed at someone else-. Imagine your outrage when your l33t IRC buddies spoof your IP address block whilst attacking www.slashdot.com - no more imbecilic, outdated "Gee, whiz!" types of posts for you to read.

3. Oftentimes, rate-limiting via CAR, traffic shaping, or other methods consumes more CPU cycles on the routers than simply blocking the offending traffic (assuming this is possible, which depends upon the attack methodology).

The best way to combat DoS attacks generally is use strong platforms which process ACLs and other features in hardware (ensuring that your config allows those features to be processed in hardware; logging ACLs like a 'deny ip any any log' just won't cut it, these days), ensure you have the ability to 'draw off the poison' by sinkholing traffic headed for the destination by advertising a null route for it on a sinkhole router (this isn't always possible, it depends upon the target of the attck; you may not want to sinkhole all requests to your Web server, for example), ensure you have as good a traffic sniffing/IDS-type capability as possible, make use of Netflow tools like CAIDA cflowd/OSU flow-tools/Flowscan/Panoptiis/FLAVIO/Arbor Networks' Peakflow DoS, and know how to get in touch with the folks at your ISP(s) who can help with identifying the (even spoofed, via Netflow tracing) sources and blocking the offending traffic upstream of you.

If you're a commercial site, strongly consider a distributed Web site, hosted at different locations and using some sort of Global Server Load Balancing technology (GSLB; Cisco's Distributed Director and 4480 are two examples of this) to send people to different sites depending up their location, network topology-wise.

finally... a cure for Slashdotting...

[constantnormal]

in a press release by the Office of Homeland Defense, it was announced that an insidious plot by hacker terrorists had been thwarted. It seems that this subversive web site, www.slashdot.org, would trigger random DDOS attacks on targets identified on their web site. It has yet to be ascertained what their intent was, as no logical pattern has been detected. The investigation continues.

Welcome to the Twilight Zone.
I certainly hope the filters used to detect true DDOS attacks are effective enough to prevent this scenario.

Re:finally... a cure for Slashdotting...

[AJWM]

The term "flash crowd" goes back waaay before Slashdot ever existed (1973). It was coined by Larry Niven in a short story of that name (concerning *actual* crowds in an era where teleport booths are as ubiquitous as phone booths).

Nah, this isn't gonna work

[GreatDave]

Most of the reasons why have been said before but to sum it up...

Sending quench packets back to the routers feeding you DDoS packets, and eventually back to the host in question, is a good idea in theory. Kinda like communism. But in practice it won't work. First of all, there are the CPU strain resources on the routers and hosts. With DDoS, you have thousands if not more hosts all banging on your router, and your router is going to get a cardio workout going through its tables and deciding what gets throttled. Secondly, the return addresses on DDoS zombie packets are forged a good 80% of the time, meaning that you'll probably only hit 2 or 3 routers upstream with your quench packets.

A better solution? Null routes come to mind, but there are the CPU issues again. I'd like to see some "technology" similar to this where a customer of a commercial ISP could modify firewall rules on the _ISP's_ router to control traffic coming into their netblock. Perhaps a few routers upstream too. This really appears to be the only logical "quick fix" at the network level for DDoS.

A better fix would be to keep those zombies from ever coming into play by nuking everyone's NT/XP boxes, but that'll have to wait until penguins or daemons rule the planet.