Slashdot Mirror
EveryDNS Under Botnet DDoS Attack
mellow marsh writes "EveryDNS, sister company to OpenDNS (which runs the PhishTank anti-phishing initiative), has been hit by a massive distributed denial-of-service attack. The attack started sometime Friday afternoon and, from all indications, was targeting Web sites that used free DNS management services provided by EveryDNS. At the height of the DDoS bombardment, EveryDNS was being hit with more than 400mbps of traffic at each of its four locations around the world. From the article: '"We were collateral damage," Ulevitch explained... Because law enforcement is involved, Ulevitch was hesitant to release details of the actual target but there are signs that some of the targets were "nefarious domains" that have since been terminated.'" OpenDNS, which makes use of EveryDNS services, was affected for a time, until they spread their authoritative DNS more broadly. The EveryDNS site is now reporting that the attack is continuing but has been mitigated and is not affecting operations.
/., like kicking a dead puppy.
This should be a collective "SUCK IT" to the spammers and phishers out there. Keep it up, EasyDNS!
"The EveryDNS site is now reporting that the attack is continuing but has been mitigated and is not affecting operations." O Rly. I see it reporting a chunky man with bad hair holding an @. Please change link to everydns dot NET to continue the /. DDoS.
At the height of the DDoS bombardment, EveryDNS was being hit with more than 400mbps of traffic at each of its four locations around the world.
I would think that normal DNS operations would generate more than 400 millibits per second of traffic. How poorly designed ARE the EveryDNS sites?
This really made yesterday difficult for me.
My comp sci networking class assignment was on my home server, and I use EasyDNS. Had to bus home and put it on a USB stick. Last day of class, and the end of a particularly brutal week.
How about linking to the correct url?
That while they attack them there'll be less spam?
I don't know the meaning of the word 'don't' - J
The site is EveryDNS.Net.
:-)
I'll keep it up for Slashdot, let me just move it around a bit.
-david
# Hack the planet, it's important.
Nothing helps out a site currently under a DDoS attack like being linked to on the front page of /.
Like people who kill attorneys willing to prosecute those in the mafia. If any phishers can be found, I hope they get jailed for life.
Since I've been getting a lot of questions from folks about EveryDNS, how we've been stable and around so long, how we dealt with this DDoS and how we manage to cover our costs I am writing a response that will probably be posted here on Slashdot tomorrow or Monday to answer all these questions.
If you have questions about this or DDoS in general, feel free to ask them here and I'll make sure to cover them in my response. I'll be writing about what we've seen and what I generally do when it comes to soaking up traffic and how we handled this event in particular. (The short answer: find the smartest people you can to help you and then start taking corrective action)
Thanks!
David Ulevitch
# Hack the planet, it's important.
A client (a pretty large retail chain) was using EveryDNS for forward lookups to the mail server's A record. Mail they were sending out started to bounce because receiving mail servers weren't happy when trying to validate the sending box. In once case, a vital piece of mail sent to a state taxing authority couldn't get through on a month-end calendar deadline, causing much grief. Yes, alternate communcations channels are always an option, but it wasn't immediately clear why the two mail servers in question appeared to be hating each other.
Worse, the state government box's spam filtering appliance blacklisted the retailer's server, and a third party admin had to get involved to free things up. Quite a mess.
But the real lesson? People who say that a "cyber attack" couldn't really hurt the economy are wrong, wrong, wrong. This stuff can be really disruptive, and this was a pissant little scaled-down example. No major damage, but a lot of thrashing around, untold manhours of lost productivity, and (in the case of the anecdote in question, involving just one retail company), probably some tax fines which will require much tail chasing to get waived once the the story is clearly told, assuming the state government in question is feeling sporting about it.
Don't disappoint your bird dog. Go to the range.
What is "nefarious"?
to some.. the pirate bay and allofmp3 are "nefarious domains"..
to others "www.f**Ktimewarner.com" and "walmartsucks.com" are "nefarious domains"
and to others "www.wikipedia.org" and "www.aclu.org" are "nefarious domains".
I have a lot of trouble with the idea that DDOS attacks were being carried out in (apparently successful) attempts to wipe domains off the face of the earth..
this implies the attackers had no legal standing to take those domains offline.. then they call them "nefarious" after the fact.
VLC FOR MAC IS DYING! IF YOU DEVELOP, PLEASE SAVE IT!!
What reason could there be for botnet owners to attack EveryDNS? I can't see that they'd gain anything from it.
[sig]
Your "ripple effect" sounds more like bad code on the side of the sites being effected. The protocol shoud be secure on a technical level and not rely on laws to protect it, because no matter how fascit you want the internet to be, you can never control it all.
I could cause a lot more problems and not do anything illegal. Shoud those acts be illegal because of a butterfly effect caused by bad programming? Get real, please.
Great Intellect...
Compromised Windows machines network.
Where are the class action suits against Microsoft for continually producing such flawed software that makes it easy to 0wn a box?
If it wasn't for 20 some years of MS indifference towards security, there wouldn't be botnets like this, being used for DDOS attacks and forwarding billions of spams a day.
Guaranteed! This comment 100% Anthrax free!
You're pricks.
/. and in usenet).
Nothing positive or lasting will come out of trolling (and yes: this means you anonymous asshats on
So why not be part of a winning team and stop script kiddie'ing around from your parents basement.
Sincerely,
The Rest of the Human Race.
Someday, I'll have a real sig.
Shoud those acts be illegal because of a butterfly effect caused by bad programming? Get real, please.
If by "bad programming" you mean: the DDoS attack on the name servers was working, and thus a receiving mail server couldn't decide whether to trust another party's sent message... then, sure. Except that's not bad programming "on the site" (as you put it), is it? No. It's a vulnerability in using DNS in the first place. The only thing that would have prevented that would have been sticking with good old IP addresses for everything. But then, what stops a massive bot-net army from launching a DDoS attack against an IP address? Prosecution against the people who do it is at least somewhat helpful.
Don't disappoint your bird dog. Go to the range.
Did anybody else read this as "Every DNS Under Botnet DDoS Attack"?
I'm no MS fan, but it is worth noting that most of the OS's you mention were even less secure than Windows at the time the internet was being developed.
The difference is that very few people knew the exploits and fewer still were in a position to actually use them.
In once case, a vital piece of mail sent to a state taxing authority couldn't get through on a month-end calendar deadline, causing much grief.
Maybe a)it shouldn't be left until the deadline and b)sent via email, if it's so damn important.
And maybe you not tell clients to use a free DNS hosting service as their sole DNS provider...
Please help metamoderate.
1. Turn off the router
2. Turn on the TV and watch Oprah.
3. Turn the router back on before going home
4. Laugh all the way to the bank
Who is the bright boy that put a spam filter on a a drop box for important tax info. This is the digital equivalent of the government refusing to accept mail and claiming you missed the deadline.
No sir I dont like it.
After reading the updated article at Security Watch (http://securitywatch.eweek.com/exploits_and_attac ks/everydns_opendns_under_botnet_ddos_attack.html) , I'm begining to think that the target of the attack was Phish Tank itself. Why else would a hacker or hackers launch such a large scale assault on one the worlds largest free DNS providing groups if only to knock them offline for only a few hours? I think that the domains in question are just cover for the attackers attempt at taking Phish Tank offline, i.e. divert System Admins attention and resources, backdoor you way into the Phish tank server, upload botnets and virus, then push your little red button and watch years worth of work in the security field go down in a flaming death-throe. This, IMVO, to me, seems the more likely reason other than just a few kids jacking around. Why else would such a high profile target be selected by the everyday hacker if he knew the full brunt of the FBI was going to come down on him? These people that launched this assault seem to know what they wanted and went about it knowing full well what they were getting into. Also they attacked the entire DNS groups world wide system. These things seem to indicate to me that they wanted Phish Tank to go offline but just weren't able to pull it off.
Myself, a month ago I missed an opportunity to collaborate on a TV miniseries. Why? Because the moron who asked me for my collaboration absolutely trusted e-mail, and it was **THE** message that bounced thanks to a network glitch, and that moron didn't think of calling me on the **PHONE**. Well, if they were stupid enough to trust e-mail like that, they probably would have made a crappy miniseries anyways.
For casual communications, there is e-mail.
For vital ones, there is registered mail, fax or phone.
I have to stress that it is EveryDNS that is under attack, and not EasyDNS.com.
That being said this is not an uncommon issue these days at DNS providers across the 'net. Before anyone starts to kick and scream about how EveryDNS is handling things, remember that these attacks can get astoundingly vicious.
No amount of "clue" or mitigation or whatnot will help when the upstream service providers themselves are having trouble with the traffic load from a large-scale botnet attack.
-- The unsig...
sue each participating machine owner for neglegence
if you have a dog and it bites someone or damages someone's property you are liable, so why not computers?
Snowden and Manning are heroes.
A Slashdotting is always Righteous
/Don't hate me because my UID is prime
[Fuck Beta]
o0t!
Who is the bright boy that put a spam filter on a a drop box for important tax info. This is the digital equivalent of the government refusing to accept mail and claiming you missed the deadline.
I believe the official policy is that things are supposed to take place by postal mail, and FAX by fallback. But folks at both ends had been swapping mail for months with no problem (and more reliably AFTER the spam filtering went in), and got seduced into assuming it would always work. That's what happens, I see it all the time.
Don't disappoint your bird dog. Go to the range.
I use DNSPark, and they were subject to a DDOS attack earlier this week, too. Are they affiliated with EveryDNS too, or is it coincidence, since they are another cheap/free DNS host?
Never start vast projects with half-vast ideas.
By the late 1970s and early 1980s, VMS was already being used in a great number of locations. Its security was tested daily, and it proved to be rock-solid. That's why many companies still use it even today for their most essential data processing tasks.
I'd hardly suggest that the TCP/IP implementation of BSD was "insecure", even during its earlier releases. It was incorporated into so many other products, including SVR4 and Windows, mostly due to its extremely high quality. BSD itself was one of the most secure systems ever implemented. If it weren't for the excellent quality of the BSD codebase, we likely would never have seen a system as completely, and almost obscenely, secure as OpenBSD. Were it not for the tremendous earlier work at Berkeley, Theo et al. would have had little to build on.
There's no reason why Microsoft could not have built upon the knowledge and experience gained while developing truly secure systems like VMS and BSD UNIX. Hell, they had the main developer of VMS working for them, and they borrowed significant BSD-derived code. Then again, only Microsoft can take truly excellent software and experience, and pervert it into a horribly insecure product.
dictionary.com defines nefarious as: "Infamous by way of being extremely wicked."
What exactly being wicked would depend on the situation (as that's a subjective term) and considering that they are trying to take down websites via DDOS attacks, I'd call that wicked.
Although, I don't understand your last statement. Is it wrong to call them nefarious after the fact? Wouldn't you call a person a murderer after they murder someone?
"Tread softly because you tread on my dreams"
I think you read that summary wrong..
the DDOS attacks were supposedly against "nefarious domains" which this DNS service then took down, bowing to these cyberterrorist actions.. and after taking them down responded to questions by saying their attackers were actually attacking "nefarious" websites.
VLC FOR MAC IS DYING! IF YOU DEVELOP, PLEASE SAVE IT!!
Had to take my home server off line as three of my sites all had time out from too many mysql connections. Notices my home cable connection was slow as hell.
by TheSpoom (715771) Uncaring Linux user here. I have nothing to add to this but please continue. *munches popcorn*
DOS attacks are easy to pervent. And in this case, at least with your example, it could have been handled on the DNS's side server easy, had they known what they were doing. Stop hiding behind law to justify technical failures, the internet is survival of the fittist and that's just the way it belongs. (And lets not try to discuss how, if they can carry this out, you are going to catch them. It's pointless.)
Great Intellect...
there wasn't a link to EveryDNS in the article.
It's true no man is an island, but if you take a bunch of dead guys and tie 'em together, they make a good raft.
Two of my clients were affected by separate DDoS attacks against their hosting companies this week, and another was affected by this one. It must be contagious ... either that or I'm cursed.
INTERNET TERRORISM.
Pestiferous you say? I say gratuitous.
Stop hiding behind law to justify technical failures
I'm a little mystified at how you come up with this, but just to be clear: all I'm pointing out is that, as we sit, the proverbial "cyber attack" CAN indeed cause considerable economic disruption. I'm not sure what you think I'm hiding behind when I say that. It's just a statement of fact, and this one little event shows how disruptive it could be. I've made no particular call to action, but I certainly wouldn't mind if people who use bot-nets to cause business damage through extortion are prosecuted, just like people who threaten to burn down office buildings should be prosecuted.
Don't disappoint your bird dog. Go to the range.
Learn to spell, get a clue.
There is nothing you can practically do to prevent someone on the internet from sending a packet addressed to you, nor two packets, nor 1000000. There is nothing you can practically do to prevent the source address on each of those packets to be different. If a DOSer has much bigger pipes than you, you are sunk, unless you can do something very smart. For a start, getting remote access to your server during a DOS attack is tricky unless you have redundancy. Then you need to profile the traffic, find patterns which you can filter.
The non-triviality of a (D)DOS is the reason why everyone is interested to learn how to defend against such attacks. This is why we want to hear how EveryDNS handled the problem so well. A second-rate admin would not be able to. While I appreciate your sentiment regarding "survival of the fittest". I feel it can be better expressed as "survival of the fittest admin for the job".
As there are lots of admins on
Both the Oxford American and Oxford English dictionaries elaborate "nefarious" as "typically of an action or activity" - it does not state or mean that it has to be in the general opinion bad, rebellious, wicked etc., but could just as well imply any entity with a noted agenda, be it "good" or "bad".
automatic complaint-letter generator
DOS attacks are easy to pervent.
Prevent? How? "Don't hang out on IRC from your server's IP?" "When you get an email demanding $50000, pay up?" "Reach through the intertubes and strangle the guy that's about to send the packets to you?"
They might be "easy to deal with": call your upstream provider and hope that they'll shut it off (or call their upstream provider) rather than go "kaching!" and let your bandwidth bill rack up.
If I have been able to see further than others, it is because I bought a pair of binoculars.
I just want to say thank you publicly, you run a service that has helped out many folks, myself included.
And a reminder, EveryDNS.net runs on donations.
EveryDNS Donations
Thank you again.
ps: Wow, slashdot uid 18.
Wax on, wax off baby!
There is nothing you can practically do to prevent someone on the internet from sending a packet addressed to you, nor two packets, nor 1000000. There is nothing you can practically do to prevent the source address on each of those packets to be different. If a DOSer has much bigger pipes than you, you are sunk, unless you can do something very smart. For a start, getting remote access to your server during a DOS attack is tricky unless you have redundancy. Then you need to profile the traffic, find patterns which you can filter. The defense against this sort of attack is to using a hosting provider with huge pipes.
Real sites that must be up us providers with huge, redundant tier 1 internet connections to multiple networks.
Then a DOS attack becomes something interesting to watch on a router traffic graph instead of a problem.
I worked for one of those providers, and it was always fun watching a DOS attack because it was like a fart in a hurricaine. If you have big enough pipes and great network guys, nothing else matters.
I expect that 99% of computer owners that have compromised windoze machines do not know that their machines have been compromised. Each person who owns a PC that is involved should be notified.
"Your computer has been compromised and is being used for criminal activities. Now that you have been notified, you will be held responsible for future criminal activity involving your computer. Here are instructions for regaining control of your computer..."
-- QED
And you still don't have a clue. I worked for a company that was hit by a DDOS attack. Their Tier I network provider was so swamped that they switched off all traffic to the company (and this was a company whose entire business relied on its website being up). No matter how many redundant pipes you buy, there is always the possibility of being hit with more traffic. Unless you buy 50% of the worldwide bandwidth. I'd like to see you try that.
And at that point, it becomes a question of how you deal with a DDOS attack. Are you prepared? Can you work with your provider? Can you filter properly? Identify legitimate and illegitimate traffic? All important stuff that's vital for surviving these things.
Those who can, do. Those who can't, sue.
And you still don't have a clue. I worked for a company that was hit by a DDOS attack. Their Tier I network provider was so swamped that they switched off all traffic to the company (and this was a company whose entire business relied on its website being up). No matter how many redundant pipes you buy, there is always the possibility of being hit with more traffic. Unless you buy 50% of the worldwide bandwidth. I'd like to see you try that.
All that means is that your contract with the hosting provider let them weasel out of their responsibilities, and that they didn't have big enough pipes to enough networks and that they didn't have big enough balls.
With huge pipes, great network guys and globally distributed servers, a DOS attack becomes something interesting to wtch while having some coffee.
It's not necessary to own 1/2 the bandwidth in the world, only more than the botnet, which although way too much for a single server on a 10 Mb ethernet connection, doesn't mean squat to hundreds of servers doing IP Anycast scattered around the world.
A lot of companies don't want to be down, but in reality, if they go down for a while, they won't go out of business and nobody will die. When a site absolutely can't be down (human life or huge fianancial losses are at stake), they simply make sure they can handle whatever happens.
Then it would be perfect.
The man who trades freedom for security does not deserve nor will he ever receive either. - Benjamin Franklin
it loaded pretty damn quick for me, (less than a second) unless i missed something..
Denial of service attacks are very difficult to defend against.
Right. And with GOD on your side, armageddon becomes something interesting to watch while having some coffee.
Pop quiz, hotshot. You are the only network guy on station, your distributed servers are all getting pounded equally hard, everyone on the network is losing money and the traffic doesn't stop. What do you do? What do you do?
Rarely does anyone care what you can do without resource limitations. This is because there are always resource limitations. Network guys cost, bandwidth costs, smart Cisco boxes costs. Wires cost money. Electricity costs, rackspace costs. Will you minimize the cost of a DOS attack (including the cost of network equipment you paid before the attack started), or are you going to watch it continue over some coffee?
Let me update you on the figures: 70,000 botnet computers, 1Mbit each, equal 70Gbit of traffic. At least one such network has been recently reported.
Pop quiz, hotshot. You are the only network guy on station, your distributed servers are all getting pounded equally hard, everyone on the network is losing money and the traffic doesn't stop. What do you do? What do you do?
If you're the only guy on call for a global network, I'd find another job.
Rarely does anyone care what you can do without resource limitations. This is because there are always resource limitations. Network guys cost, bandwidth costs, smart Cisco boxes costs. Wires cost money. Electricity costs, rackspace costs. Will you minimize the cost of a DOS attack (including the cost of network equipment you paid before the attack started), or are you going to watch it continue over some coffee? All those costs are irrelevant. If you have something that can't go down, you do whatever is necessary to achieve that.
If you can't afford to do that (hardware, netowrk or staff), then obviously your service isn't as valuable to society as you think it is. Or it's being managed by morons.
Either way, it's nothing for you to have a stroke over. One of the most valuable things you'll eventually learn is that technology is mostly mental masturbation for humans. If 90% of the internet went down, people might be a little more bored than usual, but they wouldn't stop breathing. It just isn't that important.
There's a whole world out there, and mostly it really doesn't matter if you can pay your bills online or gamble or find porn or music or video or get live radar maps of the weather in your area or pretty much anything else that takes up almost all the bandwidth.
If you want to see where the rubber meets the road, sit in on meeting with a big customer and watch what happens when they say "I don't want my site to ever be down" and the sales rep tells them what that would cost. Then you'll see them say either "I don't care. It can never go down", or "OK, what would it cost for it to almost never goe down?"
Always being offline costs nothing (unless one is stupid enough to pay for it).
Those two cases are easy, boring and unrealistic. The interesting stuff is in-between.
You have again missed my point. At the end of the day, those people that make cost estimate have already through about how to deal with DOS and have already asked their engineers/admins to minimise the cost for a given range of operation. The technical how is of interest to us, because cost minimization is entirely dependant on that how.
I think you are arguing that investment in resources is more important than technicality (i.e., the more money you pour on a problem, the better it resolves itself). I'm arguing that someone at some point has to actually think about how to resolve it, regardless of cost. That someone is the resource you have paid for. So we're saying the same thing backwards. To me, it is more interesting to see how the resources go about solving the problem efficiently, not that a larger quantity of resources must be allocated to solve a larger problem. I'm going to give up trying to get that across now.
I think you are arguing that investment in resources is more important than technicality (i.e., the more money you pour on a problem, the better it resolves itself). I'm arguing that someone at some point has to actually think about how to resolve it, regardless of cost. That someone is the resource you have paid for. So we're saying the same thing backwards. To me, it is more interesting to see how the resources go about solving the problem efficiently, not that a larger quantity of resources must be allocated to solve a larger problem. I'm going to give up trying to get that across now. There's no solution for DOS that I'm aware of (other than staggering capacity), except reworking the entire network infrastructure to allow a server to revoke a client's priviledges(along with non-spoofable addresses), or replacing Windows with something that can't be so easily compromised.
Unfortunately, neither is going to happen any time soon.
These kinds of attacks ARE TERRORISM plain and simple.
While Homeland Security is running around jailing dark middle eastern gentleman on no grounds, organized mafias are systematically stealing resources around the world and using these massive computing resources for mischief. Computing power is real power in an economy that is so heavily networked, and I wish that nation's top law enforcement looked more seriously at these threats.
It * is * a terrorist threat. Do you realize how many American desktops and business computers are under the control of foreign invaders? These could be used to cripple the American economy if the attacks shifted from targeted/DDoS/spam to something more political.
"Prevent? How?"
HEY! Can't you read?!? He didn't say *prevent* - he said PERVENT, and it's a completely different thing!
Perventation is most simplicitude itslef, if ur as cluedinous as bky1701, dontcha know?
E-mail is plenty reliable if you engineer it to be so.
Relying on a single third party DNS service is pretty stupid if reliability important.
Redundant links, geographically and geopolitically dispersed DNS, careful administration.
Engineer reliability between all important endpoints.
Do that and you can send rather important things via e-mail and be confident that they will get there on time.
Be sloppy and it's no better than relying on a cell phone with poor coverage, a weak battery, and a pre-paid plan.