Slashdot Mirror
GitHub Survived the Biggest DDoS Attack Ever Recorded (wired.com)
A 1.35 terabit-per-second DDoS attack hit GitHub all at once last Wednesday. "It was the most powerful distributed denial of service attack recorded to date -- and it used an increasingly popular DDoS method, no botnet required," reports Wired. From the report: GitHub briefly struggled with intermittent outages as a digital system assessed the situation. Within 10 minutes it had automatically called for help from its DDoS mitigation service, Akamai Prolexic. Prolexic took over as an intermediary, routing all the traffic coming into and out of GitHub, and sent the data through its scrubbing centers to weed out and block malicious packets. After eight minutes, attackers relented and the assault dropped off. "We modeled our capacity based on fives times the biggest attack that the internet has ever seen," Josh Shaul, vice president of web security at Akamai told WIRED hours after the GitHub attack ended. "So I would have been certain that we could handle 1.3 Tbps, but at the same time we never had a terabit and a half come in all at once. It's one thing to have the confidence. It's another thing to see it actually play out how you'd hope."
Akamai defended against the attack in a number of ways. In addition to Prolexic's general DDoS defense infrastructure, the firm had also recently implemented specific mitigations for a type of DDoS attack stemming from so-called memcached servers. These database caching systems work to speed networks and websites, but they aren't meant to be exposed on the public internet; anyone can query them, and they'll likewise respond to anyone. About 100,000 memcached servers, mostly owned by businesses and other institutions, currently sit exposed online with no authentication protection, meaning an attacker can access them, and send them a special command packet that the server will respond to with a much larger reply.
Akamai defended against the attack in a number of ways. In addition to Prolexic's general DDoS defense infrastructure, the firm had also recently implemented specific mitigations for a type of DDoS attack stemming from so-called memcached servers. These database caching systems work to speed networks and websites, but they aren't meant to be exposed on the public internet; anyone can query them, and they'll likewise respond to anyone. About 100,000 memcached servers, mostly owned by businesses and other institutions, currently sit exposed online with no authentication protection, meaning an attacker can access them, and send them a special command packet that the server will respond to with a much larger reply.
Now itâ(TM)s just causes a couple ad blockers to go off.
TFA doesn't give any detail around this. How does one generate that much traffic without the need of a botnet?
He's getting rather old, but he's a good mouse.
Why do people do stupid shit like this? Github is neither a bad actor nor deserving of this. Why don't they go after the fucking Trump Organization or Oracle or something like this.
why would someone go through the trouble of attacking github? For giggles? Do they like closed source or mercurial that much?
The memcache servers ARE a ready made botnet.
Imagine if they had made a beowolf cluster of mem.... oh, wait.....
On the one hand you take life too seriously, and on the other, you do not take playful existence seriously enough. Seth
Such a shame there are nefarious people who do these DDOS. What a huge waste of time and resources by their target entities to defeat the attacks.
Was checking out another blog post on, really love this resource. Keep up the awesome work
(...) as a digital system assessed the situation (...)
Who knew those analog steam powered ddos protection engines would go of fashion this fast.
0x or or snor perron?!
for new and updated software, i never noticed any outage, i guess the admin that keep github percolating has got some good skillz, kudos to github admin...
Politics is Treachery, Religion is Brainwashing
[Akamai] sent the data through its scrubbing centers to weed out and block malicious packets.
There was the challenge to handle the load, but identifying packets to drop was quite easy this time: they all came from same UDP port for memcached.
Every IP packet has a source IP. ISPs should NEVER let ANYTHING go out that doesn't have
a source IP address in their subnets. Doesn't memcached use the source IP of the datagram to send the
reply? Or does it use an ip address in the body -- which is stupid, it's already in the IP header, why
have redundant information? As I tell my co-workers, never have the same information in two places,
one of them will be wrong. And if you use the source ip in the ip header even udp works through firewalls
and you don't have to go through the idiocy VOIP does wth TURN and STUN and ICE and all that
crap. Am I missing something?
and it promptly logged in all his cashews accounts and tried looking for food?
Back in the day UDP was considered unreliable because it could be dropped by the network at any time for any reason.
It should be noted that UDP is apparently just as reliable as TCP at the network level, in that equipment in general does -not- drop UDP at all. Behaviorally speaking the network attempts to guarantee delivery of everything, which is interesting and possibly unnecessary.
We wanted freedom, openness, equality. What we got was freedom for psychopaths to openly brutalize everyone equally, without any kind of repercussions or negative feedback. The eternal september has given us an incompetent tyrant for a leader, enabled mass deception by even worse tyrants, fake revolutions in the Arab world, brutal warfare and refugee crisis, and I wouldn't be surprised if it's linked to school shootings since in the old days, news people understood that mass media coverage created copycat killers. O very.
Some other site (cough fark cough) is claiming a DDOS attack. True dat?
I feel one kind of pain for someone who buys old hardware/software and does their best. I have a whole nuther level of pain for anyone targeted by salivating short-cortexed idiots who for whatever twisted reason decide to target people doing their best (or sitting around in lounge chairs drinking Coronas, long as they aren't hurting anyone).
So what kind of costs does Github have from Akamai Prolexic? Do they charge on a per problem basis or an annual subscription?
Here is some info on the firm:
https://en.wikipedia.org/wiki/...
>/dev/null
--
"And then there was one" - The Voice
Read TFA.
"...so-called memcached servers. These database caching systems work to speed networks and websites, but they aren't meant to be exposed on the public internet; anyone can query them, and they'll likewise respond to anyone. About 100,000 memcached servers, mostly owned by businesses and other institutions, currently sit exposed online with no authentication protection, meaning an attacker can access them, and send them a special command packet that the server will respond to with a much larger reply."
Someone sends a magic packet, the server sends a bigger packet, multiply and/or daisy-chain it and you have large scale DDOS.
Forgive me for sounding naive, since I've also been told to deploy memcached in this fashion, knowing that this is insecure, while asking why is memcached deployed without requiring authenticated BY DEFAULT?
I feel naive because this is a so-simple-it's-obvious solution.
What am I missing?
Kriston
OSX doesn't honor the charset headers in HTTP because it's a piece of fucking shite.
Apple superiority is about having your head so far up their own ass that you can see sunlight.
"Prolexic took over as an intermediary, routing all the traffic coming into and out of GitHub, and sent the data through its scrubbing centers to weed out and block malicious packets."
So, they probably just filtered all UDP packets with a source port of 11211. Looks like it was not only the biggest DDOS but also the easiest to defeat...
Memcached uses UDP, so you put the target's IP in the source IP field of the datagram and it responds (with a much larger packet). It's intended to be used on a local network (or even loopback), but it's often misconfigured. As you say, in an ideal world, ISPs shouldn't allow packages off their network with a source address that isn't from their network, but it's also not always trivial to identify the correct set of IPs to permit (traffic transiting your network has to be handled as well as traffic originating on your network, and if you've got a bunch of customers who all own their own /24s, plus a bunch of downstream networks that may or may not be routing over your network, depending on dynamic configuration, and may only be routing outbound traffic over your network and having a different path for the return then this gets complicated quickly).
I am TheRaven on Soylent News
You can't possibly SHOUT at us CONFIDENTLY like you KNOW, and ask for an explanation simultaneously.
Every end has half a stick.
Because for some completely unknown reason, IP spoofing is still a thing, and most routers still pass packets that claim to come from an IP that couldn't possibly be on the interface it connected from.
I can't even fathom why this is still a thing (or even why it was a thing in the first place) but unfortunately it is, and there doesn't seem to be any way to get these things actually fixed.
This is honestly one of the absolute biggest threats on the internet. Not because it enables this particular attack, but because it is the main thing that enables almost every attack. (It also happens to be one of the things that enables spam)
The problem isn't that the server sent a response, it's that it sent a response to the wrong person. This was accomplished by spoofing an IP. If the spoofing couldn't happen, then the attacker would only be able to DOS themselves.
> We modeled our capacity based on fives times the biggest attack
> that the internet has ever seen," Josh Shaul, vice president of web security at Akamai
Uhhhh, guess we'll see a 6x attack on github tomorrow.