Slashdot Mirror
Akamai DNS Outage Messes up Net
katre writes "Checking all my favorite sites this morning, I saw that about half a dozen seem to be offline. Trying to figure out why, I found an interesting article on the front page at http://isc.incidents.org/. Seems that the problems at Akamai are screwing over Yahoo, Google, Microsoft, Fedex, Xerox, Apple, and others. Whatever happened to my decentralized net with no single point of failure?"
Yahoo is already resolving through scd instead of akamai. I didn't check any of the others.
If you clear your cache, you will probably get the new entries, unless your ISP hasn't caught onto the problem yet.
...I can't even get to http://isc.incidents.org/
This should cause some problems for akami, they had an outage may 24th. Once can be overlooked twice? these are some big companies they are going to be calling them. I bet there is some sweating techs in the cool noc right now
War isn't about who's right. It's about who's left.
Pwned by CNAME to Akamai?
(You can't have CNAME records for the base domain, hence google.com would have had an A record instead, whilst www.google.com would have been a CNAME to akamai)
It would be hard to do most of my work (Server Maint.) without the net. I might have to actually go to the servers instead of ssh. Wait, what am I talkin about, without the net I wouldn't HAVE a job.
How to Speak Leet
Uhm, the root servers are not overloaded... this has nothing to do with the root servers, this has to do with Akamai having problems.
They have a private cached network they sell access to. It's like taking a service road around crowded highways to get closer to the final destination.
One of the companies I used to work for used Akamai, nice network... not so great customer service unless you are a really big customer.
This sig is the express property of someone.
Solution to akamai problems:
go to <a href="http://www.dnsstuff.com/">your favorite DNS lookup page</a> and lookup the akamai hosted site. (getting the real address rather than the akamized version) Now open your hosts file and add that in.
Now you will always get the non-akamized version of that site. Akamai problem solved.
I keep google in my hosts just so I can be sure that DNS issues like this won't cut me off from my favorite search engine.
From NANOG:
From here neither www.google.com, nor www.apple.com work. Both seem to return CNAMES to akadns.net addresses (eg, www.google.akadns.net, www.apple.com.akadns.net), and from here all of the akadns.net servers listed in whois are failing to respond.
The real cost of a web site dropping is a lot more difficult to figure out than you might imagine. Say Amazon goes down for a couple of hours. Are all those potential sales lost forever? I doubt it. Some people will just come back and order later. The firm is unlikely to see any long term impact unless the outage becomes habitual. Non-retail sites probably have even more flexability. About the only area in which an outage could have a real, long term adverse impact would likely be in financial services. If Schwab goes down for half a day they will suffer big time for a long time. If you're talking "the economy" as in the big picture economy" suffering - forget it. Web based commerace isn't that important yet.
It appears that, at around 8:30 AM EDT (US Eastern Daylight Time), Akamai's DNS network experiened some kind of major failure. All of their DNS servers (that anybody could find) were not responding to DNS queries. It appears that Akamai started to come back online at around 10:00 AM EDT.
Since a great many big name sites use Akamai, this effectively made large parts of the Internet unreachable. The destination servers themselves were up, but clients were unable to turn names (like www.example.com) into network addresses (like 192.0.2.42).
As Akamai maintains dozens, if not hundreds, of DNS servers across the globe, it is extremely unlikely that this was due to a normal equipment failure or DoS attack. Some kind of internal system trouble is much more likely. Whether a deliberate attack, or an accident, is unknown to me at this time. It could just be an internal configuration change blew up in a really bad way. Sh*t happens.
I do not know if this was just an Akamai DNS problem, or if other Akamai services were also affected.
Due to the way Akamai is usually implemented, it happened that, in many cases, the second-level domain names (like example.com) worked, but subdomains (like www.example.com and mail.example.com) did not. This is because most organizations put in CNAME records (pointing to names in *.akadns.net) for the subdomains. You cannot use a CNAME record for a domain that has other records, though, so most domains still had traditional A records, on their own nameservers, at the second-level.
The following sites/organizations are known to use Akamai: Yahoo, Google, Microsoft, Altavista, FedEx, Xerox, Apple
dragonhawk@iname.microsoft.com
I do not like Microsoft. Remove them from my email address.
From NANOG mailing list again:
Google pulled references for akamais dns servers a short period ago. they are presently serving their own dns requests.
Also:
People seem to be getting around this by changing their DNS entries.
E.g. www.yahoo.com always used to be a CNAME for www.yahoo.akadns.net. But
now:
# host www.yahoo.com
www.yahoo.com is an alias for www.dcn.yahoo.com.
www.dcn.yahoo.com has address 216.109.118.64
www.dcn.yahoo.com has address 216.109.118.65
www.dcn.yahoo.com has address 216.109.118.66
www.dcn.yahoo.com has address 216.109.118.67
www.dcn.yahoo.com has address 216.109.118.68
www.dcn.yahoo.com has address 216.109.118.69
www.dcn.yahoo.com has address 216.109.118.70
www.dcn.yahoo.com has address 216.109.118.71
www.dcn.yahoo.com has address 216.109.118.72
www.dcn.yahoo.com has address 216.109.118.73
www.dcn.yahoo.com has address 216.109.118.74
www.dcn.yahoo.com has address 216.109.118.75
Which is owned by Yahoo! (via HotJobs.com).