Slashdot Mirror


US Amazon.com Website Down For Over 1 Hour

CorporalKlinger writes "CNET News is reporting that Amazon's US website, Amazon.com, has been unreachable since 10:30 AM PDT today. As of posting, visiting www.amazon.com produces an 'Http/1.1 Service Unavailable' message. According to CNET, "Based on last quarter's revenue of $4.13 billion, a full-scale global outage would cost Amazon more than $31,000 per minute on average." Some of Amazon's international websites still appear to be working, and some pages on the US Amazon.com site load if accessed using HTTPS instead of HTTP."

32 of 228 comments (clear)

  1. Beer on the server? by Lilith's+Heart-shape · · Score: 5, Funny

    I guess somebody spilled beer on the servers? I had no idea the guys from FARK also ran Amazon.

  2. But... by Shikaku · · Score: 5, Informative

    It works just fine for me right now.

    Also now you are Slashdotting it!

  3. This will surely help by moderatorrater · · Score: 5, Funny

    I'm sure the sysadmins appreciate Slashdot sending thousands of requests their way while they're site's already down. While we're at it, maybe we should find someone with a papercut and start squirting lemon juice all over them.

    1. Re:This will surely help by sloth+jr · · Score: 4, Insightful

      It's really not all that difficult to survive a slashdot pounding for commercial web shops, even for dynamic content. Generally speaking, a popular link is going to generate perhaps 500k views a day for a day and some.

      Only exceptions would be if there was a lot of heavy content being served on each page turn, saturation of one's uplink is a possibility - 10Gb links to the backbone aren't that common as yet, and CDNs like Akamai helps alleviate a good portion of that traffic.

      My totally unsubstantiated guess is there was some DNS fooage that directed sites to a down cluster or possibly a screwed up CDN leg, but I'll be interested to see what's truly up.

      sloth jr

    2. Re:This will surely help by Goaway · · Score: 5, Interesting

      You think traffic from SlashDot would even be noticeable on Amazon's servers? You have some delusions of grandeur there.

    3. Re:This will surely help by DrHanser · · Score: 5, Insightful

      Digg sends far more traffic to a site than Slashdot does (obviously it wasn't always this way). And digg's traffic isn't particularly noteworthy to a site of any reasonable size. (Say, Ars Technica, nevermind amazon.)

      Yahoo Buzz, on the other hand, sends *huge* amounts of traffic, noticeable to sites like, again, Ars but again no disruptions of service*. But I doubt that amazon would even hiccup. If you think slashdot would even be a blip on amazon's radar, you have some serious delusions about 1) slashdot's size 2) amazon's size or 3) both.

      * According to one of the devs.

      --
      What is humor if not pain tempered by time?
  4. Patents by QuantumRiff · · Score: 5, Funny

    Wait until a patent comes out for: "Taking a web presense offline, to generate discussion about the web presense, thereby increasing awareness about the site." Also, sucks to be the guy that stepped on the surge protector laying on the floor....

    --

    What are we going to do tonight Brain?
    1. Re:Patents by Samus · · Score: 5, Funny

      The Twitter guys will have prior art.

      --
      In Republican America phones tap you.
  5. So, it finally happened... by geekgirlandrea · · Score: 5, Interesting

    Believe me, if you've seen the code that runs that site, it's impressive it runs as well as it does. Try to imagine 900M static binaries that take almost an hour to link because of some tiny little code change, because they can't be fucked to make their deployment system deal with dynamic libraries reasonably.

    1. Re:So, it finally happened... by geekgirlandrea · · Score: 5, Interesting

      Um, I used to work there. Believe it or not, there are some people here with real jobs and stuff.

    2. Re:So, it finally happened... by Anonymous Coward · · Score: 5, Funny

      Okay mythical female Slashdot reader, with your unicorn and leprechauns friends chasing chupacabres out of the Amazon codebase. We believe you.

    3. Re:So, it finally happened... by geekgirlandrea · · Score: 5, Funny

      Yeah, it wasn't like that at all. There used to be a few chupacabras, but I think the shoggoths ate them all, and those leprechauns never did any work. They'd just wander around all day, picking people's pockets and teleporting away.

    4. Re:So, it finally happened... by AuMatar · · Score: 5, Informative

      Well, if you worked there in the last 2 years you'd know that the giant monolithic app is dead and not mourned. I drew the short straw and had to sit in all day on the con call when they were taking it down area by area.

      --
      I still have more fans than freaks. WTF is wrong with you people?
    5. Re:So, it finally happened... by pclminion · · Score: 4, Interesting

      Believe me, if you've seen the code that runs that site, it's impressive it runs as well as it does. Try to imagine 900M static binaries that take almost an hour to link because of some tiny little code change, because they can't be fucked to make their deployment system deal with dynamic libraries reasonably.

      Fuck up a dynamic library and you fuck everything. Fuck up one of those 900M programs and you've fucked 1/900M'th of everything.

      What does Amazon's back end compile for? If it's Linux, that's an issue right there. The GNU linker has pathological behavior when linking large numbers of static libraries. I work on a relatively small (~1 million line) codebase and it takes about ten minutes to link. Link it on another platform (e.g. Solaris) and it links in about five seconds.

      The problem isn't the huge number of libraries. The problem is that the linker blows.

    6. Re:So, it finally happened... by Anonymous Coward · · Score: 4, Informative

      GP meant a single ~1GB binary, not 900 million binaries. See Obidos
      GP is approximately 3 years out of date. See Gurupa

      Since I can't give any details directly, I'll let wiki do it.

  6. Re:How is this news? by felipekk · · Score: 5, Insightful

    Because this represents 31k USD every minute.

  7. D&D did it. by Silicon+Jedi · · Score: 5, Funny

    One of the top sellers on Amazon is the D&D 4th Edition Core Rules giftset. It apparently is only shipping to some pre-orders. The geeks are freaking out and untintentionally DoS'ing Amazon.

    1. Re:D&D did it. by OrangeTide · · Score: 4, Funny

      I sure am glad you didn't take his statement seriously. Would have been pretty silly to argue a point against that.

      --
      “Common sense is not so common.” — Voltaire
  8. Re:We're sorry... by felipekk · · Score: 5, Funny

    Hmmmm... sounds like they are being DoS'ed by MediaDefender.

  9. Re:How is this news? by Anonymous Coward · · Score: 5, Insightful

    Because this represents 31k USD every minute. That assumes that everyone who would have bought something doesn't just try again when the site's back up. Nevermind that the number quoted is talking about a global outage -- this is just a partial outage.
  10. Re:How is this news? by mixmatch · · Score: 5, Insightful

    Exactly, except that not everyone that would have purchased their products in those 60 minutes will buy elsewhere. They hour they came back online they could make 1.9 x typical USD per minute. That and the fact that this is not really a holiday season of any sort, so sales are likely nowhere near the peak rates they reach around Christmas, New Years, etc...

  11. Re:do a whois. Looks like DNS got pwn3d. by Em+Adespoton · · Score: 4, Informative

    That whois lookup says absolutely nothing... I could add amazon.com.myserver.net as a dns record too, and it would have nothing to do with the lookups for amazon.com. The trick is to use whois to see what IP address www.amazon.com currently points at.

    However, as has been pointed out, HTTPS works, so it's defininitely not a DNS issue. More likely someone along the chain corrupted a pooling link to the main http server and it propogated. I've done the same thing on apache2 servers in the past and had the same result; https still works fine, but http returns an error on key pages.

  12. They Think I'm a Robot by Bryansix · · Score: 4, Funny

    When I try I get to a page that says they think I'm a robot and I don't have access to see their website.

    Well I think THEY are the robot. I don't know if I can win this argument...

  13. Re:do a whois. Looks like DNS got pwn3d. by quazee · · Score: 5, Informative

    Of course, it's not a hack.

    A fully-qualified DNS domain name ends with a dot, so you should type 'whois amazon.com.' instead.
    Those "hacked" results you are getting are just bogus amazon.com.foo.bar. subdomains.

    --
    throw new SuccessException("Sig read successfully");
  14. The problem isn't Amazon . . . by UnknowingFool · · Score: 4, Funny

    It's all you people not typing in the write web address. Try it again. Make sure you put in the umlaut correctly. What do you mean there's no umlaut in Amazon.com? *Unplugs toaster and plugs back in Amazon's server* Wait 5 minutes and try again. --BOFH

    --
    Well, there's spam egg sausage and spam, that's not got much spam in it.
  15. Re:OH NOES by VENONA · · Score: 4, Insightful

    That reasoning doesn't really work for me.

    You'd have to factor in the ratio of income from the
    US site v others (UK, etc.). IMHO, the US site is likely to be more profitable than others. You'd have to plow through an annual report to really know, and factor that in.

    The larger flaw, though, is that you're subtracting one minute, when the title states > 1 hour. That implies going on A couple of million US$ in losses, which is significant, as investors don't know the reason, and caution would indicate that it could be recurring, such as the problems SalesForce has had. That hit their stock prices, etc.

    The Amazon outage is more complex--TFA indicates that some of their services were unavailable for different amounts of time, etc. What are those service worth? All anyone has is a number--from CNET. Did they do anything like a real analysis, reading quarterly reports, etc? No, by long odds. Amazon does application hosting. What customers were affected, what percentage of the business is involved, and what do CxOs of large clients think?

    The odds are actually quite good that many people give a crap. Investors (and CxOs) don't like uncertainty. It wouldn't surprise me to find some Wall Street analyst(s) making calls. Maybe it was an outage on a critical replication server, problem identified, fixed, and will provably never happen again. But maybe not. We'll see.

    --
    What you do with a computer does not constitute the whole of computing.
  16. $31,000 per minute! by DirePickle · · Score: 5, Insightful

    would cost Amazon more than $31,000 per minute on average.
    Because obviously if someone tries to buy something and Amazon is broken for an hour, they're just going to not-buy it or buy it from a competitor. Because you definitely can't wait an extra hour to place an order when it'll take 2-10 days for the product to get shipped to you anyway.
  17. Re:Analogy by ColdWetDog · · Score: 5, Funny

    Where is BadAnalogyGuy when you need him!!!!

    Out in the back, working on his car.

    --
    Faster! Faster! Faster would be better!
  18. Re:AWS and EC2 by dave420 · · Score: 5, Informative

    Not particularly. Their S3 and EC2 services are completely seperate from their webserver. All throughout this outage, S3 and EC2 have been running flawlessly, as usual. If anything, this is a great reflection on how resilient their clusters are.

  19. Re:Great move! by fireboy1919 · · Score: 4, Funny

    Yes, they do.

    Which is cheesy awesome.

    --
    Mod me down and I will become more powerful than you can possibly imagine!
  20. Re:Analogy by CarAnalogy · · Score: 5, Funny

    No, that's me you're referring to.

  21. Amazon declares outage is over. Light on details. by CorporalKlinger · · Score: 5, Informative

    CNET has updated the post to include a statement from Amazon.com that the outage is over. The total downtime was something like 5 hours. From the CNET follow-up article:

    "But as to the explanation, the company only hinted that its complicated computing infrastructure was, unsurprisingly, a culprit.

    'Amazon's systems are very complex and on rare occasions, despite our best efforts, they may experience problems. We work to minimize any disruption and to get the site back as quickly as possible," the company said, declining to comment further.'"