Slashdot Mirror
US Amazon.com Website Down For Over 1 Hour
CorporalKlinger writes "CNET News is reporting that Amazon's US website, Amazon.com, has been unreachable since 10:30 AM PDT today. As of posting, visiting www.amazon.com produces an 'Http/1.1 Service Unavailable' message. According to CNET, "Based on last quarter's revenue of $4.13 billion, a full-scale global outage would cost Amazon more than $31,000 per minute on average." Some of Amazon's international websites still appear to be working, and some pages on the US Amazon.com site load if accessed using HTTPS instead of HTTP."
I guess somebody spilled beer on the servers? I had no idea the guys from FARK also ran Amazon.
I write sci-fi for metalheads
It works just fine for me right now.
Also now you are Slashdotting it!
I'm sure the sysadmins appreciate Slashdot sending thousands of requests their way while they're site's already down. While we're at it, maybe we should find someone with a papercut and start squirting lemon juice all over them.
Wait until a patent comes out for: "Taking a web presense offline, to generate discussion about the web presense, thereby increasing awareness about the site." Also, sucks to be the guy that stepped on the surge protector laying on the floor....
What are we going to do tonight Brain?
Believe me, if you've seen the code that runs that site, it's impressive it runs as well as it does. Try to imagine 900M static binaries that take almost an hour to link because of some tiny little code change, because they can't be fucked to make their deployment system deal with dynamic libraries reasonably.
Now then when they do come back on line, thet will get slashdotted by everyone trying to see what's going on.
Sig temporarily out of service.
Because this represents 31k USD every minute.
One of the top sellers on Amazon is the D&D 4th Edition Core Rules giftset. It apparently is only shipping to some pre-orders. The geeks are freaking out and untintentionally DoS'ing Amazon.
Posting on /. about a website with difficulties is like throwing a bucket of water on someone drowning.
Actually, considering how big /. is, this is more like using a fire hose on someone drowning.
I bet they briefly had Wiis in stock and the servers got overloaded with rapid-fire orders. Or perhaps a mouse ate through a wire in the server room. anything is possible.
Looks like they JUST put up a 'We're sorry' page within the past few minutes. I had happened to be on Amazon AS it went down.. it was working one moment then suddenly died while I was browsing their MP3's for download.
Slashvertizement at work.
I was just about to post saying that I had no problems getting to the site. I hit Amazon's home page, and it came up just fine for me...the first time. I was about to hit submit until I decided to also try navigating around the site a bit, log into my account, etc.; so I went back to try, and ran into the problem.
So, it seems to be working...at times.
The irony is you in fact have confirmed the worthiness of this article by making the obligatory "Why is this news?" post.
4th edition D&D books came out today. Coincidence?
just change the URL from HTTP to HTTPS and it works
so only port 80 servers are down
Not to nitpick or anything, but at $31,000 per minute, an hour outage would cost $1,860,000, not $31,000.
My sci-fi novel, Ghost Thief, is now available from Amazon.com.
"$31,000 per minute"
Even if accurate, that's assuming everyone who sees the error message will go somewhere else to buy their books.
I imagine some people would just wait to buy the book from amazon later when it is up again (probably very soon).
I hope it's better than The Cuckoos Egg but I wouldn't know, I couldn't place my order for Stoll's book.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
Exactly, except that not everyone that would have purchased their products in those 60 minutes will buy elsewhere. They hour they came back online they could make 1.9 x typical USD per minute. That and the fact that this is not really a holiday season of any sort, so sales are likely nowhere near the peak rates they reach around Christmas, New Years, etc...
So, because some loser can set up a few records in a completely different domain, this is supposed to somehow have some effect on Amazon's DNS?
Looks like you are the one who got pwn3d.
No, the real shocker is that they use Network Solutions which is a company that perpetually has it's thumbs stuck up their asses.
That whois lookup says absolutely nothing... I could add amazon.com.myserver.net as a dns record too, and it would have nothing to do with the lookups for amazon.com. The trick is to use whois to see what IP address www.amazon.com currently points at.
However, as has been pointed out, HTTPS works, so it's defininitely not a DNS issue. More likely someone along the chain corrupted a pooling link to the main http server and it propogated. I've done the same thing on apache2 servers in the past and had the same result; https still works fine, but http returns an error on key pages.
When I try I get to a page that says they think I'm a robot and I don't have access to see their website.
Well I think THEY are the robot. I don't know if I can win this argument...
With Amazon, this isn't really shocking at all. I'm soooo glad I don't work there any more.
Of course, it's not a hack.
A fully-qualified DNS domain name ends with a dot, so you should type 'whois amazon.com.' instead.
Those "hacked" results you are getting are just bogus amazon.com.foo.bar. subdomains.
throw new SuccessException("Sig read successfully");
Oddly enough this came up the last time something amazon went down.
;)
if I specify microsoft.com.looksatporn.com as a nameserver for a domain I have, it doesn't mean I'm hacking microsoft.com
Hush, the troll ran out of fingers and toes.
There, there little troll. Please continue your nonsensical rant.
Whatever this DNS thing is you're pointing out, it's not the first time it's happened, I am finding stuff from May 4, January, and even June 27 of last year with some of the same strings.
Slay a dragon... over lunch!
Any reason why? I had an interview with them so I'm curious.
It's all you people not typing in the write web address. Try it again. Make sure you put in the umlaut correctly. What do you mean there's no umlaut in Amazon.com? *Unplugs toaster and plugs back in Amazon's server* Wait 5 minutes and try again. --BOFH
Well, there's spam egg sausage and spam, that's not got much spam in it.
It's a giant cube farm, and their code is like some sort of crawling horror of reanimated spaghetti which long ago swallowed up and devoured all documentation. And then there's the deployment system. As I mentioned in another comment on this article, it can't deal with dynamic libraries. When I left, it was a real and immediate issue how we were going to keep a certain product's dependencies small enough that it would be able to *link* in a 32-bit virtual address space. The linker was up to something like 2.8 GB of working set.
That reasoning doesn't really work for me.
You'd have to factor in the ratio of income from the
US site v others (UK, etc.). IMHO, the US site is likely to be more profitable than others. You'd have to plow through an annual report to really know, and factor that in.
The larger flaw, though, is that you're subtracting one minute, when the title states > 1 hour. That implies going on A couple of million US$ in losses, which is significant, as investors don't know the reason, and caution would indicate that it could be recurring, such as the problems SalesForce has had. That hit their stock prices, etc.
The Amazon outage is more complex--TFA indicates that some of their services were unavailable for different amounts of time, etc. What are those service worth? All anyone has is a number--from CNET. Did they do anything like a real analysis, reading quarterly reports, etc? No, by long odds. Amazon does application hosting. What customers were affected, what percentage of the business is involved, and what do CxOs of large clients think?
The odds are actually quite good that many people give a crap. Investors (and CxOs) don't like uncertainty. It wouldn't surprise me to find some Wall Street analyst(s) making calls. Maybe it was an outage on a critical replication server, problem identified, fixed, and will provably never happen again. But maybe not. We'll see.
What you do with a computer does not constitute the whole of computing.
tm
Support TBI Research: http://www.raisinhope.org
Sounds kinda like my current job. Maybe I'll just pass if they decide to give me a second interview... Thanks for the info.
Maybe this should be modded Funny, but it's sure as hell not Informative (it's a joke/troll, duh)
I tried to click on a link for a DIY Home Chemistry Experiment Book and got:
We're sorry!
An error occurred when we tried to process your request. Rest assured, we're already working on the problem and expect to resolve it shortly.
If you were trying to make a purchase, please check Your Account to confirm that the order was placed.
We apologize for the inconvenience.
on the Amazon.com home page
...the future crusty old bastards are already drinking the Kool-Aid.
tried to access it from holland just now, got this message: We're sorry! You have been denied access to this feature because we believe you violated the terms, conditions, rules, guidelines or policies of our site in the past. If you believe we have taken this action in error, you may contact us at ad-help-us@amazon.com. We apologize for the inconvenience. Frequently Asked Questions Q: Why am I seeing this page? A: This page is usually shown when we believe that the request is coming from a robot or other automated source of requests. If you are not a robot please contact us immediately by emailing ad-help-us@amazon.com and we will reinstate your access to our website.
I would login to post this but I'm afraid of losing my "mole".
I received word about 30 minutes ago that Amazon has been the victim of a DDoS attack this morning. At first, their Ops team didn't realize they were under attack and thought it was a traffic spike related to a promotion, but after about an hour of throwing hardware at the surge they realized what it was. And once they tubed the source IPs in the botnet another crop of zombies showed up.
It looks like they are getting a handle on it now as things are better. Bad day to work in Ops at Amazon I guess. I'm cracking a beer in your honor now, fellas. Good luck.
Assuming that every one of those purchased was lost either to another seller or to some sort of desire-destroying void. I very much doubt either was the case for the vast majority of the "lost" sales.
How are sites slashdotted when nobody reads TFAs?
is that it assumes that they are loosing 31k a minute.. because people won't try back and buy the product. This is on par with the "a billion people DL'd my song from Napster, I lost all that money" when in reality, it sucks, was deleted 1/2 way thru the first listen.
amazon.com.myserver.net as a dns record too
OLD CROTCHETY MAN MODE:
HOST RECORD
--- I do not moderate.
A bit strange, the people wondering why this is news. Amazon provides the backend for a number of web services with their EC2 and AWS platforms. This is going to make third parties seriously consider whether or not they want to trust Amazon with their business.
That is yet another reason why this is Real News(tm).
What is humor if not pain tempered by time?
Amazon: A credit to Jeff Bezos. I love Amazon prime, I enjoy my Kindle, I like the prices and the one click purchases and the mp3 previews and the look inside the book and the no-bullshit mp3 store (which I don't use) and the useful reviews and the decent recommendations, etc ! Amazon almost never leaves a bad taste in my mouth and keeps innovating with features that are actually not RETARDED or HOSTILE to me! ZOMG!
Amazon is as good as eBay-Paypal is evil. Both are outstanding products but one is loved and one is hated.
Sooo...in the time that I wrote this post, Amazon lost enough money to sustain me my entire life. That's sad.
Stephan
http://stephan.sugarmotor.org
-- Language is a virus from outer space.
Who's the ass-clown that marked this informative?
whois information has nothing to do with DNS. You should use whois amazon.com -h whois.networksolutions.com to get proper info. What you saw is a result of wildcard search in wrong whois server.
The DNS servers for a domain name are announced in root DNS servers - and there, everything is fine. For example, dig NS amazon.com @a.gtld-servers.net return correct DNS servers: udns1.ultradns.net. and udns2.
However, dnsreport show lots of errors with nameservers:
http://private.dnsstuff.com/tools/dnsreportsmpl.ch?domain=amazon.com
I love how you make some random completely uninformed guess and say "more than likely". I could think of thousands of other possible problems, but somehow, because it happened to you once, it's "more than likely".
Maybe it was due to people who also wanted to commend about how Ubuntu is the worst game ever?:
http://digg.com/linux_unix/Ubuntu_Worst_Game_Ever
How will we stop the turrists if we cannot shop?
IMDb, which is owned and operated by Amazon, was down for a little while as well.
Because a communication disruption could only mean one thing: invasion.
Well, they lost my business.
I was all set to make quite a few purchases on Amazon, my first ones in several years, only to have this happen in the middle of my shopping. I ended up finding some of the same products over at eBay and then later I found the remaining ones at a store in the mall. Now, Amazon gets nothing from me.
There are other venues that sell the same items, so I seriously doubt they're going to get 1.9x sales. The fact that brand loyalty is extremely low on the internet doesn't help any; people won't miss a beat in jumping ship and finding another store that offers the same item. People do internet shopping for the convenience, and if the store is down, it's inconvenient to wait.
It's down again.
It was just down for me again at 15:08 PST (same "service unavailable" HTTP error), after it had been working again for a while, so they clearly have not completely resolved whatever the issues were.
I run a blog that uses one of amazon's wishlist widgets, and my first indication that something was wrong was that my wishlist wasn't being loaded in the side bar.
I just waited for them to come back before I bought the stuff I wanted (around $200). I would rather wait on Amazon than give my money to Ebay and Paypal (via sellers on Ebay).
CNET has updated the post to include a statement from Amazon.com that the outage is over. The total downtime was something like 5 hours. From the CNET follow-up article:
"But as to the explanation, the company only hinted that its complicated computing infrastructure was, unsurprisingly, a culprit.
'Amazon's systems are very complex and on rare occasions, despite our best efforts, they may experience problems. We work to minimize any disruption and to get the site back as quickly as possible," the company said, declining to comment further.'"
The Mason HQ wiki says the front-end is still all done in Mason.
Actually its probably more like 15-20k USD per minute. Presumably, like most retailers, they make a lot more money in November-December than the rest of the year.
You completely overlooked the revenue vs profit mistake in the summary. If they are losing $31000 in revenue every minute, they are also not-losing 0$N31000 worth of merchandise. Assuming a phenomenal 50% profit margin, they would only actually be losing-money losing $15500 per minute.
Gold is about 5 times faster than the regular GNU linker. It will only work on x86 code (64bits included) and ELF targets (linux/solaris)
Half the technology news organizations just gave them a free advertisement. I think they can make up for one hour.
Unless & until they issue a retraction, I seriously think that they were hacked the old fashioned way. Forget this new age DDOS & DNS crap that isn't a "hack" so much as it's a weapon. I just hope the hackers didn't get __my__ credit card number. Chances are they caught onto what was happening too late.
Yes, but there are only so many hours in a day. Purchases are made 24/7 - surely not everyone abandoned their purchasing plans, but there were several hours where they could have been making money, but weren't.
They're $31,000/hr * hours poorer this year. It's a massive stiffing you of your hours this week, but you might get a few more next week (or next year).
Not to mention the extra money it must cause to repair equipment/buy new equipment/pay people to do said things/etc.
DATABASE WOW WOW
IMDB has been down for well over an hour. Thats kinda news isnt it?
As an Amazon.com employee, I vaguely know what happened. All I'll say is that the outage wasn't completely due to internal sources :)
grandparent's info is years out of date.
Yesterday I was having major issues with Amazon's CC auth. I placed an order, it accepted my CC credentials, but about two hours later, I'd get an E-mail saying:
Your credit card payment for the above transaction could not be completed.
An issuing bank will often decline an attempt to charge a credit card if
the name, expiration date, or ZIP Code you entered at Amazon.com does not
exactly match the bank's information.
Valid credit card information must be received within 3 days, otherwise
your order will be void.
I re-tried the same CC, two hours later, got the same. Then I called the CC company and they said there was no issues with the card and they could even see the pre-auth from Amazon for the correct amount. Then I called Amazon's customer (non-)service and was fed a "Well, it's gotta be your financial institution line."
Re-tried the card again, same issue.
Called Amazon cust-service again, only this time I got a more helpful person on the phone. She put me on hold for four minutes to investigate, then came back on and just as she began explaining what the problem was, the call was dropped.
At his point, I re-tried with another card (one that I use all day, every day), same issue.
At this point I called Amazon again, but was told they couldn't access my order because their customer services system was being "upgraded."
Did anyone else experience, similar?
I should also mention that today I've used both cards and different retailers with no issues.
"So is the BSD licence even more 'free' (than GPLv2)? Yes. Unquestionably." --Linus Torvalds (TinyURL.com/2vugzl)
After searching for something using kanji (just out of curiosity to see if it would be on the US amazon), amazon started giving me a "we're sorry" page today, basically saying that I was banned.
I sent them a couple e-mails, but I guess I'm seeing that because their system is messed up...I hope. Either way, it's really annoying.
No, but Father's Day is just around the corner.
Non, je ne veux pas coucher avec toi ce soir.
I don't know if it's just me doing something wrong, but it seems IMDb is down. Amazon seems to be working fine right now though.
...and as soon as I wash the handles of my pliers, I'll start working on the really interesting bits.
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
I guess this has something to do with why IMDb is now down to. At least for me and http://downforeveryoneorjustme.com/
A friend interning at Amazon says it was a DDoS attack, and it happened right when they were having an all-hands meeting.
He used up his vowel quota in the first word.
Amazon is another business that needs to run to the library for "I.T. Wars: Managing the Business-Technology Weave in the New Millennium." The irony is, Amazon sells it. I urge every business person and IT person, management or staff, to get hold of a copy of this book. Our CEO has read it. Our project managers are on their second reading. Our vendors are required to read it (they can borrow our copies if they don't want to purchase it). Any agencies that wish to partner with us: We ask that they read it. Do yourself a favor and read this book - then ask your boss to read it - then ask your staff and co-workers to read it. If you get a chance, read the author's interview here: http://www.businessforum.com/DScott_02.html
with this
A major web-site off-line for hour, oh the horror, the HORROR!
I love how AC says I say "more than likely" when I said "more likely (than a DNS issue)". Why not suggest one of your thousands of other problems instead of criticizing me for suggesting a common error that matches the scenario is more likely than an error that obviously hasn't happened?