Slashdot Mirror
Inside Amazon's Data Centers
1sockchuck writes "Amazon Web Services usually doesn't say much about the data centers powering its cloud computing platform. But last week the company held a technology open house to discuss the company's infrastructure, sharing cost data and a glimpse of a modular data center design. The key point: AWS is growing like crazy. 'Every day Amazon Web Services adds enough new capacity to support all of Amazon.com's global infrastructure through the company's first 5 years, when it was a $2.76 billion annual revenue enterprise,' said AWS Engineer James Hamilton, whose presentation (PDF) is available online."
Don't read this if you are a (non-tech) manager. It might contradict the self-evident truth that the cloud really is up in the sky, managed by security conscious fairies and run on pixie dust.
Then again, I'm likely posting on the wrong site here...
Please use Coral Cache to reduce the load on the original.
"Every day Amazon Web Services adds enough new capacity to support all of Amazon.com's global infrastructure through the company's first 5 years, when it was a $2.76 billion annual revenue enterprise,"
What good is that when their network design is so flawed that a single human error can bring the whole service down for a week?
Amazon Web Services? No, thanks, I prefer reliability.
I think that it was manager-safe enough. There were a few diagrams that you should be careful to skip over(a few had percentage numbers that weren't finance related!); but the rest had numbers with dollar signs in front of them safely ensconced next to a variety of recognizable corporate logos with a generic-but-high-tech photo of some server racks in the background.
A few good buzzphrases as well. I'm definitely going to ask my department about why they aren't being more proactive and action-oriented about getting our network "on the Moore's law path". We'll never be ready for exascale cloud-centric computing if they don't...
presentation said average is $1450 per server which is an entry level 1U. why not just buy fewer larger servers to share the workload? it would probably save on power costs?
Larger servers are prohibitively more expensive than el cheapo low-end servers, but fail at effectively the same rate.
As for power costs, no, not necessarily - unless your large consolidation boxes are underpowered. Most of the 4U boxes I've dealt with have massive power requirements.
I thought this was the common consensus on how the "cloud" looks like: http://i55.tinypic.com/2i8j8mf.jpg
Probably the most interesting slide is "Sea Change in Networking" where they compare side-by-side the openness and accessibility of software and hardware for commodity servers vs. the complete lack of it for networking equipment. I.e., on the server side you have access to APIs and hardware information to create and customize your own good stuff. Whereas on the networking equipment side, whatever comes out of the box you'll have to take it or leave it. With Amazon being such a large consumer of this equipment, I wonder (A) what's the opportunity for new networking equipment companies with a different attitude toward openness and how Amazon might like to start buying their gear, or (B) whether the incumbent suppliers will respond by tweaking their offerings so that Amazon would be less disgruntled.
Am I the only one that thinks that cloud computing is really NOT going to take off? Amazon is spending a boatload of money, in the hopes that other people are going to be willing to offload a boatload of money...in the hopes that they can process data faster/better/etc off-site. Just the security issues alone concern me, but the additional bandwidth is what really gets things going... Just saying - from a business perspective (and my perspective as a network admin), it seems that Cloud Computing is a 90's .com bubble that will pop in the near future...can Amazon (or any other company) REALLY offer a reduction in costs, or an increase in performance that justifies the (obvious increase in) cost, especially when you look at the whole package? I think not.
Same reason not every company has made the switch on the backend to free server-level operating systems (FreeBSD or Linux, namely) - it doesn't matter if the platform may perform better in every regard or has inheritantly better security and even can do the job just as well or better...it gets rejected simply because the costs will increase - in the case of linux/unix, because the skillset to operate and maintain it is more rare and rare = higher price...and unlike a product you purchase (optionally) once like software...support/maintenance is annual. Cloud computing...would be annual, vs a piece of hardware you own, with software that you own the rights to use
You need to factor in the cost of cooling those KwHs too. Not just that, but there are large area's of the planet where you can't buy a KwH for 0.10 US dollar. Try 3 times as much for an average European country. You'd be looking at two to six times your $350/year easily for running your 400W server just in power costs. Also, those $4500 CPUs won't cost that much for large companies like Amazon and Google. Rest assured they pay probably substantially less than $1000 for those CPUs. These companies are big enough to have their own servers made to their own specs. They don't pay list price to AMD, Intel, Dell, HP or IBM, but they do have to pay for actual power bills, until they have their own power plants.
I was promised a flying car. Where is my flying car?
I've seen a few different data centers, they all look about the same, why should we care? I'm pretty sure it was SUN Microsystems who came up with the modular shipping container design, which was back in the day.
Is this the same web service systems that couldn't support the demand(how much could it be) for lady GaGa's album?
the opportunity to crow about how effective it might be in generating bitcoins.
There is no right to feel safe thru security vaudeville at the expense of everyone's freedom, privacy and tax money.
The problem with networking is, you need dedicated hardware (ASICs) to able to push the packets at those speeds.
Those ASIC's are not the same for every vendor, I wouldn't be surprised if their are big differences in implementations and very strongly tied to the 'control plan' (the software running on the management-CPU).
Ofcourse the control plane could be a lot more open.
The closest things I can think of are the open source software which can be used to program FCGA's: the NetFPGA and liberouter project do something like that. They both can handle packets at 10GB/s and use FPGA's.
And supposedly their is one vendor who created open source code for their switches. But isn't a big vendor and I don't think anyone has actually seen the code yet.
Also I don't remember the name right now. :-)
New things are always on the horizon
The anonymous coward claimed that "the whole service [was] down for a week". The link you posted isn't even remotely close to that.
www.clarke.ca
Is that they are, IMHO, extremely hypocritical in how they conduct themselves.
On the one hand, they put themselves forward as this oh so "wholesome" family environment, while doing absolutely NOTHING to stop their "Pro" sellers from listing the most vulgar items you would care to search for.
These pages are also listed with the helpful "Sell yours here!" but if you actually set up an account and attempt to sell said items yourself, you run the definite risk of having your account terminated by what might as well be an anonymous email with no way to explain EVEN IF you intent was simply to alert them to their flawed system (meaning if you weren't just trying the old "they do it too" excuse)
Now, before someone mistakes me for a conservative (little "c", not in relation to anything political) prude, let me say I know this because I set up such an account :)
I saw that the adult items were selling and since I didn't have a "Pro" account -- that is too say, I couldn't sell anything that wasn't already listed on Amazon's site, you can't create pages yourself unless you have a "Pro" account -- I simply took it on faith that if Amazon was allowing the pages to remain, they must have been kosher with it. That is until about my first week in when my account was permanently suspended.
Now I want to make clear that I know full well what Amazon says -- I know they forbid "pornography" (though their description of "pornography" is, in my opinion, open enough to include almost anything they might simply decide they don't like) and I'm not claiming I should get some special exemption from the rules -- I'M SAYING THAT THEIR BEHAVIOR -- leaving pages with items they claim to forbid up and encouraging others to "join up" -- WOULD -- if done by a police officer on the street -- BE CONSIDERED ENTRAPMENT IN A COURT OF LAW.
I've since switched to eBay, where they have a section for "adult" items with age verification, etc. and that's cool. I guess my complaint, though, is that eBay never claimed they don't allow such products, Amazon does and, for lack of a better term, that makes Amazon a liar. Reminds me of the scene from the pilot episode of "Boardwalk Empire", when the kid shoved the envelope into Steve Buscemi's hand after robbing the bootleggers in the woods, he said "I didn't ask for this". The kid turned to him and said "That's your cut, regardless -- you can't be HALF a gangster".
His point was that if you're going to be a crook, THEN BE A CROOK and stop pretending you're anything else. Likewise, if Amazon intends to make money selling "unwholesome" merchandise THEN DO WHAT EVERYONE ELSE DOES -- speaking of setting up an area with age verification, etc. -- AND STOP PRETENDING YOUR SOMEHOW "BETTER" THAN YOU REALLY ARE.
"Don't be a martyr -- BE THE ONE WHO GOT AWAY!"