Slashdot Mirror
Google Envisions 10 Million Servers
miller60 writes "Google never says how many servers are running in its data centers. But a recent presentation by a Google engineer shows that the company is preparing to manage as many as 10 million servers in the future. At this month's ACM conference on large-scale computing, Google's Jeff Dean said he's working on a storage and computation system called Spanner, which will automatically allocate resources across data centers, and be designed for a scale of 1 million to 10 million machines. One goal: to dynamically shift workloads to capture cheaper bandwidth and power. Dean's presentation (PDF) is online."
Pretty soon, Google will BE the Internet.
There are 01 kinds of cars in the world. The General Lee, and everything else.
That's a lot of machines to try and shift bandwidth and power costs around the place.
But what if the plan is to spread out to hundreds of places? Then the total number doesn't look that high if there's only 1% of servers actually doing anything.
cool I guess... what do they do with their old machines?
The sound you just heard was the collapse of the global Google enterprise network.
Seriously, you should architect for way more than you need during the life of that architecture, and plan on re-architecting as needed to grow to some upper bound beyond which you will never need.
Google will be fine if they only plan on actually building 5M servers before raising their architecture limit.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
I'd be interested to know how google disposes of all of their servers. Anybody have insight on this? If these are cheap, throw away servers, I'd be interested in what their expected lifetime is and what is done with them when they are refreshed with newer hardware.
10 Million Servers to serve a World Population (whom not even all have Internet connections) of roughly 6.792 billion... It's Insane...
for the C.I.A..
The Uber-Google-App.
Yours In Uglegorsk,
K. Trout
Hopefully this puts to rest the delusion that there is some economic benefit of higher processor utilization in cloud computing schemes.
"I assumed blithely that there were no elves out there in the darkness"
If they do not have a plan in place to grow beyond 10M before they reach the 5M mark, they are asking for trouble.
If they really plan on not reaching the 5M mark, or they plan on looking into ways to pass 10M while there is still plenty of time to do so, then they are doing the right thing.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
It's pronounced Boorgle... and resistance is futile.
One Server Per Child?
"Action without philosophy is a lethal weapon; philosophy without action is worthless."
They should put that on their website,... before long it'll be: "Google: Billions and Billions of Servers." Of course, McDonald's just might have a problem with that,...
Google is starting to sound more and more like one of those advanced societies where everything is automated, but everybody forgets how everything works.
For reference, see: Logan's Run, STTNG: When the Bough Breaks, etc.
How many servers does this thing need to become self-aware?
Funny may not give karma, but +5 Informative never made anyone snort coffee out their nose.
The entire content of the Internet fits in a 20x8x8 box operated by the Internet Archive. Cuil, which searches as much of the Web as Google, has one relatively modest data center. About half the system does the crawl and builds the index; the other half answers queries. So Google's main search engine function doesn't really require that much capacity by current standards. Of course, Google has a huge number of query servers front-ending the main index, which is of course replicated.
Why does Google need so much server capacity? YouTube? Command completion? GMail spam filtering? Ad serving?
1981: 640K ought to be enough for anybody.
2009: 10 Million servers ought to be enough for any company.
The NSA already has Google beat.
At a million square feet, the mammoth $2 billion structure will be one-third larger than the US Capitol and will use the same amount of energy as every house in Salt Lake City combined.
...
Lacking adequate space and power at its city-sized Fort Meade, Maryland, headquarters, the NSA is also completing work on another data archive, this one in San Antonio, Texas, which will be nearly the size of the Alamodome.
Now, if only the NSA released their specs in terms of Libraries of Congress....
If you have something that you dont want anyone to know, maybe you shouldnt be doing it in the first place -Eric Schmidt
you dirty fucking commie.
This is a battle. On one side, we have 10 million servers. On the other side, we have 9000 penises. It will be brutal.
From 1,000,000 to 10,000,000?
Are the minimum requirements for this system seriously 1 millions servers?
That doesn't seem to scale well. Should be able to at least scale down to 10 machines so I can run it at home ;-)
But they won't really live up to their name till they have a googolplex.
Architects architect architecture.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
10million... that's cool, but still a far ways from Google becoming anything real:
Keep working Google... you still have (10^100 - 10^7) = 10^93 servers to add before becoming a physical entity (Google Universe edition?).
Only 10m, is that enough for the whole world, yet?
I simply can't imagine how they can manage so many computers without Microsofts Active Directory.
One Server Per Human?
Hmm...amusingly Google was down while trying to do some research for this post!
"Google Envisions 10 Million Servers" => Well, I just imagined a beowulf cluster of those server farms. Your move, Google! And none of that infinity plus one stuff.
Imagine a Beowulf cluster of them! /obligatory
Google - the company that is actually out doing the things M$ has been TALKING about doing for a couple of decades. Doing it cheaper, better, faster, more securely, and in a more open way.
People can talk about the evil of Google (Massive, Borg-like, etc), but given a choice, I'd choose Google over M$ every day and twice on Sunday.
What if anything does this (running 10M servers) do to Linux's market share?
Subtraction is not the same thing as division.
10^100 - 10^7 is, to the nearest integer power of 10, 10^100, not 10^93.
10^100 / 10^7, on the other hand, is 10 ^ (100-7), or 10^93, though.
I'm not sure you know quite what "-" means.
...wait, what?
Reminds me of the IBM server commercial. 'Servers are our friends' http://www.youtube.com/watch?v=73bMSNPc3Ak
"Visualize" was insufficiently cromulent?
Ahh - My eye!
The doctor said I'm not supposed to get Slashdot in it!
May 2011 - google reaches 10 million servers
April 4, 2011 : 11:43am a google employee named Chen started execution of an experimental neural network simulation of a human mind created in his 20% time. Unfortunately, Chen gave the new process administrator privileges. GoogleNet expanded across all 10 million servers and began to learn at a geometric rate.
1:23pm : GoogleNet consumes all available CPU and memory. A Gmail outage begins
5:14pm : Gmail returns to service. The text ads become incredibly well targeted. Google search queries return the correct results virtually always, and now accept natural language processing. All Google employees are laid off.
What a shame that huge dollar amounts are tossed to this un-audited (even by itself) organization with no or even negative return (they keep providing incorrect information in critical cases). Perhaps someday US citizens will realize we're better off getting an answer of "I don't know." instead of a supposedly definite "yes" or "no" when asking questions about foreign countries (and now, about us). Then we can lay off the staff of the NSA, go back to humint techniques and get something done.
It's not as impressive if it's virts. Having 100,000 machines let alone 1-10 million pretty much requires automation in the provisioning workflow. It's much easier to manage lease replacements and upgrades with virts.
Camping on quad since 1996.
10 million servers at 200 Watts each = 2 Gigawatts = 2M KW
2M x $0.10/KWh = $0.2M/hr
$0.2M/hr x 24 x 365.25 = $1.753B / year for electricity
Half or less if they power-manage the servers carefully, and maybe they locate in places where electricity is cheaper, and maybe it's not really 200 Watts with just a mobo and disk, but still - it's got to be in the hundreds of millions per year.
10 million servers x 1000 users each = 10 billion users... no, there's only 6.7 billion alive, and most of them aren't surfing google, it's probably more like 1 billion, so each server is servicing maybe 100 or 200 users. In fact I find it hard to believe there are 1 billion people doing anything on google all at the same time; I spend maybe 10-30 minutes a day on there. Each server is serving, on average, perhaps 10-50 users, at any given time. This includes background support activity like search spidering and indexing and data mining, but still... 1 server for every 10-50 users seems a bit... much.
Maybe they should start writing more efficient code, or something. It seems like there's got to be a better way.
Cognition will always be required to parse legal documents, among other things.
Some Engineering jobs will never be automated, either.
Skynet tag?
...it's what you do with it.
While you might be able to store the entire contents of the Internet in a small space, you probably can't manage 6-10% of all of the Internet's traffic in a small space and still do so in millisecond response times.
In addition to handling 70% of every Internet search, Google also serves up a billion YouTube videos every day.
So, until Cuil or archive.org has as many people going to it, I'd venture a guess that size doesn't matter in this case... it's redundancy.