Google Running 900,000 Servers

1sockchuck writes "How many servers is Google using? The company won't say, but a new report places the number at about 900,000. The estimate is based on data Google shared with researcher Jonathan Koomey, for a new report on data center power use. The data updates a 2007 report to Congress, and includes a surprise: data centers are using less energy than projected, largely due to the impact of the recession (buying fewer servers) and virtualization."

Comments

How about lower wattage CPUs?

[Enry]

We've moved from 1U systems with 90-125W systems to blade enclosures with 60W CPUs and also getting 4 or 6 cores per physical CPU rather than 1 or 2. While our HPC cluster core count has increased by a factor of 4 (allowing researchers to do more work), the amount of energy and floor space required did not increase that much at all.

Re:How about lower wattage CPUs?

[alen]

and ARM is gimped in that it doesn't support a lot of features x86/x64 does that makes it run so fast compared to ARM

biggest advantage of ARM is that the SoC includes the GPU and the RAM. this is going away. x64 is now shipping with GPU on board for most CPUs.

Re:How about lower wattage CPUs?

[Enry]

Hard to say. We were already moving to blade servers when we started the expansion. With the previous chassis servers with 90W CPUs, we'd have to get a rack rated at 30KW rather than the standard 20. With the low power CPUs, we can easily fit in a 20KW rack. Our data center folk (who really know the numbers) started to panic when we had a 20KW rack 1/2 full of 1U systems.

Re:How about lower wattage CPUs?

[Enderandrew]

In addition to this, Google runs DC power supplies, with a low-voltage on board battery as opposed to large rack UPS. I've heard they have some innovative tricks for server room cooling as well, but I've never seen confirmation of exactly what they're doing. But Google goes to great lengths to cut down data center power usage.

Re:How about lower wattage CPUs?

[grimmjeeper]

GPUs provide substantially faster floating point processing than a general purpose CPU. Putting these new Intel/AMD integrated chips in big iron supercomputers will give research teams orders of magnitude more computing power (for less power and money) than the current CPU-only based offerings.

As far as Intel trying to make an ARM chip, that's for an entirely different market.

garbage

[StripedCow]

With the current pace of technology, those machines will be outdated in a few years.

Imagine the pile of garbage that will create...

Re:garbage

[Pieroxy]

Send it to the local PC recycler.

Should we send the 900,000 units in one shipment?

Re:garbage

[Sitrix]

I am quite sure that not all of it is what we call "physical" servers. It's most likely a cluster of beefy hardware running a ton of VM's. As that hardware becomes obsolete, engineers will run less VM's on it and later move it out of main production environment to handle less stressful tasks. It's common now, seeing several servers (48Cores, 512GB ram) running a few hundred virtual servers. So it will take a long time before that hardware will be completely thrown away...

Re:garbage

[swb]

A lot of non-profits won't take donated systems anymore because it's a nuisance to deal with so many antiquated systems. Non-profits need working and reasonably contemporary systems to do their work, a bunch of 256 meg Win 98 systems is really more of an insult than a benefit.

Impressed

[Verdatum]

No comment about it being over 9000 yet. I'm impressed Slashdot.

Mainframes

[instagib]

I wonder how a few hundred mainframes plus storage arrays would fare in terms of TCO.

The Last Question

[roman_mir]

So does it have enough data to answer the last question meaningfully yet?

Re:Shameless plug

[Nidi62]

Seems like they always keep what's in their locations a secret. My father was a manager at a distribution center for a fairly large national electrical supply chain, and several times people would come in to buy things for a complex they were building nearby. Apparently they worked for Google (they were always wearing Google shirts) and they were never allowed to tell them what they were building or what kind of work they were going to be doing.

Re:Real Question

[mat catastrophe]

The real question is how the hell do you manage that many servers? How do you even name them

1hahaha
2hahaha
3hahaha
4hahaha ....
899999hahaha
900000hahaha

900000 servers! Hahahaha!

Re:Real Question

[tibit]

I don't think that's that big of a problem, once you plan for having that many from the get go. All of those servers must be automatically provisioned, and their names are irrelevant and are machine generated. No one ever needs to know those names. Their management software probably manages servers by function. Say they have so many storage nodes, so many storage indexers, so many load balancers, so many static content servers, so many web spiders, etc. The configurations for any particular server must be generated, too, from some sort of a global configuration for their whole "system".

Lower Wattage: Google may be test-driving Tilera

[1sockchuck]

There are reports that Google has been testing servers using low-power many-core servers from Tilera and Quanta. Facebook is also test-driving Tilera chips and seeing promising results when using them on key-value pair apps like memcached. When you have 900,000 servers, you get plenty of attention from processor and server vendors.

Re:900,000 servers...

[Hatta]

What?! 900,000?! There is no way that could be right.

The standard Google server

[Quila]

Dual-processor, two SATA hard drives, 12V PSU, 12V Lithium battery. It's not even sealed in a case, just a frame holding a board, with the PSU, battery and hard drives held on with Velcro.

Most of these will be about that spec.