Google Running 900,000 Servers

640k by Anonymous Coward · 2011-08-01 03:05 · Score: 0, Funny

640k ought to be enough for anybody.

How about lower wattage CPUs? by Enry · 2011-08-01 03:05 · Score: 4, Informative

We've moved from 1U systems with 90-125W systems to blade enclosures with 60W CPUs and also getting 4 or 6 cores per physical CPU rather than 1 or 2. While our HPC cluster core count has increased by a factor of 4 (allowing researchers to do more work), the amount of energy and floor space required did not increase that much at all.

Re:How about lower wattage CPUs? by Skapare · 2011-08-01 03:09 · Score: 1

ARM for even lower wattage.

--
now we need to go OSS in diesel cars
Re:How about lower wattage CPUs? by elsurexiste · 2011-08-01 03:10 · Score: 1

While our HPC cluster core count has increased by a factor of 4 (allowing researchers to do more work), the amount of energy and floor space required did not increase that much at all.
How much did it increase, then? Just curious about the efficiency...

--
I rarely respond to comments. Also, don't ask for clarifications: a brain and Google are faster, believe me!
Re:How about lower wattage CPUs? by Anonymous Coward · 2011-08-01 03:15 · Score: 0

But longer wait-times for it to finish crunching numbers ;-)
Re:How about lower wattage CPUs? by alen · 2011-08-01 03:15 · Score: 2

and ARM is gimped in that it doesn't support a lot of features x86/x64 does that makes it run so fast compared to ARM
biggest advantage of ARM is that the SoC includes the GPU and the RAM. this is going away. x64 is now shipping with GPU on board for most CPUs.
Re:How about lower wattage CPUs? by Enry · 2011-08-01 03:20 · Score: 2

Hard to say. We were already moving to blade servers when we started the expansion. With the previous chassis servers with 90W CPUs, we'd have to get a rack rated at 30KW rather than the standard 20. With the low power CPUs, we can easily fit in a 20KW rack. Our data center folk (who really know the numbers) started to panic when we had a 20KW rack 1/2 full of 1U systems.
Re:How about lower wattage CPUs? by poetmatt · 2011-08-01 03:25 · Score: 1

uh, I believe we're talking about server situations, not consumers. Having a GPU on an ARM chip or on an X64/x86 chip is a nonsequitur. Or did I miss something here? I fail to see where you come up with this shit considering even Intel is trying to make an ARM chip.. You think they're doing it because supposedly arm doesn't run as well or about having GPU's on the chip? Hint: Intel is shitting their pants over ARM right now.
Re:How about lower wattage CPUs? by Enderandrew · 2011-08-01 03:26 · Score: 3, Informative

In addition to this, Google runs DC power supplies, with a low-voltage on board battery as opposed to large rack UPS. I've heard they have some innovative tricks for server room cooling as well, but I've never seen confirmation of exactly what they're doing. But Google goes to great lengths to cut down data center power usage.

--
http://blindscribblings.com - Tasty pop-culture in conceptual fashion.
Re:How about lower wattage CPUs? by tibit · 2011-08-01 03:35 · Score: 1

They should only worry about enterprise/server market when there's a full featured JVM for ARM. So far, there isn't one.

--
A successful API design takes a mixture of software design and pedagogy.
Re:How about lower wattage CPUs? by grimmjeeper · 2011-08-01 03:47 · Score: 2

GPUs provide substantially faster floating point processing than a general purpose CPU. Putting these new Intel/AMD integrated chips in big iron supercomputers will give research teams orders of magnitude more computing power (for less power and money) than the current CPU-only based offerings.
As far as Intel trying to make an ARM chip, that's for an entirely different market.
Re:How about lower wattage CPUs? by Desler · 2011-08-01 04:45 · Score: 1

You do realize that a lot of supercomputing clusters use GPUs for their number crunching, right? So how exactly is it a non sequitur?
Re:How about lower wattage CPUs? by Anonymous Coward · 2011-08-01 07:55 · Score: 0

Your link is to what originated as an unsubstantiated trade rag filler piece from well over a year ago that had no real backing from Intel, and I believe there was some countering piece even written. Intel wouldn't have to "try" to make ARM chips. They made ARM chips for years and sold the business to Marvell voluntarily. Some people claimed they were realizing their "mistake"... but there is no good evidence that Intel is changing their original strategy.
Notice that well over a year has passed and I have yet to hear of the follow-up to your bullshit link despite this nugget: "Wednesday that servers based on ARM multicore processors should arrive within the next twelve months."
Just like IBM and BlueGene and PPC cores, there will be a niche for ARM in the server arena, but I don't think Intel is shitting anything over it, and you need to find something more exciting than EETimes references from last year.
Re:How about lower wattage CPUs? by poetmatt · 2011-08-01 07:58 · Score: 1

GPU clusters and GPU's embedded into processors are not the same. Desktops might have iGPU's but server processors don't. Servers that do GPU clusters use discreet graphics cards, commonly. Why do people associate the two? Do people not really understand that on an 8 processor chip if you have a single integrated GPU (let alone 1 per core) its performance is still going to be shit in comparison to even a cheap discreet graphics card?
Re:How about lower wattage CPUs? by poetmatt · 2011-08-01 07:59 · Score: 1

putting an integrated graphics chip onto a server cpu is a joke. Even if you put 32 integrated graphics cards onto a single 8 core server cpu, the flops will be shit in comparison to any discreet graphics card which costs orders of magnitude less.
Re:How about lower wattage CPUs? by blacklint · 2011-08-01 08:21 · Score: 1

Curious: in what way is compiling the OpenJDK for ARM not full featured? Not supported by Oracle? Or are you referring to hardware support / Jazelle?
Re:How about lower wattage CPUs? by grimmjeeper · 2011-08-01 08:37 · Score: 1

I should have explained myself better. You're thinking about small blade servers doing simple tasks. Sure, for that the GPU is a complete waste. But that's not what I'm talking about. I'm not talking about a computer that's cranking out killer graphics. I'm talking about computers that do "real work" that is seriously floating point intensive. Far beyond what you'd find running on a small server.
Get that small blade server out of your head. Think big iron. Think supercomputers. Think racks of cards all interconnected to act as one big computer. Think hundreds (or thousands) of individual processors all coordinated to act as one big computer. Now think about putting in a chip that does really fast floating point calculations in place of a general purpose CPU. That's where the GPU becomes useful in the "servers" I'm talking about. If you're a physicist wanting to model a nuclear explosion, you need a lot of floating point calculating power. You need a supercomputer with huge capabilities. Same with the weather service trying to generate weather forecasting. Things like that.
To do this kind of work with older generation supercomputers (i.e. last year), you were talking about running on a computer full of Intel or AMD x86_64 chips. Now, they do an acceptable job for floating point calculations. But now you can put a different chip in there. Each individual chip comes with a GPU that does floating point calculations much faster than the general purpose CPU by a substantial margin. Instead of a thousand CPUs doing your calculations, you can get a thousand GPUs doing far more calculations. That's where the integrated GPU works well in the "server" environment.
Re:How about lower wattage CPUs? by grimmjeeper · 2011-08-01 08:39 · Score: 1

Oh, and I forgot one more thing. The newest integrated CPU/GPU from AMD uses the same processor core that goes into the HD6xxx series chips coming from ATI. So, contrary to what you say, the FLOPS are not ". . . shit in comparison to any discreet [sic] graphics card . . ."
Re:How about lower wattage CPUs? by Skal+Tura · 2011-08-01 08:46 · Score: 1

Single discreet GPU does more work than over 100 dual core & quad core CPUs for the workload i need them for ...
Integrated GPUs still did about 15 CPUs worth of work for that workload ... That is significant, no matter how you put it.

--
Pulsed Media Seedboxes
Re:How about lower wattage CPUs? by ls671 · 2011-08-01 11:29 · Score: 1

Depends how much numbers you have to crunch. I do not know Google architecture well enough to tell, but typically, x86 systems crumble under IO not lack of CPU cycles. Seti or distributed.net use cases are rather seldom.

--
Everything I write is lies, read between the lines.
Re:How about lower wattage CPUs? by tibit · 2011-08-01 15:13 · Score: 1

Yeah. Oracle supports x86 and 64 bit SPARC. The only "news" I could find about OpenJDK for ARM is that support is due to be dropped from IcedTea. So all I know is that whatever "Java" exists for ARM is abandoned at this point. Anyone who knows otherwise, please chime in. Do note that a full featured Java should IMHO have both interpreter and JIT, and perhaps be in somewhat widespread use so that there'll be enough real-life test coverage. I wouldn't use it for any major ARM-based project at this point, unless I had at least one experienced maintainer and a trainee on my payroll...

--
A successful API design takes a mixture of software design and pedagogy.
Re:How about lower wattage CPUs? by poetmatt · 2011-08-01 18:42 · Score: 1

integrated GPU's don't really come anywhere near the flops of a discreet card (I don't get why people call it discreet instead of discrete, I always assumed that was the proper spelling for the term, anyway). Yes, it's better than the processor by itself, but no it is not substantial. GPU clusters already exist, remember the one from china? One single GPU can perform about 20x as much as a single xeon's GPU pretty easily. I understand what's needed for science research and understand supercomputer clusters as well. There's not only performance increases, but cost savings in doing so. The math is not there as far as economy of scale with igpus. You do pay a whole lot more for a whole lot less though. There's also a complete and utter lack of cuda/equivalents for research environments involving integrated gpu's.
Also, you can drop in new GPU's since they essentially rely on PCIE - as long as you have a server board designed for PCIE support. Good luck long-term trying to make that work with processors.
Re:How about lower wattage CPUs? by grimmjeeper · 2011-08-02 00:40 · Score: 1

You really should read up on the new generation of integrated GPUs. They have come a long way in just the last year. AMD has integrated a full up ATI 6xxx GPU on die. Intel is making remarkable strides as well.
Having a discrete graphics card hanging on a PCIe bus may have been 20x faster last year. Integrated graphics were really inadequate when they were in the northbridge. Again, that was last year. Things have evolved significantly since then.
Re:How about lower wattage CPUs? by poetmatt · 2011-08-03 00:57 · Score: 1

Is this a joke? I do read up on this continually, and have a 6970 at home and an E350 netbook. Not commenting on my work. It's not a "6xxx" GPU, it's equivalent to the lowest end of the 6 series gpu. The highest end version is 1/10th that of the flagship graphics card and has only better performance than the lowest end of the flagship design from the last iteration. Also, that hasn't changed *since* at least the last two iterations (we're talking 4x series). I don't know what planet you're on but FLOPS do matter substantially for server compute environments (clusters). Parallelism is far more important than "we have less than a tflop extra per *chip* - not "per core".
Having the latest graphics card on a PCIE is easily 10x faster this year. Intel's strides have been an improvement but are still generally shit. AMD has done a whole lot more. AMD will never allow themselves to cannibalize other markets via making integrated graphics anywhere even remotely near the current performance of discreet cards, for a lot of reasons. Dont' let yourself be mislead with consumer performance vs enterprise/research utility. They aren't even close to the same. Consumer performance also "looks good", but the reality is that the graphics core on the AMD is still easily 2-3x as fast as intel.
This stuff has always improved in performance, but that doesnt' mean it's caught up with the times. The purpose is low power, not performance.

Re:FUCK Google by Lysander7 · 2011-08-01 03:09 · Score: 1, Informative

I believe you dropped your tin foil hat.

Re:FUCK Google by obergfellja · 2011-08-01 03:10 · Score: 0

here is the viagra... you'll need it if you are going to FuCk GoOgLe amount. lol

900,000 servers... by FalafelXXX · 2011-08-01 03:11 · Score: 1

900,000 servers, and they are all data-mining the internet for porn. Awesome.

Re:900,000 servers... by trum4n · 2011-08-01 03:21 · Score: 1

Yet Google Images sucks harder than ever.
Re:900,000 servers... by Anonymous Coward · 2011-08-01 03:22 · Score: 0

That's what the Internet is for, after all.
Re:900,000 servers... by jcwayne · 2011-08-01 03:38 · Score: 1

That's the idea.

--
Failure to follow this advice may result in non-deterministic behavior.
Re:900,000 servers... by Hatta · 2011-08-01 04:27 · Score: 2

What?! 900,000?! There is no way that could be right.

--
Give me Classic Slashdot or give me death!
Re:900,000 servers... by gorzek · 2011-08-01 05:12 · Score: 1

Did you expect it to be...
...OVER 900,000?

--
Check out my world simulator thingy.
Re:900,000 servers... by TheSHAD0W · 2011-08-01 10:53 · Score: 1

RRAH!

garbage by StripedCow · 2011-08-01 03:12 · Score: 2

With the current pace of technology, those machines will be outdated in a few years.

Imagine the pile of garbage that will create...

--
If Pandora's box is destined to be opened, *I* want to be the one to open it.

Re:garbage by jedidiah · 2011-08-01 03:20 · Score: 1

Send it to the local PC recycler.
We have one of those in rural Texas. Surely there are some in the same neck of the woods where Google operates.

--
A Pirate and a Puritan look the same on a balance sheet.
Re:garbage by captainpanic · 2011-08-01 03:23 · Score: 1

I expect quite a high percentage of that to be recycled, actually.
Re:garbage by Anonymous Coward · 2011-08-01 03:26 · Score: 0

With the current pace of technology, those machines will be outdated in a few years.
Imagine the pile of garbage that will create...
One would hope they're upgradable. Throw out a few CPUs, maybe some memory, keep the perfectly good boards, chassis and PSUs.
Re:garbage by Pieroxy · 2011-08-01 03:33 · Score: 2

Send it to the local PC recycler.
Should we send the 900,000 units in one shipment?

--
Write boring code, not shiny code!
Re:garbage by GodfatherofSoul · 2011-08-01 03:33 · Score: 1

No, people like us will buy them for our home networks. Low-end and low-income users will happily use them. Old machines have *a lot* of life, maybe not on the front lines anymore.

--
I swear to God...I swear to God! That is NOT how you treat your human!
Re:garbage by LWATCDR · 2011-08-01 03:34 · Score: 1

Well that will depend on a lot of things. If a socket upgrade is available they can just put in a new CPU. Even if the "server" is taken out of service you are just talking about the mainboard, CPU, and memory. The CPU and memory might be offered for sale as "used" or "refurbished". The board will be recycled, the power supply and rest will probably be reused unless a more efficient solution is available. Servers tend to have a longer life than say cell phones, desktops, and PCs.

--
See my blog http://ilovecookes.blogspot.com/ for light hearted technical information.
Re:garbage by onepoint · 2011-08-01 03:50 · Score: 1

I was able to recycle a dell inspiron 8500 back in 2005. Just died on Friday, 6 years working well, the drive finally crashed. I figured I would fix it and give it to a non profit that might be able to use it.

--
if you see me, smile and say hello.
Re:garbage by Sitrix · 2011-08-01 03:59 · Score: 2

I am quite sure that not all of it is what we call "physical" servers. It's most likely a cluster of beefy hardware running a ton of VM's. As that hardware becomes obsolete, engineers will run less VM's on it and later move it out of main production environment to handle less stressful tasks. It's common now, seeing several servers (48Cores, 512GB ram) running a few hundred virtual servers. So it will take a long time before that hardware will be completely thrown away...
Re:garbage by swb · 2011-08-01 04:03 · Score: 2

A lot of non-profits won't take donated systems anymore because it's a nuisance to deal with so many antiquated systems. Non-profits need working and reasonably contemporary systems to do their work, a bunch of 256 meg Win 98 systems is really more of an insult than a benefit.
Re:garbage by franciscohs · 2011-08-01 04:21 · Score: 1

Not sure were you work, but in all places I've worked hardware was stockpiled once it was put out of service. I'm not so sure about the reason, but there seems to be a lot of accounting issues for a company if it wants to get rid of stuff. Maybe someone more knowledgeable can comment on that.
Re:garbage by BetaDays · 2011-08-01 04:46 · Score: 1

Sure, why not? It's cheaper to send in bulk.

--
Paul: Father... father, the sleeper has awakened! - Dune
Re:garbage by Desler · 2011-08-01 04:48 · Score: 1

Send it to the local PC recycler.
That may not be the good idea you think it is.
Re:garbage by Desler · 2011-08-01 04:51 · Score: 1

Low-end and low-income users will happily use them.
No, they would more likely rather having some recent that actually works well rather than some 12 year old dumpster dived computer that runs as slow as molasses.
Re:garbage by onepoint · 2011-08-01 05:07 · Score: 1

Really? then i guess I'll fix it up for myself and use it as a travel computer. Thank you for advising me.

--
if you see me, smile and say hello.
Re:garbage by Pharmboy · 2011-08-01 05:09 · Score: 1

Depends. Did they replace all 900,000 in one day? No? Didn't think so.

--
Tequila: It's not just for breakfast anymore!
Re:garbage by cpghost · 2011-08-01 06:00 · Score: 1

Send it to the local PC recycler.
It's actually a shame that perfectly working machines are being destroyed this way... while in Asia, Africa etc. people (schools e.g.) would be more than happy to use those machines for 5 to 10 more years, at least.

--
cpghost at Cordula's Web.
Re:garbage by TheRaven64 · 2011-08-01 06:23 · Score: 1

Schools in the USA and Europe would be grateful for them too! Businesses often throw out machines as part of a 3-year rolling upgrade cycle, while schools are stuck with machines 5+ years old because they don't have budget for new ones.

--
I am TheRaven on Soylent News
Re:garbage by Anonymous Coward · 2011-08-01 06:28 · Score: 0

Do you know what a google server looks like ( hint no case, rack mounted device, possibly running on pure DC)? I doubt a google server would be of much use to any consumer. I suppose they could use it for a file server or what not if it took AC, and they built a cover for it.
Re:garbage by the_B0fh · 2011-08-01 06:44 · Score: 1

depreciation. Also it's an "asset" and an "asset" can't just go missing.
Re:garbage by georgesdev · 2011-08-01 07:42 · Score: 1

sure, but what do citizens like you and me do about it? close to nothing I'm afraid. Isn't there about a billion pcs operating world-wide today? what do we do to recycle a few hundred million of them each year?
Re:garbage by Anonymous Coward · 2011-08-01 08:44 · Score: 0

That's fine, just send me those machines, I would be very happy
Re:garbage by Guido+von+Guido · 2011-08-01 10:32 · Score: 1

I am quite sure that not all of it is what we call "physical" servers.
Google has a different model. Quila mentioned what their hardware is like; here is a slightly outdated Wikipedia article describing it. Google would prefer to avoid the overhead associated with running virtual machines. 10-20% overhead may not be a lot for an organization with smaller computing needs, but with Google that would mean adding another 90,000-180,000 servers. Their computing needs are way beyond what any individual server can do anyway.
What they've been doing instead is writing their apps for clusters. When they need more performance, they just add nodes. When a node fails, it's only a minor nuisance as the load is carried by thousands of other nodes.
Re:garbage by bluegreen997 · 2011-08-01 14:25 · Score: 1

Did you ever see the video about how Google disposes of/recycles their hard drives? (Video Google 'google data center')
I'm quite sure they have a protocal for how they recycle all of their hardware.
Re:garbage by DaVince21 · 2011-08-02 00:36 · Score: 1

You say this when there are people using Commodore 64s as web servers.
I think the machines will find a spot somewhere. That, or they're mostly recycled.

--
I am not devoid of humor.

Typo by Anonymous Coward · 2011-08-01 03:12 · Score: 0

data centers are using less energy than projected

Re:Typo by Anonymous Coward · 2011-08-01 03:41 · Score: 0

You think the editors would actually proofread and edit the summary before posting it? Hahahahahahahahahaha.

Impressed by Verdatum · 2011-08-01 03:13 · Score: 3, Funny

No comment about it being over 9000 yet. I'm impressed Slashdot.

Re:Impressed by saihung · 2011-08-01 03:14 · Score: 1

You absolute bastard.
Re:Impressed by Baloroth · 2011-08-01 03:25 · Score: 1

I'm not as impressed with that as I am with the fact that no one ha yet pointed out

data centers are using less energy that projected

. "Data centers that projected what?", you might ask, as I did. Nope, its a typo. Or an omission, since data centers certainly could project... but more likely a type.

--
"None can love freedom heartily, but good men; the rest love not freedom, but license." --John Milton
Re:Impressed by Hooya · 2011-08-01 03:33 · Score: 1

"but more likely a type" what?, you might ask, as I did. Nope, it's a typo. Or an omission, since you could have a type of grammatical error... but more likely a typo.
Re:Impressed by Anonymous Coward · 2011-08-01 03:37 · Score: 0

Please, this is slashdot.

Btw, IT'S OVER 9000x10^2!!!1!eleven1!
Re:Impressed by Anonymous Coward · 2011-08-01 03:38 · Score: 0

... but more likely a type.
I'm not sure if you were expressly being ironic with this, or you just happened to make a typo while complaining about typos and it's normally ironic.
...Or maybe it's just coincidental, like rain on your wedding day-eee-ay.
Re:Impressed by Nidi62 · 2011-08-01 03:48 · Score: 1

Don't worry, eventually there will be at least 9000 comments all with some variation of "But that's over 9000!"
Including this one

--
The only thing necessary for evil to triumph is for it to be pitted against a slightly greater evil

Is that a Googleplex? by Kreylix · 2011-08-01 03:14 · Score: 1

Couldn't resist.

Mainframes by instagib · 2011-08-01 03:15 · Score: 2

I wonder how a few hundred mainframes plus storage arrays would fare in terms of TCO.

Re:Mainframes by Anonymous Coward · 2011-08-01 08:42 · Score: 0

Add the price of the licenses and support, and it may well surpass the price of the 900,000 servers, while leaving less room to equipment failure.
Re:Mainframes by Anonymous Coward · 2011-08-01 08:56 · Score: 0

Someware between a factor of 5 to 10 worse. ie: Really really bad.
The cost per mips is much much much higher on a mainframe. What mainframes do give you is stability, but google solved that problem with distributed software instead.

Shameless plug by ArhcAngel · 2011-08-01 03:18 · Score: 1

OT Google is using all kinds of renewable sources for their energy.
Back OT Do you think they keep all their servers in mobile homes so they can keep the number of servers a secret?

--
"A person is smart. People are dumb, panicky dangerous animals and you know it." - K

Re:Shameless plug by Nidi62 · 2011-08-01 03:33 · Score: 2

Seems like they always keep what's in their locations a secret. My father was a manager at a distribution center for a fairly large national electrical supply chain, and several times people would come in to buy things for a complex they were building nearby. Apparently they worked for Google (they were always wearing Google shirts) and they were never allowed to tell them what they were building or what kind of work they were going to be doing.

--
The only thing necessary for evil to triumph is for it to be pitted against a slightly greater evil

The Last Question by roman_mir · 2011-08-01 03:20 · Score: 2

So does it have enough data to answer the last question meaningfully yet?

--
You can't handle the truth.

Re:The Last Question by Meneth · 2011-08-01 03:24 · Score: 1

No. There is still more data to collect.
Re:The Last Question by Anonymous Coward · 2011-08-01 03:50 · Score: 0

"There is as yet insufficient data for a meaningful answer"
Re:The Last Question by Anonymous Coward · 2011-08-01 04:30 · Score: 0

Imagine having a rack full of 1U servers, each with 5 small diameter, high speed fans pushing air across the CPU/RAM/HDDs (5*42=210). Now, take that times 30 racks per row, and how many rows per room... Since you're standing between two long rows, you've got quite a bit of (semi)white-noise always hitting your eardrums.
At the DC I worked in, it wasn't as dense/full as Google's, but you still had a slight ringing in your ears after being in there for so long. When we'd come out to the office area afterwards, I'd describe it as a static-sound in my ears rather than ringing and everything seemed especially loud/sharp sounding for a while.
Plus, having over-the-ear ear protection will help keep the ears warm! Also a big plus in some datacenters.
Dan
Re:The Last Question by MC68040 · 2011-08-01 06:18 · Score: 1

Well.. the answer is 42, we already knew that :)
Re:The Last Question by TheRaven64 · 2011-08-01 06:30 · Score: 1

For not knowing the difference between the Last Question and the Ultimate Question, your geek card is hereby revoked.

--
I am TheRaven on Soylent News
Re:The Last Question by AdmiralXyz · 2011-08-01 07:41 · Score: 1

Yes

--
Dislike the Electoral College? Lobby your state to join the National Popular Vote Interstate Compact.
Re:The Last Question by roman_mir · 2011-08-01 07:55 · Score: 1

I agree with this guy. You are done.

--
You can't handle the truth.
Re:The Last Question by tweakerbee · 2011-08-01 11:39 · Score: 1

I'm sure they'll eventually get back from the "Lights out" principle.
Re:The Last Question by Anonymous Coward · 2011-08-02 00:08 · Score: 0

You mean if it can figure out the question to the answer yet?
Re:The Last Question by roman_mir · 2011-08-02 00:20 · Score: 1

no, you fail. The question is possibility of reversal of entropy.

--
You can't handle the truth.

Let me fix that for you by bigredradio · 2011-08-01 03:21 · Score: 1

With the current pace of technology, those machines will be outdated in a few months .

Imagine the pile of garbage that will create...

--
Flexible bare-metal recovery for Linux/UNIX

Re:Let me fix that for you by jdgeorge · 2011-08-01 04:03 · Score: 1

Maybe more to the point, they'll be unable to fulfill their mission for Google within a few years. They'll be good enough for a while, though.
Re:Let me fix that for you by abarrow · 2011-08-01 06:35 · Score: 1

So, why is that, exactly? Google has proven by it's actions that their solution to the need for more processor power is just to add more servers. Granted, it'd be tough to pull one of the old blades and play the most recent edition of Duke Nukem, but really, these systems will still be able to crunch numbers for a very long time. Would buying faster systems with more cores would allow a single system to crunch more? Sure, but really, those old systems can still happily serve their original purpose. As Google needs more systems, they buy the latest and fastest, but the old systems don't actually have to be replaced until they actually break.
Re:Let me fix that for you by bingoUV · 2011-08-02 04:12 · Score: 1

Not exactly. When cost of not replacing hardware (in increased power consumption, rack space, cooling requirement etc) exceeds the cost of replacing hardware ; they replace. Even when the trashed hardware doesn't "actually break".

--
Bingo Dictionary - Pragmatist, n. A myopic idealist.

Real Question by Anonymous Coward · 2011-08-01 03:22 · Score: 0

The real question is how the hell do you manage that many servers? How do you even name them, let alone manage their allocations for IP addressing, maintenance, kernel patches, etc.? They must have some seriously complex management software that allows them various level views into this ginormous server farm in order to deal with it. Once you have that base level of managing the servers themselves, how do you allocate your workloads across that many machines? Shit, no wonder they always introduce a new feature and say things like, "this will be rolling out to all users over the next week". They don't know where the hell they need to replicate the new bits to and how long it will take.

Re:Real Question by Anonymous Coward · 2011-08-01 03:28 · Score: 0

That just might be the reason why they have a small army working in Operations and are always hiring more people...
Re:Real Question by Enderandrew · 2011-08-01 03:29 · Score: 1

I'm guessing only front-facing web servers get constantly regular security patches. The rest might not get rebooted or patched at all if they replace the servers frequently enough (2-3 years). We are talking Linux servers here.
With that many servers, I'd tie the naming scheme to rack location. IP addressing would go in order along those racks.

--
http://blindscribblings.com - Tasty pop-culture in conceptual fashion.
Re:Real Question by instagib · 2011-08-01 03:29 · Score: 1

Bash scripts.
Re:Real Question by mat+catastrophe · 2011-08-01 03:39 · Score: 4, Funny

The real question is how the hell do you manage that many servers? How do you even name them
1hahaha
2hahaha
3hahaha
4hahaha ....
899999hahaha
900000hahaha
900000 servers! Hahahaha!

--
sig not found
Re:Real Question by tibit · 2011-08-01 03:40 · Score: 3, Interesting

I don't think that's that big of a problem, once you plan for having that many from the get go. All of those servers must be automatically provisioned, and their names are irrelevant and are machine generated. No one ever needs to know those names. Their management software probably manages servers by function. Say they have so many storage nodes, so many storage indexers, so many load balancers, so many static content servers, so many web spiders, etc. The configurations for any particular server must be generated, too, from some sort of a global configuration for their whole "system".

--
A successful API design takes a mixture of software design and pedagogy.
Re:Real Question by alen · 2011-08-01 03:46 · Score: 1

the applications do all the work and everything is redundant
i've read years ago that if a server at google goes down it may take a month for the data center ops people to get around to replacing it
Re:Real Question by onepoint · 2011-08-01 03:53 · Score: 0

some how I am thinking of the Muppet's and that guy the "count"

--
if you see me, smile and say hello.
Re:Real Question by mat+catastrophe · 2011-08-01 03:59 · Score: 1

Well, you should be.

--
sig not found
Re:Real Question by DMoylan · 2011-08-01 10:52 · Score: 1

one
two
three
many ...
lots
yeah i'm trollin' :-)
Re:Real Question by Anonymous Coward · 2011-08-01 12:32 · Score: 0

It's a shame that your question didn't get any serious answers, because it should've lead to a lot of interesting responses. Just goes to show, once again, that Slashdot is filled with a bunch of posturing idiots who have no clue about life in the real world.
For you onanists who think the sun shines out your ass because you can write a few scripts, managing 10 servers is very different from managing 300 servers, which is very different from managing thousands of servers.
Screw Slashdot, what a boring poseur site this has become.
Re:Real Question by Anonymous Coward · 2011-08-01 19:27 · Score: 0

More interesting is how they were able to get that damned owl to count that high...
Kid: Mr. Owl, How many servers does Google have?
Mr. Owl: Lets find out...1... 2... 3... CHRUNCH!!!! 3.
Re:Real Question by Anonymous Coward · 2011-08-01 21:22 · Score: 0

WHOOSH!

And, how many do the Feds have logins on? by GodfatherofSoul · 2011-08-01 03:30 · Score: 1

n/t

--
I swear to God...I swear to God! That is NOT how you treat your human!

and soon there will be by Anonymous Coward · 2011-08-01 03:36 · Score: 0

over 900,000!

Lower Wattage: Google may be test-driving Tilera by 1sockchuck · 2011-08-01 03:44 · Score: 2

There are reports that Google has been testing servers using low-power many-core servers from Tilera and Quanta. Facebook is also test-driving Tilera chips and seeing promising results when using them on key-value pair apps like memcached. When you have 900,000 servers, you get plenty of attention from processor and server vendors.

Re:Lower Wattage: Google may be test-driving Tiler by Enry · 2011-08-01 03:56 · Score: 1

Not all 900k servers are being used for memcached. You will need higher speed CPUs for crawling, operating all the back-end Google services, transcription, etc.

Noisy Room? by seven+of+five · 2011-08-01 03:57 · Score: 1

Noticed that the gentleman in the picture is wearing fullsize earphones or ear protection. Is the room that noisy, or is he just enjoying some tunes?

Re:Noisy Room? by Bengie · 2011-08-01 04:55 · Score: 1

Google had a YouTube video of their security practices. They do actually have hearing protection for the server rooms.

And in their downtime... by Anonymous Coward · 2011-08-01 04:04 · Score: 0

I hear that Google is totally mining Bitcoins.

Virtualization saves energy? by timeOday · 2011-08-01 04:06 · Score: 1

Virtualization is very inefficient compared to simply running multiple server processes on a single box, because each VM allocates resources to an instance of the OS, and RAM is more-or-less statically allocated beetween them. This makes sense when running several different services that each require a different operating environment, or to enforce complete user separation, e.g. a hosting service. But I would imagine google is running tens of thousands of identical servers running the same server daemon, so why would Virtualization make sense and save energy there?

Re:Virtualization saves energy? by CBravo · 2011-08-01 05:51 · Score: 1

Young padawan, there are many sorts of virtualization.

--
nosig today
Re:Virtualization saves energy? by dmpot · 2011-08-01 05:59 · Score: 1

I would imagine google is running tens of thousands of identical servers running the same server daemon, so why would Virtualization make sense and save energy there?
Who said that Google uses virtualization to run identical servers?
Just running "git log --grep=virtualization" on the Linux kernel, you can see that Google does not contributed much to virtualization in the Linux lernel, in sharp contrast to other part of the kernel such as ext4.
Re:Virtualization saves energy? by Anonymous Coward · 2011-08-01 06:26 · Score: 0

They're probably running a custom, purpose built, abstraction layer that provides the advantages of virtualization without the drawbacks you think of when you think of virtualization. I promise they're isolating the crap out of their daemon processes so that they can roll them out in stages without affecting other services, and they're able to pop the services up on another server if a datacenter goes offline unexpectedly.

Wolfram by Anonymous Coward · 2011-08-01 05:02 · Score: 0

Maybe Wolfram has more servers? They seem to be working on the problem anyway... Can entropy be reversed? I asked google, but they just directed me to some dumb sci/fi story.

Re:FUCK Google by thetoadwarrior · 2011-08-01 05:24 · Score: 0

You'll need to try harder, Ballmer.

The standard Google server by Quila · 2011-08-01 05:46 · Score: 3, Informative

Dual-processor, two SATA hard drives, 12V PSU, 12V Lithium battery. It's not even sealed in a case, just a frame holding a board, with the PSU, battery and hard drives held on with Velcro.

Most of these will be about that spec.

Can you imagine... by Anonymous Coward · 2011-08-01 05:52 · Score: 0

A beowulf cluster of those?

Can you imagine a by future+assassin · 2011-08-01 06:09 · Score: 1

beowulf cluster of these?

--
by TheSpoom (715771) Uncaring Linux user here. I have nothing to add to this but please continue. *munches popcorn*

Re:Lower Wattage: Google may be test-driving Tiler by JamesP · 2011-08-01 06:49 · Score: 1

I'm guessing for IO bound servers (that is, all those that take care of storage) the use of a fast CPU is a waste (unless they're also running MapReduce)

Since most modern CPUs can 'go around the world' while the HD is fetching data, kind of makes sense.

Of course, the cost/benefit analysis is not only this.

--
how long until /. fixes commenting on Chrome?

Wasteful by Anonymous Coward · 2011-08-01 08:02 · Score: 0

640K should be enough for anybody

1 000 000 computers... by Bost · 2011-08-01 08:50 · Score: 1

... that's a square 1000 x 1000 meters. Now place 5 "normal" computers next to each other in two layers and you need about 100 000 square meters. Divide 100 000 by 8 data centers (Atlanta, North/South Carolinas, Chicago, California, Oregon, Taiwan, Ireland) = 12 500 square meters per data center. If every data center has 2 floors than you need a building like 80 x 80 x 5 meters. And you still have enough place for the guys with wheelbarrows :) Anyway a data center like this would be about a size of an industry bakery for a slightly bigger town - 100 000 inhabitants. About the same size, about the same power consumption. Nothing spectacular.

Re:FUCK Google by treeves · 2011-08-01 12:55 · Score: 1

That's googol, not Google.

--
...the future crusty old bastards are already drinking the Kool-Aid.

Slashdot Mirror

Google Running 900,000 Servers

127 comments