Slashdot Mirror
Google's Secret Plans For All That Dark Fiber?
beat.net writes "Robert X. Cringely details the plan for all the dark fiber Google has been buying up: "The probable answer lies in one of Google's underground parking garages in Mountain View. There, in a secret area off-limits even to regular GoogleFolk, is a shipping container. But it isn't just any shipping container. This shipping container is a prototype data center. Google hired a pair of very bright industrial designers to figure out how to cram the greatest number of CPUs, the most storage, memory and power support into a 20- or 40-foot box. We're talking about 5000 Opteron processors and 3.5 petabytes of disk storage that can be dropped-off overnight by a tractor-trailer rig. The idea is to plant one of these puppies anywhere Google owns access to fiber, basically turning the entire Internet into a giant processing and storage grid. While Google could put these containers anywhere, it makes the most sense to place them at Internet peering points, of which there are about 300 worldwide.""
I'm sure google has. It's not like you can't have another truck towing a generator following the truck towing the portable datacenter.
I used to work at a datacenter and we had a generator small enough that you could fit 12 of them in a shipping container, and the genny was enough to run a 500 machine datacenter for three days without refueling. The portable datacenter may well have a generator included.
Reinvent the wheel only at either a lower cost, greater effectiveness, or your own personal enrichment and satisfaction.
This is the man who brought us the mathematically impossible 6.5 mile 802.11 link with a passive repeater. The repeater that he never showed to anybody. He also shows us an idealistic world of a community cable and telephone company that nobody's ever seemed to find evidence of.
Saying that, when it comes to technology at least, he is speculative is something of an understatement. Take what he says with an extremely large grain of salt.
> what's the point of putting network latency between all those shipping containers?
To remove the network latency between them and you.
They're not being used "for computing" in the sense you're envisioning. For one thing, 5000 Opterons is enough to tackle pretty much any problem you'd care to throw at it, so there's no need to talk to anyone else. For another thing, they wouldn't be doing big computations, they'd be doing massive numbers of small ones. Think Gmail. 3.5PB is enough to store an awful lot of email, and a few thousand Opterons can run rather a lot of simultaneous HTTP connections from people accessing the mail. Add in a fast network link (for talking to all those many people accessing the mail, and for replicating everything offsite), and you're set.
Cringeley's penchant for sensationalism aside, it's pretty clear that Google's got the expertise and the mindset to deal with problems that start with "if we had 10,000 fast CPUs, 10,000 hard disks, and 10,000 GB of RAM...". Google's rapidly expanding, and has been ever since they started. Back when Google fit in a closet, a new server constituted a big expansion. I'm not surprised that these days their unit of expansion is a tractor trailer with a few dozen racks in it. And if you've got something that packages up that nicely, it only makes sense to pepper the globe with capacity.
High-speed Road Trip (18.000KPH)
A 40 ft shipping container has a surface area of something more like:
(40 * 8) * 4 + (8 * 8) * 2 ==
1408 sq. ft.
which, for 1 megawatt, is more like 710 watts/ sq. ft.
"I assumed blithely that there were no elves out there in the darkness"
Even with liquid cooling it's a hard problem. For one thing you need 5000 little hoses, and beefy pumps to get the water through there at a reasonable speed. Opterons are specced to run up to about 85C (depends slightly on model / family). Suppose you've got incoming water at 10C, and heat it up all the way to 85C. That's 75C difference, or 313.5J/g of water you're taking away. That works out to 5.75 million grams of water per hour, or just under 6000 liters per hour. You can't just dump it into a lake or river or you'll completely nuke the resident ecosystem. It's a manageable number from the point of view of getting it through the machines, but it's still an awful lot of energy to get rid of.
The sort of temperature-differential energy recovery you speak of is technically possible but isn't efficient enough to substantially reduce the cluster's power requirements, and thus its need to vent waste heat.
High-speed Road Trip (18.000KPH)
Taken from http://www.incendiary.ws/node/194
Crmblznski's Limit, sometimes spelled Crizmblski's Limit, has its origins in Keith Laumer's novel "The Great Time Machine Hoax" [1].
The basic theorem is that there is a finite limit to the complexity of any given machine, which specifically precludes the operation of "a machine with sufficiently extensive memory banks, adequately cross-connected and supplied with a vast store of data, [that by its very essence] would be capable of performing prodigious intellectual feats simply by discovering and exploring relationships among apparently unrelated facts." The Limit is an irrational number, much like Pi, in that the total complexity of machine is wholy dependent upon both hardware and software designs.
Promote freedom; fight fascism.