Google Reveals "Secret" Server Designs

Hugh Pickens writes "Most companies buy servers from the likes of Dell, Hewlett-Packard, IBM or Sun Microsystems, but Google, which has hundreds of thousands of servers and considers running them part of its core expertise, designs and builds its own. For the first time, Google revealed the hardware at the core of its Internet might at a conference this week about data center efficiency. Google's big surprise: each server has its own 12-volt battery to supply power if there's a problem with the main source of electricity. 'This is much cheaper than huge centralized UPS,' says Google server designer Ben Jai. 'Therefore no wasted capacity.' Efficiency is a major financial factor. Large UPSs can reach 92 to 95 percent efficiency, meaning that a large amount of power is squandered. The server-mounted batteries do better, Jai said: 'We were able to measure our actual usage to greater than 99.9 percent efficiency.' Google has patents on the built-in battery design, 'but I think we'd be willing to license them to vendors,' says Urs Hoelzle, Google's vice president of operations. Google has an obsessive focus on energy efficiency. 'Early on, there was an emphasis on the dollar per (search) query,' says Hoelzle. 'We were forced to focus. Revenue per query is very low.'"

Comments

The New Mainframe

[AKAImBatman]

Most people buy computers one at a time, but Google thinks on a very different scale. Jimmy Clidaras revealed that the core of the company's data centers are composed of standard 1AAA shipping containers packed with 1,160 servers each, with many containers in each data center.

Mainstream servers with x86 processors were the only option, he added. "Ten years ago...it was clear the only way to make (search) work as free product was to run on relatively cheap hardware. You can't run it on a mainframe. The margins just don't work out," he said.

I think Google may be selling themselves short. Once you start building standardized data centers in shipping containers with singular hookups between the container and the outside world, you've stopped building individual rack-mounted machines. Instead, you've begun building a much larger machine with thousands of networked components. In effect, Google is building the mainframes of the 21st century. No longer are we talking about dozens of mainboards hooked up via multi-gigabit backplanes. We're talking about complete computing elements wired up via a self-contained, high speed network with a combined computing power that far exceeds anything currently identified as a mainframe.

The industry needs to stop thinking of these systems as portable data centers, and start recognizing them for what they are: Incredibly advanced machines with massive, distributed computing power. And since high-end computing has been headed toward multiprocessing for some time now, the market is ripe for these sorts of solutions. It's not a "cloud". It's the new mainframe.

Re:The New Mainframe

[spiffmastercow]

But wasn't the mainframe just the old cloud? I seem to remember there was a reason we moved away from doing all the processing on the server back in the 80s.. If only I could remember what it was.

Re:The New Mainframe

[AKAImBatman]

I don't know which 80's you lived through, but mainframe processing was alive and well in the 80's I lived through. Minicomputers were a joke back then, and were seen as mostly a way to play video games. (With a smattering of spreadsheet and word processing here and there.) In the 90's, PCs started to take hold. They took over the word processing and spreadsheet functionality of the mainframe helper systems. (Anybody here remember BTOS? No? Damn. I'm getting old.)

Note that this didn't retire the mainframe despite public impressions. It only caused a number of bridge solutions to pop up. It was the rise of the World Wide Web that led to a general shift toward PC server systems over mainframes. All we're doing now is reinventing the mainframe concept in a more modern fashion that supports multimedia and interactivity.

Welcome to Web 2.0. It's not thin-client, it's rich terminal. The mainframe is sitting in a cargo container somewhere far away and we're all communicating with it over a worldwide telecom infrastructure known as the "internet". MULTICS, eat your heart out.

Re:The New Mainframe

[AKAImBatman]

Derr... minicomputers should say microcomputers. My old brain is failing me. Help! Help! Help! He-- wait. What was I screaming for help for again?

Re:The New Mainframe

[es330td]

You forget that fault tolerance is not of utmost importance to Google. I read an article somewhere that said, in essence, that since these are search results, and not financial transactions it is okay if some parts of the overall network don't know everything that every network knows. Having access to 95% (or 99%) of the data is still acceptable in the search world.

Patents & Catch-22

[eldavojohn]

From 2007, the modular data center patent (where the bottommost image of the article comes from). There's no lack of patents revealing piece by piece how their power management setup works.

Ah, the catch--22 of the patent--being forced to reveal your hand in order to protect it while underpaid workers at Baidu figure out how to integrate your ideas into their hardware.

Re:Patents & Catch-22

[dfenstrate]

Ah, the catch--22 of the patent--being forced to reveal your hand in order to protect it while underpaid workers at Baidu figure out how to integrate your ideas into their hardware.

That's not a catch-22, that's the point. In exchange for everyone learning from what you've done, you get society's protection for a limited number of years.

Also, the workers at Baidu are not underpaid- if they where, they'd leave for better oppurtunities. The workers in question have obviously decided they're better off making stuff for google- they don't need your 'superior' judgement to tell them they should go back to subsistenance farming or melting hazardous materials for precious metals in their homes.

A decision to work, or not to work, and to hire, or not to hire, are based on realistic alternatives, not what some westerner sitting at a keyboard 9,000 miles away thinks is best.

Re:Patents & Catch-22

[Anonymous Coward]

Wow, you missed the point. Poster is contending that the patent FAILS to protect IP, BY MAKING AVAILABLE the instructions to REPLICATE said IP.

Yeah, it may work against Yahoo!, but it doesn't save you from companies in China and India, who can undercut you on labor costs, and have a much more rapidly expanding market.

Kidding Me?

[wtbname]

he said. "I worked 14-hour days for two and a half years,"

Get that man a beer.

Stop the lies

[Thanshin]

We all know the searches are actually being done by a large amount of people in suspended animation, being fed the corpses of the previous people.

The thing about each server having its own battery is a cruel joke.

Onboard UPS not new

[Y2K is bogus]

The in-computer onboard UPS is not a new idea. I don't see how they could have gotten any patents on it since I used it have one of these (my day might still). The device I saw had a gel cell mounted on an 8-bit ISA card, full length. It had +5/12v pass through connectors for powering the drives and it powered the computer through the main bus. There was more logic to it, as it had some monitoring capabilities too.

What's next, patenting a hard drive on a plugin board? Been there, it was called the Hard Card and put a 20mb HDD in an 8 bit full length ISA slot, a truly neat idea for upgrading old XT computers back in the day. You could make them work with AT computers too by putting a regular disk controller, without a drive connected, on the bus too and the BIOS would see the XT controller and boot from it.

Re:Onboard UPS not new

[ColdWetDog]

The in-computer onboard UPS is not a new idea

Indeed. (Stares at laptop).

Re:Onboard UPS not new

[geekoid]

A patent is an implementation of an idea.

You can have the idea of how to put an UPS in a computer one way, and I can do it another way and both be valid patents.

I do know this gets abused, and companies try to sue becasue it's there 'idea', but that's ot how it works.

If you find a different way to do a hard drive plugin board, then yes you can patent it. I would advise you only do it if it's better in some way, and there is a demand.

Re:Hey google, want to save some money?

[Bill, Shooter of Bul]

Google claims they did the math and found it was cheaper with commodity hardware. I advise everyone else to do the same and run the calculations for themselves to determine the optimal hardware for their particular load. With out the specifics of their situation, its difficult to criticize in an intelligent fashion, other than a more generalized statement expressing surprise at their configuration.

No shit?

[LordKaT]

When the weather gets warmer, Google notices is that it's harder to keep servers cool.

Brilliant journalistic work there.

Date centre fire risk?

[David Gerard]

Many data centres expressly forbid UPSes or batteries bigger than a CMOS battery in installed systems - because when the fire department hits the Big Red Button, the power is meant to go OFF. IMMEDIATELY.

So while this is a nice idea, applying it outside Google may produce interesting negotiation problems ...

Re:They are computers, no more advanced than befor

[AKAImBatman]

Google is basically re-implementing the efficiency that already exists in a laptop.

You have a laptop with >1000 processors, consisting of several times that many cores, with its own built-in gigabit ethernet running on built-in gigabit switches?

I'd hate to sit next to you on an airplane!