Slashdot Mirror
A Closer Look At Immersion Cooling For the Data Center
1sockchuck writes "Want to save money on data center cooling? Tip your racks on their side, fill them with mineral oil, and submerge your servers. Austin startup Green Revoluton Cooling first profiled here) has a video demo of its immersion cooling solution, which it says can handle racks using up to 100kW of power. A photo gallery on the company web site shows some early installations."
Well, on the other hand, if they're supposed to be air-tight, I guess they're baby oil-tight, too.
But there's got to be something or another that doesn't react well with mineral oil, right?
I guess this means they save on fans, and the power to run fans. That's additional power and heat savings right there.
OK, I've got it: what about the CD/DVD drives? Or is it all network IPL in data centers? I'm racking my brains trying to think of something this would mess up.
I'm not a lawyer, but I play one on the Internet. Blog
Sorry, the link is supposed to be goatse, but they took down the blog I used for this :-(
Next time folks, today isn't my day
Anyway, the post is actually correct, I did read such an article sometime ago.
Don't remember the link unfortunelly
1999 Have I been reading Slashdot that long?
I'm sorry if I haven't offended anyone
The setup looks nice as it is, but having 42" racks laying on their backs never gives the same rack density than the same racks standing upright.
and I still get to go home smelling like fries!
I imagine that some sort of pneumatic/hydraulic piston system(like the one used on many vertical-open car doors and the like) would be well within the realm of the possible, and offer a zero-apparent-weight slide-up ; but I can't see whether or not there is anything of the sort from the pictures provided.
The one picture showing "easy serviceability" shows the operator having completely unracked a relatively small server. That doesn't scream 'easy' to me. Pull server up, hold with one hand, unclip a bunch of fiddly cables with the other(don't lose them in the oil pit, or you'll have to give the temp a snorkel...) lay it flat, pop the top? Aw hell no. The whole point of slide rails is that you can get to the guts without having to disconnect any cables(unless connected to the particular bit you are swapping, of course) or unrack anything.
All blades would be OK, assuming you never need to get at the back of the blade chassis; but I pity the fool who has to disconnect and reconnect(and be sure to get them back in the right places... those 8 are on different VLANS!) a nasty mess of ethernet cables just to swap a PSU or something.
I would expect oil to be far more efficient than air. It has a hugely greater thermal capacity (hundreds of times), so it can extract much more heat from the chips and similarly hand it over better to the cooling vanes. You use thermal paste to connect the chip to heat sinks better than air - this is a larger scale version of the same thing, where the whole system is immersed in a sort of thermal paste.
Consciousness is an illusion caused by an excess of self consciousness.
There's a good play-by-play report of a hobbyist's adventures in mineral oil cooling here. The first page is just an introduction, but contains links to all the juicy bits on successive pages.
Sorry, no goatse.
When I experimented with mineral oil based cooling, the main issue I had were water droplets condensing on the surface of the cold mineral oil and then promptly sinking... towards the motherboard sitting in the bottom of the old aquarium I was using as a case. Of course there were solutions to this problem but it was a quick and dirty (you can take that dirty word quite literally) test back when I was a student, so we gave up on the idea pretty quickly.
I wonder how they have managed to solved the condensation problem.
> I don't fancy the messy job of making hardware changes though.
THIS. If it's made by man, it will eventually fail and will require service or replacement.
The cost in labor (and cleanup!!!) (and replacement oil!) (and trips to the emergency room for employees who slipped and fell in the oil on the floor!) (not to mention the lawsuits) make this a supremely dumb idea. Now add in the cost of the hermetically-sealed rack(s) and it would be difficult to imagine a dumber idea.
Google is pretty innovative about stuff like this. They use their own in-house version of Linux on commodity hardware, thousands upon thousands of PCs in each data center. But they still use air cooling and air conditioning because, at the end of the day, it's the best bang for the buck. You would save more money by using more efficient equipment, I think -- as you replace each unit, just try to find one that's a little "greener." Or, consolidate servers and use virtualization. Be creative. But DON'T seal the danged thing in a rack filled with oil!
Now, if someone would design a liquid-cooled rack mount computer with coolant connections, you might could make an argument for that. Run hoses and put the radiators and fans on the side of the building in the shade, maybe. But I still don't see how it would be cost-effective, and the square footage required for the hoses and radiators (or whatever else you plan to use) will hardly improve the expense.
Not that I'm opposed to liquid cooling, per se. Continental makes a liquid-cooled 30KW FM tube transmitter and it's nice and quiet. My big transmitters still use big tubes and require huge blowers. They're so loud you can't use the phone in the building, and if I'm doing PM and have both the main and auxiliary transmitters on at the same time, it's so loud you can hardly think. As the prices on high-powered solid-state transmitters continue to drop, we're replacing the older tube units. Now THAT'S cost-effective.
Just my opinion, but if the boss ever told me to fill our racks with mineral oil, I'd take him out to dinner and have a LONG talk with him. (Not that he'd ever propose something so foolish. We have enough trouble with leakage on our hermetically-sealed, oil-filled dummy loads.)
Cogito, igitur comedam pizza.
Having worked in a fair number of server rooms, I'd say that the freequency of needing to service equipment has been dropping significantly over the lat 15 years. These days, it's almost a non-issue. I don't think I've pulled a single server for anything but replacement in the last 4 years.
Transfering heat to fluids is significantly more efficient, both on the recieving site (in the server) and the giving side (in the cooling tower). It requires less energy to transfer heat from components to the water (ie: no fans or heat sinks). And it requires less energy to transfer heat from the water in the cooling tower (ie: much smaller chiller/AC unit). So it is more efficient. Acording to the article, their solution consums 50% less energy than the traditional air conditioning and fans.
-Rick
"Most people in the U.S. wouldn't know they live in a tyrannical state if it walked up and grabbed their junk." - MyFirs
I once drank half a bottle of mineral oil, and let me tell you, leakage was definitely a concern.
You are welcome on my lawn.
Yes, fans are removed. RTFA.
You're right, it takes a lot of energy to heat up that much oil, that's the point. It doesn't take much energy to heat up a cubic foot of air, it takes a lot more to heat up a cubic foot of oil, the effort here is to remove as much of the energy that would go into melting chips as is possible.
Liquid to air heat exchangers can be made as big as you need pretty cheaply, it's easier to pump 800 cubic feet of air through a big radiator cooling one cubic foot of oil than it is to pump 800 cubic feet of air through an OEM heat-sink.
Cray-2 used Fluorinert. In 1985. Related jokes and memes abounded until... dunno. Certainly they were still part of HPC culture when I started my career in 1994.
"... and more and more now there are all kinds of electronic goodies available" -- Pink Floyd 1972
Additionally, it has the nasty tendency to dissolve some plastics over time.
From what I understand, this has been the main problem with immersion cooling. Mineral oil softens PCBs.
"When information is power, privacy is freedom" - Jah-Wren Ryel
A mineral oil or liquid petroleum is a liquid by-product of the distillation of petroleum to produce gasoline and other petroleum based products from crude oil. And it isn't exactly non-toxic nor non-flammable (see link below). Not to mention all the heavy metals still found in many servers. Inevitably some of those metals will be picked up by the circulating oil so disposal might become an issue as well. Don't get me wrong, I like new ideas that save energy but touting it is "totally green" is skipping a few steps.
http://www.jtbaker.com/msds/englishhtml/m7700.htm
Oil will have a far higher heat capacity than air, so the heating rate after pump failure would be far slower than after an air handler failure.
Excellent brainstorming ... Farmville servers will replace nuclear plants for the purpose of boiling water to spin turbines. Ingenious. (Saying this half sarcastically, yet some marketroid will actually run with it.)
I'm not a lawyer, but I play one on the Internet. Blog
Those figures don't give the full picture.
1 kg of air takes up a lot more space than 1 kilogram of oil.
So the heat capacity of oil in a server is far more than twice the heat capacity of air in a server.