Slashdot Mirror
Intel Embraces Oil Immersion Cooling For Servers
1sockchuck writes "Intel has just concluded a year-long test in which it immersed servers in an oil bath, and has affirmed that the technology is highly efficient and safe for servers. The chipmaker is now working on reference designs, heat sinks and boards that are optimized for immersion cooling. 'We're evaluating how (immersion cooling) can change the way data centers are designed and operated,' said Mike Patterson, senior power and thermal architect at Intel. 'I think it will catch on. It's going to be a slow progression, but it will start in high-performance computing.' Intel's test used technology from Green Revolution Cooling, which says its design eliminates the need for raised flooring, CRAC units or chillers. Other players in immersion cooling include Iceotope and Hardcore (now LiquiCool)."
Is it really a good idea to put computers and hydrocarbons that closely together?
What if there's a fire?
This brings back good memory for the liquid metal CPU cooler that I used a while back
A review is at http://www.guru3d.com/article/danamics-lmx-superleggera-review/
Unfortunately the vendor already closed its doors, or I would have bought more coolers from them
Muchas Gracias, Señor Edward Snowden !
I hope this stuff hits the discount rack soon.
They feared that it could be used to suppress protest or support unpopular rule.
One example of non-flammable oil is Silicone Oil
http://en.wikipedia.org/wiki/Silicone_oil
Muchas Gracias, Señor Edward Snowden !
Cray did this decades ago: http://en.wikipedia.org/wiki/Cray-2
Breathe continuously
I cool my computer with cold beer.
I use a beer cooler!
And when it gets too warm, I serve the beer to Englishmen.
I cool with air you insensitive clod.
What happens when a server has a hardware failure? Take it out of the oil? That must be fun!
...but if you put the server room near the cafeteria, you can make fries too.
Everything old is new again.
Oil works great until you have to remove something...
Seems to me this would add a considerable load to whatever flooring is in place.
I welcome our new UK computing overlords.
Have gnu, will travel.
I was considering something like this a few years ago. But instead I went with conventional air cooling inside an ornately carved wooden case instead.
Note that capillary action inside the cable tends to create oil drips all over the place unless you inject glue/epoxy into all your cables to seal the the tiny gaps between insulators..
http://www.pugetsystems.com/aquarium_computer/V2/module.php
so intel embraces more efficient cooling. Any other good reason they did not do this years ago?
dell 'Copper' arm server:
http://content.dell.com/us/en/enterprise/d/campaigns/project-copper
mineral oil cooled vps host:
http://midasgreentech.com/
From Tom's Hardware 2006 article: http://www.tomshardware.com/reviews/strip-fans,1203.html
And have plenty of carbon for a nutritional diet.
June 30, 2000: Slashdot reports that some overclockers have solved their cooling problem by immersing their motherboard in Fluorinert. Crazy kids. Who knew it would eventually catch on?
"Imaginary solutions to real problems."
Another, but somewhat expensive option is fluorinert. We used to use it for boards that would eventually be potted, but would arc in air. We could use mineral oil (but a dog to clean off the board for potting later) and fluorinert, which was better, but depending on the temp/voltage resistance needed could be helishly expensive
Arrrgh!
Have you ever worked at a company where middle management could not have used daily bunches of fries with extra laxatives?
"There is more worth loving than we have strength to love." - Brian Jay Stanley
I keep my server in my refrigerator.
The first place I ran across the concept was Tom's Hardware, and you can still see the original article. "High Performance Computing" says Intel? Pish Tosh. Kids, you really can try this at home... but get a grown-up to assist you!
http://www.tomshardware.com/reviews/strip-fans,1203.html
http://web.archive.org/web/19991006062047/http://www.accsdata.com/drffreeze/TestBox2.htm
Sadly, all the pictures appear to have been lost.
I remember this guy going through and dunking his systems in Mineral oil over a decade ago, back when I was in 11th grade. You know, back with the BP6 was amazing shit and slotkets were an essential overclocker's tool.
Am I the only one who cringes at the thought of yanking a giant quad-processor card out of a tank of oil, dripping wet, then trying to coordinate the movement over a tupperware container (as to not make a mess on the floor) so I can place it on a desk covered in paper towels for servicing- only to get all my nice ESD safe tools covered in the same gunk, which will require a trip to the bathroom or kitchen to clean up afterwards?
How do you even store one of these boards in a permanent fashion? Wash it down with de-ionized water? Stick it in the dishwasher? I can't imagine you just pop it out, dry it down with some sort of an ESD-friendly towel (assuming things like BGA chips are actually sealed around the edges, so crap doesn't get under them), then chuck it in a plastic container.
I dunno. This seems like one of those marketing things that sounds fantastic on paper, until some dude has a data centre full of 40 racks that are all filled with oil (how heavy would that be?), and has to deal with the issues of swapping out parts on occasion. I bet that same dude isn't going to think oil filled computers were such a great idea. Even IBM isn't crazy enough to do this- their zSeries mainframes have some pretty insane cooling options, but none of them are full PCB-immersed cooling.
-AC
I wonder if Intel acquired any of SGI's patent portfolio. I good friend of mine's brother was a ME at SGI and worked on all the exotic ways to dissipate heat, one of which was submersion in oil and other liquids. Nothing new here. Somebody go check on Cray and Rackspace and see what they're doing.
something that was proven to be fine in 1985?
sure it wasnt oil, it was an exotic chemical developed by 3M but the point still stands
'We're evaluating how (immersion cooling) can change the way data centers are designed and operated,'
its been proven on machines that produced much more waste heat than today 27 years ago in the cray 2
not to mention countless people using oil to cool their high voltage transformers and overclocekd P4's, but yay, GO Intel, grats on the prior art, obious patent in the near future
sigh
you're assuming that they will submerse the entire rack in one vat of oil. if they instead do each unit level in its own vat with lines for circulation, you could shut off that unit, drain it and swap the offending blade. might add 10 minutes to the whole ordeal. the oil can be rinsed off in a parts cleaner like you pick up at harbor freight, using odorless mineral spirits or the like.
I've always wondered - Why aren't cold places full of datacenters? Just pump the air in from outside. Yakutsk (well known to anyone who plays Risk) is a city that gets down to -50C in the winter, or something like that. In the summer it rarely gets above +20C. You'd figure there'd be a booming business building datacenters in these places.
A: Don't put it in.
Help stamp out iliturcy.
Do you want fries with that?
intellectual property law is philosophically incoherent. it is your moral duty to ignore it or sabotage it
Then go somewhere else, dipshit. As I recall from the minutes of our last meeting, the Nerd Mafia aren't forcing anyone to read /. against their will yet.
Floor loading. How much is a rack of oil filled servers going to weigh?
"To those who are overly cautious, everything is impossible. "
Cray used to do this.
http://en.m.wikipedia.org/wiki/Cray-2
inserts in the valves for better heat transfer. That's near water and air and gasoline! Apparently for disposal you're supposed to drill a hole in them and drop them in a bucket of water.
I spent most of the 80's working on flight simulators that had rows of cabinets on raised flooring. One sim was supposed to be at 70F and the temp was usually so stable that if it was up more than a few degrees we could tell by feel and smell as soon as we walked in the room.
By shear luck I worked on simulators in Las Vegas, New Mexico and South Korea, all places that in the summer you really wouldn't want to be working outside. The constant temp during working hours was great ( though I think it made me more of a wimp for temp extremes when I went outside ) Thinking about the oil immersion and what I'd guess would be warmer ambient temps in computer rooms is a little sad. It was the extra cool computer rooms that I worked in that added to the appeal of my job back then.
After the oil has been heated up by all those processors, pipe it to nearby fast food restaurants to cook French fries and all those other delicious, fattening foods! Yum!
And I was using a mineral oil bath (bar frige guts were used to keep the oil cool) to cool my over clocked Pentium. HDD, optical drives and power supply sat on a grate at the top of the coleman cooler and every thing else was submerged. I even did it with distilled water for a bit but it was to hard to keep the water clean.
"hard drives [...] withstood the oil just fine"
I'd like to know if they used off-the-shelf hard drives for this. I find it hard to believe that a hard drive would work in oil. They usually have breathing holes, wouldn't oil get into the drive and interfere with the moving parts?
So, still haven't licked that optical chip problem yet?
I am John Hurt.
Oil immersion works, but why bother?
99.99% of datacenters still rely on moving cool air through hot servers to cool them. in some instances you could make a case for mineral oil bath cooling. if you want to push the envelope of server cooling, try using our Vertically cooled servers. www.cirrascale.com. we believe that hot air naturally wants to rise.
we pack 72 18"tall 1U wide servers in one rack, or 96 13" tall servers. 10's of thousands deployed
we've been able to cool over 30KW of load in one rack with Air on a non raised floor datacenter. We've been doing it for years, and we don't void warranties to do it.
"excuse me, Mr. Dell/HP/IBM service center person... this server isn't working.. can you take a look at it please"
"Be happy to... Um uh, why is it in a plastic ziplock bag?"
"oh that's to keep the mineral oil from dripping on your service lab floor"
RMA declined!, Warranty Null & Void!
mark.skinner@cirrascale.com
GOD I hate this fucking site
You called?
Oh yes, one would think that most geeks would be quite happy to watch shills fuck, as long as they are pretty enough.
Please close the door after you hand in your geek card.
Oil immersion works, but why bother? 99.99% of datacenters still rely on moving cool air through hot servers to cool them. in some instances you could make a case for mineral oil bath cooling. if you want to push the envelope of server cooling, try using our Vertically cooled servers. www.cirrascale.com. we believe that hot air naturally wants to rise. we pack 72 18"tall 1U wide servers in one rack, or 96 13" tall servers. 10's of thousands deployed we've been able to cool over 30KW of load in one rack with Air on a non raised floor datacenter. We've been doing it for years, and we don't void warranties to do it. "excuse me, Mr. Dell/HP/IBM service center person... this server isn't working.. can you take a look at it please" "Be happy to... Um uh, why is it in a plastic ziplock bag?" "oh that's to keep the mineral oil from dripping on your service lab floor" RMA declined!, Warranty Null & Void! mark.skinner@cirrascale.com
Sure you can cool that with air but if you're adding an extra 50-60% in energy costs to cool it and using immersion cooling will only add 4% in energy costs, that'll certainly get people's attention. Add in reduced costs for CRACS and other related equipment and immersion cooling looks more viable. Finally, if your OEM provides equipment that is spec for immersion cooling, I'm pretty sure that they'll provide warranty support for it as well.
"When you sit with a nice girl for two hours, it seems like two minutes. When you sit on a hot stove for two minutes, it
Glancing through the headlines with divided attention, I mentally juxtaposed the headings of two successive stories, yielding "Iran embraces oil immersion for critics".
http://en.wikipedia.org/wiki/Full_immersion_cooling#Liquid_submersion_cooling
> using immersion cooling will only add 4% in energy costs
How so? The servers don't need suddenly less power (except a few percent for the now missing fans) and they still produce heat, which one has ultimately get rid off. The individual server might need less energy, but the data center will not.
I see advantages for overclocking (quicker heat transfer), but fail to see the green aspect. Not to speak of how to get rid of the oil once it is contaminated or the mess, if there would be a leak.
you can pack the chips in higher density with oil, with shorter datapaths.
world was created 5 seconds before this post as it is.
It is greener because
a) you don't have to cool entire rooms, just the servers. So there is much less heat dissipation.
b) heat transfer from metal to liquid and then liquid to metal is more efficient than having a heat transfer from metal to air, and then air to metal.
it is not distilled water you want, it is deionized water: http://en.wikipedia.org/wiki/Purified_water
distilled water is still conductive so either you were very lucky to not fry your computer or you are lying about it.
Well, well, well. Looks like I can get into computing business with this hydraulic engineering degree after all.
http://en.wikipedia.org/wiki/IBM_7302
This was in the days when IBM employees all wore white shirts and ties, and I head stories that the techs would not bother to roll up their sleeves when they worked on these units, because they knew that they would get soaked in oil not matter what they did.
Why is Snark Required?
distilled water is still conductive so either you were very lucky to not fry your computer or you are lying about it.
Distilled water is not meaningfully conductive, but it is still corrosive. Deionized water is not. However, water reacts with itself to form ions, and so DI water rapidly becomes normal water.
The obvious solution is to include a distiller in the loop :)
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
Been immersing chips in oil for years. Part of the reason I'm the shape I am.
Distilled water is not meaningfully conductive, but it is still corrosive. Deionized water is not. However, water reacts with itself to form ions, and so DI water rapidly becomes normal water.
even if parent is getting double-distilled water, i'm not sure i would trust its purity to be enough...
The obvious solution is to include a distiller in the loop :)
or you add a deionization step to the loop instead, since the process is cheaper than distillation : )
The company I work at is currently testing this exact same setup and it is some fairly slick stuff. The cost to retrofit a normal raised floor datacenter to use this would be cost prohibitive though I think.
People have been putting their computers in mineral oil for years, clocking their rigs to numbers that would be impressive even today. Must have been well over ten years since I read about it here on /. for the first time.
/ The Arrow
"How lovely you are. So lovely in my straightjacket..." - Nny
So we have to take our servers to Jiffy Lube every 6 months for an oil and filter change? Will they try to up sell us while we are there? More memory, faster processor, change the electrolytic capacitor electrolite?
or you add a deionization step to the loop instead, since the process is cheaper than distillation : )
Well, you're still going to need a filter, and that means a pump that can overcome it. Perhaps you need both. What do you need for DI, electrodes and a power supply?
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
It's about time dep't:
The Cray-II was immersed in a tank of Fluorinert (type of Freon). This allowed circuit boards to be stacked eight high in a single module:
Six months later Cray had his "eureka" moment. He called the main engineers together for a meeting and presented a new solution to the problem. Instead of making one larger circuit board, each "card" would instead consist of a 3-D stack of eight, connected together in the middle of the boards using pins sticking up from the surface (known as "pogos" or "z-pins"). The cards were packed right on top of each other, so the resulting stack was only about 3 inches high. With this sort of density there was no way any conventional air-cooled system would work; there was too little room for air to flow between the ICs. Instead the system would be immersed in a tank of a new inert liquid from 3M, Fluorinert. The cooling liquid was forced sideways through the modules under pressure, and the flow rate was roughly one inch per second. The heated liquid was cooled using chilled water heat exchangers and returned to the main tank. Work on the new design started in earnest in 1982, several years after the original start date.
It's easier to be a result of the past, but more fun to be a cause of the future! http://www.spacefinancegroup.com/
.. it should be with oil :)
Insert
Distilled water is not meaningfully conductive, but it is still corrosive. Deionized water is not. However, water reacts with itself to form ions, and so DI water rapidly becomes normal water.
Pure water always has ions: H+ and OH-. Deionized water is less pure than distilled water. Only the mineral ions (salts) are removed.
http://powerelectronics.com/thermal_management/thermal_management_products/thin-cavity-cooling-of-electronic-circuitry-0629/
High purity deionized water can be more corrosive than distilled water. I work with some water cooled equipment, that includes some immersed high voltage parts, that works fine with grocery-store quality distilled water and a small deionizing filter. But the filter is on a bypass, as otherwise the resistivity of the water sky-rockets beyond the warnings and limits set by the equipment with risks about damaging metal components. A simple water resistivity meter is used to keep an eye on the resistivity and to keep it within the right range (not too high or too low) by just controlling how much flows through the deionizing filter. Typically very little goes through the filter and it hasn't need replacement in years.
Umm I did fry my computer $3000 down the crapper. I did manage to get it to work for a month that is why I switched to mineral oil. BTW my parents had a distiller so it was easy to get the water.