Slashdot Mirror
Liquid Cooling On the Rise As Data Centers Crunch Bigger Data
1sockchuck writes The use of liquid cooling will accelerate in the next five years, according to experts in high performance computing, who cite the data-crunching requirements of scientific research, cloud computing, bitcoin and "big data" analytics. "In the HPC world, everything will move to liquid cooling," said Paul Arts, technical director of Eurotech. But there's still plenty of resistance from data center operators wary of bringing liquid near servers, and cost is also an issue. Liquid cooling can offer significant savings over the life of a project, but the up-front installation cost can be higher than those for air-cooled systems. Immersion cooling has gotten a surprise boost from the rise of bitcoin, including a large bitcoin mine inside a Hong Kong high-rise.
big data here big data there -- too much ppl have to compensate.
OVH has been doing this for more than a decade. They credit it as one of the reasons they're able to undercut competitors by so much, by eliminating most of their cooling costs. They get their power usage efficiency, which is the ratio of IT equipment power consumption versus facility power consumption, under 1.1 for their newer datacentres.
Immersion cooling has gotten a surprise boost from the rise of bitcoin, including a large bitcoin mine inside a Hong Kong high-rise
Nevermind the cost, the immersion liquid that the Hong Kong bitcoin mine is using is actually very ecologically unfriendly
Until someone can come up with immersion liquid that is not so ecologically unfriendly, and with much lower cost, I am afraid it will be a tall order to convince data center operator to cool all their heat generating electronic parts via the immersion method
So we are moving to liquid cooling... You didn't have to do all the name dropping to make your point.
Buzz words I read in this story: "High performance Computing", "data-crunching", "Cloud computing", "bitcoin", "big data", 3x "bitcoin"..
BitCoin mining is certainly NOT a good reason to move to liquid cooling and it is not driving innovation in data center construction and design. Anybody building a data center for a mining operation clearly hasn't done the math and *will* loos money, liquid cooled or not. Cloud computing is really nothing more than the data centers of yore with IP connectivity, it might be driving people to BUILD new data centers, but nothing about the cloud drives you to liquid cooling. Big Data" might be driving this, but it's claim to fame is the ability to use lots of low power processors in parallel, much like RAID uses lots of spindles to spread out the data. Big data is not driving us to liquid cooling. HPC is the same. All this stuff MIGHT be adding to demand for data processing, but nothing about it drives one to liquid cooling over forced air.
The only reason we will be seeing a rise in liquid cooling is if it is CHEAPER than forced air cooling. Cheaper by taking up less space, being more power efficient, more reliable or any other way of reducing data center operator's costs. Until then, all the name dropping you can do with current buzz words won't really help get liquid cooling adopted. It's all about cost.
I guess it would have been a snoozer of an article to read without all the buzz words.
"File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
Frankly I think the future of liquid cooling is the samething used on the desktop just centrally managed pumping and storage. Honestly all these data centers have just been wasting all this heat, and even worse generating more running air conditioners. All that heat has value, pretty high value actually if they were just willing to spend the extra dollars to collect it all. You just set up a industrial park next to your data center, build in some heat transfer systems and offer the waste heat as value add for a bit of $ into whatever medium the customer wants (air, water, etc). In no time at all you will have all sorts of setups that require heat for their industrial use and are happy to pay less than the cost of using gas or electricity and you end up eliminating air conditioning costs and monetizing 20% of electricity use as heat transfer.
Done right with some proper industrial engineering the system could be relatively maintenance free and rather than spending twice to deal with the heat (paying to generate and paying to cool) you only pay once then monetize the asset you've generated. This is one thing the Scandinavians and Germans have always understood, once you make the heat you might as well use it because it's damned foolish just to waste it. They use waste heat all the time for community driven heating and for all sorts of things and it probably ends up saving all kinds of money.
I think such techniques may tend to be highly locale-specific.
For instance, in desert regions, there's no shortage of waste heat, so the idea of trapping and re-using it elsewhere may not make any sense. In those environments, it's probably much more effective to try to make use of abundant solar resources to offset energy costs used. Liquid cooling make make more sense to try to offset the cooling costs here.
For data centers in colder climates, you've essentially got a big helping hand from mother nature in the cooling department, especially in the winter. Any generated waste heat can be gladly reused by someone else to offset normal heating costs they'd otherwise incur. Also, it seems like this heat may only going to be useful in a limited area around the data center, as I'd imagine it would be difficult to transport waste heat over long distances - at least without investing so much in infrastructure that it negates any potential advantage.
I suppose one potential solution is to utilize distributed data center clusters that can be located in building basements, and act as supplementary heating elements, rather than the giant data centers we see today. We've seen some companies moving in this direction, though of course, this would have challenges of it's own.
Irony: Agile development has too much intertia to be abandoned now.
I suspect a liquid future isn't too far off if the price of 3M Novec drops just a little bit more
"A person is smart. People are dumb, panicky dangerous animals and you know it." - K
There is a realization that with cheapest, lousiest cooling a substantial number of servers may kick the bucket, but replacement costs are still lower that energy costs long term. See this paper for example. Liquid cooling doesn't fit the bill for general computing, although it may for very specialized cases like quantum computers that need to be cooled by liquid helium.
95 degrees has no economic value. Anything higher and you start losing components faster.
For instance, in desert regions, there's no shortage of waste heat, so the idea of trapping and re-using it elsewhere may not make any sense.
Not that this is directly relevant but your comment made me think about something interesting; in Arizona in the Phoenix valley during the summer the water coming in underground pipes heats up so much before it even gets to the house that you don't even need a hot water heater. I have family in Arizona and used to live there; Arizona is hot.
Very unlikely. What comes out of a data center is diffuse, low grade heat. Maybe useful for running a dehydrator or drying system, maybe replacing building heating systems... but not much more as the temperature is too low.
It's common in Scandinavia because of the climate - but not many people live in that cold of a climate. (The equivalent latitudes in the America's are way the hell up in Canada.) Geography matters.
but replacement costs are still lower that energy costs long term.
I'm not buying it, my VMWare hosts are pretty large boxes and they've used 630kWhrs since June when they were installed, that comes out to $128/year or so, and that's for primary usage, DX CRAC units have a PUE of ~1.28 which means it costs around $36/year to cool. Even with really cheap servers you'd have to have a LOT of them and have very little effect on AFR to justify it. I'm sure at some scale it makes sense or everyone wouldn't be researching it so hard and doing so many pilot datacenters, but if you don't have thousands and thousands of identical servers (99.999+% of installations) it's just not worth it.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
As someone living in south florida I can tell you that heat is not valuable.
You are sort of right but it is temperature differential that is valuable not just heat. In Alaska or Northern Canada 40c water has some value. In Phoenix it is useless most of the year.
Now if you can get CPUs that are happy at 200c you now have some valuable temperature differential
See my blog http://ilovecookes.blogspot.com/ for light hearted technical information.
You could run a Stirling engine, perhaps using geothermal (instead of the ambient air) to get a more reasonable temperature differential.
Indeed, geography matters: the equivalent latitudes are way the hell up in Canada, but the equivalent climate is much farther south in the United States because Scandinavia benefits from the Gulf Stream. For example, Oslo, Norway doesn't get as cold as Minneapolis, Minnesota. (The average January low is 19.8F in the former vs 7.5F in the latter.)
"[Regarding the 'cloud,'] ownership was what made America different than Russia." -- Woz
everything old is new again... Mainframes for the longest time were water cooled, and a goodly part of any datacentre's back room infrastructure was the whole setup for chilling water and dumping the heat it picked up on the cooling cycle. There was always a nice large water temp gauge in every mainframe frame..
Still not going to get much, even Stirlings need a good bit of log-deltaT to work well enough for a ROI.
Apocalypse Cancelled, Sorry, No Ticket Refunds