Slashdot Mirror


Immersion Cooling Drives Server Power Densities To Insane New Heights (datacenterfrontier.com)

1sockchuck writes: By immersing IT equipment in liquid coolant, a new data center is reaching extreme power densities of 250 kW per enclosure. At 40 megawatts, the data center is also taking immersion cooling to an entirely new scale, building on a much smaller proof-of-concept from a Hong Kong skyscraper. The facility is being built by Bitcoin specialist BitFury and reflects how the harsh economics of industrial mining have prompted cryptocurrency firms to focus on data center design to cut costs and boost power. But this type of radical energy efficiency may soon be key to America's effort to build an exascale computer and the increasingly extreme data-crunching requirements for cloud and analytics.

19 of 80 comments (clear)

  1. The eighties called... by Anonymous Coward · · Score: 2, Informative

    ...they want their cooling back: https://en.wikipedia.org/wiki/Cray-2

    1. Re:The eighties called... by fahrbot-bot · · Score: 4, Interesting

      ...they want their cooling back: Cray-2

      I was actually one of the admins for a Cray-2 (and other systems) at NASA LaRC from 1988-1992. It was pretty cool (no pun intended). The chassis was Plexiglas (or something else clear) and you could see the 3D circuit boards immersed in the Fluorinert - which was wicked expensive back then. I always wanted to put some plastic fish inside the system... The system was moved to the Virginia Air and Space Center (VASC) for a while after being decommissioned sometime later.

      --
      It must have been something you assimilated. . . .
  2. Really? by argStyopa · · Score: 5, Funny

    "INSANE" new heights?

    Has Slashdot been sold to the Gawker network now?

    --
    -Styopa
    1. Re:Really? by thegarbz · · Score: 4, Funny

      No don't understand. This is revolutionary. The article has such amazing facts such as:
       

      The Novec liquid inside a BitFury cooling enclosure actively boils as it changes phase, removing heat from bitcoin mining hardware.

      How have we not had stuff that ACTIVELY boils as it changes phase before! It's INSANE!

  3. Despite the summary, this is somewhat new... by jeffb+(2.718) · · Score: 5, Informative

    The difference with this approach is two-phase cooling, where they're actually boiling the heat transfer fluid. That can remove heat a lot more quickly, as long as you can keep a few issues under control:

    1) Getting a working fluid with an appropriate boiling point and otherwise acceptable physical parameters (non-flammable, doesn't dissolve your circuitry, etc). 3M has already stepped up to the plate on that.

    2) Recondensing the vapor fast enough. This is a lot easier than cooling the circuits directly.

    3) Preventing the hot chips from forming a vapor barrier, which insulates the chips from the coolant. The Leidenfrost effect is an example of this, but you can lose efficiency long before you reach the droplets-skittering-around level, especially if there are lots of nooks and crannies where bubbles can get stuck. Presumably the designers have handled this as well.

    If they go with a transparent enclosure and some gratuitous lighting, this could become the new mad-scientist/Big Scary Computer visual trope. Let's face it, lab coats, blinking lights and reel-to-reel tape drives are really tired...

    1. Re:Despite the summary, this is somewhat new... by bobbied · · Score: 3, Interesting

      Two phase cooling eh? I don't know if that's a good idea.

      I would contend that it's usually not a good idea to cool something using a liquid when you let that liquid boil when in contact with what you wish to cool. This is especially true for things you wish to keep evenly cool. I understand that you do gain a lot of heat transfer capability by vaporizing the liquid, but you loose the ability to easily keep heat evenly flowing from a surface when you let vapor bubbles form on it. Perhaps you could deal with that issue using conductive materials to spread the heat out (you are going to need some of that anyway) but it might be cheaper to implement a single phase solution. Also, presumably they are suggesting a "closed loop" system for this liquid, where the vapor would need to be recycled by compressing and condensing it back into liquid. This puts the ambient temperature as the lowest you can get the liquid, without some other multi-phase process (and associated expense).

      I would think that it would be better to stay a liquid at all times and pump the liquid though a heat exchanger to be cooled using conventional refrigeration methods. You avoid vapor bubbles causing hot spots, only need to come up a suitable liquid based on it's non-reactive nature that will stay liquid and not have to worry about it having the necessary phase change pressure/temperature for your application. Plus, water chillers are already standard fare at current data centers and in industrial cooling equipment. Just pump liquid though the whole thing and push the thermodynamically expensive processes that involve phase changes off onto existing efficient equipment designs which exist. In short, avoid inventing the wheel...

      --
      "File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
    2. Re:Despite the summary, this is somewhat new... by jeffb+(2.718) · · Score: 3, Informative

      I would think that it would be better to stay a liquid at all times and pump the liquid though a heat exchanger to be cooled using conventional refrigeration methods.

      The thing is, you typically move immensely more heat via phase changes than by simply raising and lowering a liquid's temperature. For water, heating one mole (about 18g, or 18 ml, or 1.2 tablespoons) of liquid from the freezing point to the boiling point takes about 7.5 kJ; converting that same amount of water from liquid at the boiling point to gas at the boiling point takes over 40 kJ. (Standard pressure, etc, etc.)

      That confers a huge advantage in two-phase systems. Yes, you have to deal with bubbles and vapor barriers, but you also get free vigorous agitation, reducing the risks of boundary layers and poor mixing that complicate all-liquid systems.

  4. Re:Voice from that hot and wet hole. by The-Ixian · · Score: 2, Insightful

    The inherent value from some mineral from the ground is exactly zero too.

    It is all about scarcity and the value we place on things.

    --
    My eyes reflect the stars and a smile lights up my face.
  5. Re:The sad part, evil pays the highest rent. by Pinky's+Brain · · Score: 2, Informative

    Which sucker trying to front run suckers can I front run this nanosecond.

  6. Re:The sad part, evil pays the highest rent. by ShanghaiBill · · Score: 4, Informative

    Better hurricane models (the current prediction technology had the worst hurricane on record cause -zero- deaths in Mexico.)

    That was caused by sensational journalism rather than bad models. Although Patricia was a record storm out at sea, models showed that it would lose energy as it passed over cooler waters close to the coast. The models correctly predicted that the winds would drop from 200mph to about 165 by the time it came ashore.

    The next big use for large machines is HFT. A microsecond or two on a fast pipe can mean millions in stock gains.

    The glory days of HFT are in the past. Speed is no longer an advantage when everyone is doing it. Besides, HFT needs fast pipes, but doesn't really need a lot of computation.

  7. Re:Destroying our world by theIsovist · · Score: 3, Interesting

    I would think his comment goes to the idea of bitcoin mining, which to my knowledge, serves no other purpose than to prop up bitcoin itself. It's a massive waste of energy on something completely intangible. I'd like to see a more in-depth study, but some estimate that a single bitcoin transaction could power a house for a day and a half (http://motherboard.vice.com/read/bitcoin-is-unsustainable). What a fucking waste. At least folding@home used energy for scholarly purposes.

  8. Re:Voice from that hot and wet hole. by gman003 · · Score: 2

    Most minerals have intrinsic utility. Gold is useful as both a conductor and as a reflective coating. Silver is useful in various chemical compounds. Platinum and iridium are very useful catalysts.

    A better comparison would have been to fiat currency, which is equally useless except as a means of economic exchange.

  9. Re:Higher performance assumes higher energy use by bobbied · · Score: 2

    It's not really a law, but more of a set of guidelines... Moore or less.... He said so himself.

    --
    "File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
  10. Re:Voice from that hot and wet hole. by ShanghaiBill · · Score: 2, Insightful

    The inherent value from some mineral from the ground is exactly zero too.

    Gold mining is a filthy business. It causes extensive erosion, silted creeks and rivers, and mercury poisoning. Bitcoin mining is far better for the environment, and most of the electricity used comes from clean and cheap hydro-power.

  11. Re:Voice from that hot and wet hole. by thegarbz · · Score: 2

    You can make bitcoin as scarce as you want. Until I can cut ceramic with it I think diamonds are more valuable.

    Now you are right in case of pretty things for prettyness, but minerals have properties that make them useful for a lot of different applications. For instance gold is a noble element and thus has great corrosion resistance. Platinum while great for making nice looking rings is also a great catalyst for chemical reactions. While bitcoin's scarcity and value is based on what someone thinks about it, the scarcity and value of minerals is anchored in human's continuous demand for them.

  12. Re:Voice from that hot and wet hole. by fustakrakich · · Score: 2

    Gold is useful as both a conductor and as a reflective coating. Silver is useful in various chemical compounds. Platinum and iridium are very useful catalysts.

    Those things are only useful for people who need/want them. There is no intrinsic value in anything.

    --
    “He’s not deformed, he’s just drunk!”
  13. Re:Higher performance assumes higher energy use by Arnold+Reinhold · · Score: 2

    An interesting architecture might combine a processing unit with a solar panel on a simple stand. Large numbers of these units could could then be placed in a desert area, communicating via a mesh network, with almost no additional infrastructure, not even roads. Coining cryptocurrency and code breaking do not require much interprocessor communication. Energy costs would be zero. There would be no need to locate near a power transmission grid, so the most desolate desert areas might be suitable, minimizing land costs. Robotic dune buggies or drones could be used to dust off the panels periodically. and remove units as needed for repair or hardware upgrade. If batteries get cheap enough, they could be incorporated to allow processing to continue at night, perhaps at a faster clock due to easier cooling. Human staffing might be limited to security and backup maintenance.

  14. Re:Destroying our world by swillden · · Score: 2

    The whole system results in a massive waste of energy for an inefficient currency.

    Cite? Transactions with traditional currencies aren't free, you know. The bookkeeping, auditing, forensics and prosecutions needed to keep normal transactions sufficiently secure do have a significant real-world cost. Not so much in energy, I'd expect, but much more in person-hours. How does bitcoin compare, all costs considered? I don't know. If you know of some research that tries to answer this question with data, I'd very much like to see it.

    --
    Note to ACs: I usually delete AC replies without reading them. If you want to talk to me, log in.
  15. Re:Voice from that hot and wet hole. by Kjella · · Score: 2

    Those things are only useful for people who need/want them. There is no intrinsic value in anything.

    That they're useful for people who need/want them is the definition of intrinsic value. If you're dependent on other people to need/want it like fiat currency or bitcoins, then it has no intrinsic value. The reason gold is so successful is because it doesn't decay and is easily portable so you can store it under your bed for fifty years or bring it anywhere to whoever needs it which makes it easy to pass along. Of course it's theoretically possible that gold once had value but today we're only trading it around but nobody wants gold except for trade. The use in industrial applications and jewelry suggest otherwise though, as long as there's end users they'll come around to using your supply. Or you can at least sell it from generation to generation until somebody does.

    --
    Live today, because you never know what tomorrow brings