Slashdot Mirror


Cooling Bags Could Cut Server Cooling Costs By 93%

judgecorp writes "UK company Iceotope has launched liquid-cooling technology which it says surpasses what can be done with water or air-cooling and can cut data centre cooling costs by up to 93 percent. Announced at Supercomputing 2009 in Portland, Oregon, the 'modular Liquid-Immersion Cooled Server' technology wraps each server in a cool-bag-like device, which cools components inside a server, rather than cooling the whole data centre, or even a traditional 'hot aisle.' Earlier this year, IBM predicted that in ten years all data centre servers might be water-cooled." Adds reader 1sockchuck, "The Hot Aisle has additional photos and diagrams of the new system."

28 of 135 comments (clear)

  1. Yes, but how much does it cost? by captaindomon · · Score: 2, Insightful

    That's really nifty, and I'm sure it works ok and everything, but... how much does it cost?

    --
    Just because I can hook a shark from a boat, I do no offer to wrestle it in the water.
    1. Re:Yes, but how much does it cost? by jgtg32a · · Score: 3, Funny

      About 7% as much as whatever you are using today

    2. Re:Yes, but how much does it cost? by jaggeh · · Score: 3, Informative

      That's really nifty, and I'm sure it works ok and everything, but... how much does it cost?

      Figures cited by Iceotope show that the average air-cooled data centre with around 1000 servers costs around $788,400 (£469,446) to cool over three years. The Iceotope system claims to eliminate the need for CRAC units and chillers by connecting the servers in the synthetic cool bags to a channel of warm water that transfers the heat outside the facility. This so-called “end to end liquid” cooling means that a data centre, fully equipped with Iceotope-cooled servers, could cut cooling costs to just $52,560 - a 93 percent reduction, the company states.

      taking the above figures into account as long as the cost to install is under the 200k figure theres an incentive to switch

      --
      I would give everything i own for a little bit more.
    3. Re:Yes, but how much does it cost? by Smidge204 · · Score: 4, Insightful

      The idea that the mainboard components are sealed inside a liquid-filled compartment seems like a major point against the system. Extra proprietary vendor lock-in components mean extra costs of owning and operating, which probably offset any savings from cooling... if any.

      I'm skeptical that it will significantly reduce cooling costs (Compared to, say, a chilled cabinet system) because the total cooling load stays the same. If you're generating a billion BTUs of heat you still need to remove a billion BTUs of heat. Any savings will only be from the higher energy densities water allows versus air and maybe initial installation.

      Plus, based on their exploded view, there is no less than three heat exchanges before it even gets out of the cabinet: Chip to liquid (via heat sink), submersion liquid to module liquid, module liquid to system liquid. Each time to go through an exchange your temperature gradient goes up.

      What they need is a system that is compatible with commodity components to leverage low cost hardware against lower cost cooling. Why not fit water blocks directly to existing mainboard layouts and circulate chilled water from the main loop directly through them via manifolds and pump at each rack? You can still enclose the mainbaord and cooling block in a sealed, insulated compartment to eliminate condensation problems, but not being submerged means you can actually repair/upgrade the modules.
      =Smidge=

    4. Re:Yes, but how much does it cost? by fuzzyfuzzyfungus · · Score: 3, Interesting

      Their demo, at least, seems to be aimed at blades, so the inability to just slap any old motherboard into the system would not be a significant change.

      As for water blocks, I suspect that all the various minor chips in the system would be problematic. Even if your cooling of the CPU, or even the top 3-5 chips, by thermal output, is perfect, there are loads of other components that will heat up and die without airflow. CPU voltage regulators, northbridge, RAM, RAID controllers, ethernet, etc. You can't waterblock them all(at least without a serious redesign that makes using commodity components impossible, or a plumbing scheme that would make Escher wince). Either you go with a hybrid waterblock/conventional air cooling system; which gives you the vices of both; or you have to go with the fluid bath as in this setup.

    5. Re:Yes, but how much does it cost? by rainmaestro · · Score: 2, Funny

      And mutated sea bass patrolling the aisles to maintain security?

    6. Re:Yes, but how much does it cost? by Lueseiseki · · Score: 2, Funny

      The price for the bags, knucklehead.

  2. Ugh. by Pojut · · Score: 2, Interesting

    For some reason, the filters at work won't let me view the article. Does it happen to mention how much the upfront cost for these bags are?

    1. Re:Ugh. by BadAnalogyGuy · · Score: 2, Insightful

      Like the unpriced bottle of wine at Applebees. If you have to ask...

  3. Do I get at least a pair of rubber gloves? by Itninja · · Score: 5, Insightful

    Seriously. What do we do when a RAM module or a backplane fails? Will a simple hardware swap become a task for those trained in hazmat handling? I do not want to be on the help desk when someone calls and says "Help! The servers are leaking!"

    --
    I judt got a nre Kinesis keybiartf so please excusr ant egregiou typos.
    1. Re:Do I get at least a pair of rubber gloves? by dintlu · · Score: 3, Insightful

      You pull that server out of the farm and let other servers pick up the slack while you make repairs.

      It's hype, based on the assumption that every server on the planet will be virtualized by 2019, and that the separation of hardware from the software that runs on it will allow IT departments ample time to offload work into "the cloud" while they swap out RAM.

      Either that or it's made for large datacenters with multiple redundancies and enormous cooling costs. :)

    2. Re:Do I get at least a pair of rubber gloves? by ByOhTek · · Score: 4, Funny

      What, they won't? Oh man, this virtualization thing is brilliant.

      So you virtualize a box, so that, if there's a hardware failure, the box can be brought back up on another machine, with minimal downtime! Also, you can run multiple systems on a box saving money!

      We virtualized all our servers around here, went from about 200 servers to 8 machines, each with 16 CPU cores. It went well. So we decided to repeat the process. We then had 4 machines, each taking two VM hosts! It was great, more savings, more vodka for my drawer... So I thought, how could I make this even better...?

      That's right, I put all four of THOSE VM hosts on a 486 in the back room that doesn't even need special cooling. Let me tell you, in terms of Vodka, this virtualization thing has been *quite* productive.

      How could it not be all pervasive by 2019? I'm sure everyone will be virtualizing all of their VM hosts on VM hosts running on 486s by 2019!

      --
      Self proclaimed typo king, and inventor of the bear destroying coffee table (patent not pending).
  4. Re:Excess Heat by von_rick · · Score: 3, Insightful

    In winter you'd get quite a few kilowatt hours worth of heating if you route the dissipated heat properly.

    --

    Face your daemons!

  5. A few questions by Reason58 · · Score: 4, Interesting

    Won't this cause accessibility issues for the administrators who have to support these servers? Additionally, Google's evidence supports the idea that warmer temperatures are better for the life of some components, such as hard drives. Last, this may work well for traditional servers, but I fail to see how this can be made to support a large SAN array or something similar.

  6. Quick Release by srealm · · Score: 4, Informative

    The problem with all this is you need a good piping and plumbing system in place, complete with quick release valves to ensure you can disconnect or connect hardware without having to do a whole bunch piping and water routing in the process. Part of the beauty of racks is you just slide in the computer, screw it in, and plug in the plugs at the back and you're done.

    I'm not saying it's impossible, but just building a new case, or blade, or whatever isn't going to do it - you need a new rack system with built in pipes and pumps, and probably a data center with even more plumbing with outlets at the appropriate places to supply each rack with water. This is no small task for trying to retrofit an existing data center.

    Not to mention that you have to make sure you have enough pressure to ensure each server is supplied water from the 'source', you cannot just daisy chain computers because the water would get hotter and hotter the further down the chain you go. This means a dual piping system (one for 'cool or room temperature' water and one for 'hot' water). And it means adjusting the pressure to each rack depending on how many computers are in it and such.

    The issues of water cooling a data center go WAY beyond the case, which is why nobody has really done it yet - sure, the cost savings are potentially huge, but it's a LOT more complicated that sticking a bunch of servers with fans in racks that can move around and such, and then turning on the A/C. And there is a lot less room for error (as someone else mentioned, what if a leak occurs? or a plumbing joint fails, or whatever. Hell, if a pump fails you could be out a whole rack!).

    1. Re:Quick Release by FooAtWFU · · Score: 2, Interesting

      If you have enough space for a spare rack, and you have a sufficiently virtualized infrastructure, you could just swap in the spare and do rack-at-a-time maintenance. If you're really saving 93% on cooling that could be worth it. (Maybe leave your SAN boxes and other less-failable components on an old air-cooled setup.)

      --
      The World Wide Web is dying. Soon, we shall have only the Internet.
  7. Water cooling on that size is no small feat... by wandazulu · · Score: 3, Interesting

    The ES/9000 that I had contact with was a series of cabinets that were all water-cooled from the outside in...it was a maze of copper pipes all around the edges and back and looked like a fridge. When you opened a cabinet, you could feel a blast of cold air hit you.

    It was no trivial feat to do this, they had to install a separate water tank, some generators (I remember one of the operations guys pointing to a Detroit Diesel generator outside in the alley and saying it was just for the computer's water system), moved a bathroom (only water they wanted around the computer was the special chilled stuff), and I can distinctly remember seeing the manuals(!)... 3-inch thick binders with the IBM logo on them, and all they were for was the planning and maintenance of the water system.

    No wonder it took almost a year to install the machine.

  8. Water is a hassle by BlueParrot · · Score: 4, Informative

    I work with particle accelerators that draw enough power that we don't have much choice but to use water cooling, and even though we have major radiation sources, high voltage running across the entire place, liquid helium cooled magnets, high power klystrons that feed microwaves to the accelerator cavities etc... the only thing that typically requires me to place an emergency call during a night shift is still water leaks.

    Water is just that much of a hassle around electronics. Even an absolutely minor leak can raise the humidity in a place you really don't want humidity, it evaporates and then condenses on the colder parts of the system where even a single drop can cause a short circuit and fry some piece of equipment. After it absorbs dirt and dust from the surroundings it starts attacking most materials corrosively, which may not be noticed at first but gives sudden unexpected problems after a few years. If you don't keep the cooling system itself in perfect condition valves and taps will start corroding and you get blockages. Maintenance is a pain because you have to power everything down if you want to move just 1 pipe etc...

    I just don't see why you would go through the hassle with water cooling unless you actually have to, and quite frankly if your servers draw enough power to force you to use water for cooling then you're doing something weird.

    1. Re:Water is a hassle by tuomoks · · Score: 2, Insightful

      Water (and liquid coolants, even metals) can be a hassle if not deigned correctly. I have had my experiences with water cooled systems but mainly the "over efficiency", well, one burst which shouldn't have happened (LOL).

      One thing I have learned (from my son) - in cars, everything replaced with military and/or airplane grade fittings, valves, tubes, etc - makes life much easier. Not much more expensive but very fast pays back. If I would have known that (much) earlier instead of accepting engineering (good enough) / accounting (cheap enough), my life would have been easier but maybe it's a learning process?

  9. Cray-2 by fahrbot-bot · · Score: 3, Insightful

    "The Iceotope approach takes liquid - in the form of an inert synthetic coolant, rather than water - directly down to the component level," ... "It does this by immersing the entire contents of each server in a "bath" of coolant within a sealed compartment, creating a cooling module."

    Hmm... The Cray-2 was cooled via complete immersion in Fluorinert way back in circa 1988. I was an admin on one (Ya, I'm old). So, this is a bit different, but certainly not ground-breaking.

    --
    It must have been something you assimilated. . . .
    1. Re:Cray-2 by jcaren · · Score: 3, Interesting

      The crays full immersion coolant model hit a big problem - the coanda effect.

      This is where layer of fluid near the actual component flows much slower than actual flow - in layers slowing down exponentially as it gets closer to the stationary components.

      For air this is not too much of a problem - only a very fine layer of stationary air over compenents that does not affect cooling. But with liquids the effect is both noticable and severely impacts coolant flow over hot surfaces - with some then "next gen" cray chips actually boiling the fluid. As todays chips run much hotter and generate a lot more heat than those Cray chips I can see this being a major problem today...

      Crays fix for this was to move from full fluid immersion to immersion in droplets 'injected' using a car fuel injector.
      This got everywhere and evaporated taking the heat away from components.

      Rumor has it that during devlopment, engineers bought fuel injectors for a wide range of cars and the ones for certain porsche worked best so they bought the entire stock of fuel injectors for this car in the mid-west and used them...

      I remember staff at cray giving away Porsche style sunglasses with Cray written on them instead of Porsche and when I enquired why - the above was the tale I was told by sales staff.

      Whether true or not is something else - the cray sales staff in those days had a seriously odd sense of humor...

    2. Re:Cray-2 by jbengt · · Score: 2, Informative

      The crays full immersion coolant model hit a big problem - the coanda effect.

      You did not describe the Coanda effect, you described boundary layer issues. I don't know enough about the story to know whether boundary layers were the real issue: It's pretty routine to take into account the fact that friction causes the fluid velocity to approach zero at the surface of a stationary object, and to account for a lack of turbulence in the laminar part of the boundary layer that reduces heat transfer. It could just be that they needed to get a phase change at the surface in order to pull away enough heat.

      By the way, the Coanda effect is the property that causes cold air being thrown horizontally out of a ceiling diffuser to stick to the ceiling rather than fall in cold drafts. The air stream is squeezed by the supply opening, and therefore has to speed up to conserve mass flow. It gets that extra velocity by losing pressure, so the air supplied is at a lower air pressure than the room air, and, as a consequence, is pushed up against the ceiling until it mixes with the warmer room air and slows down.

  10. weight? by Clover_Kicker · · Score: 4, Interesting

    How much does a rack full of water-cooled blades weigh?

    Never thought I'd see the UPS become the lightest thing in the server room.

  11. night time freezing of liquid would save more by Locutus · · Score: 2, Interesting

    The technique of using cheaper off-peak energy to freeze liquid and then use that liquid for daytime cooling loads is already used in a very few places. Combine that technique with the direct server cooling mentioned in the article and....wait a minute....they are already claiming a 93% cooling cost cut? Either their is huge waste now or they're already expecting to use off-peak energy. But then again, maybe the remaining 7% is still large enough to merit further savings.

    Direct cooling makes far more sense than cooling rooms like I keep seeing around now.

    LoB

    --
    "Anyone who stands out in the middle of a road looks like roadkill to me." --Linus
  12. Re:Coming back full circle by Digestromath · · Score: 3, Funny

    Next up, cooling servers with a bag of frozen peas?

  13. Re:Data Centre/Center? by osu-neko · · Score: 2, Informative

    Ok, so they are British and they spell 'center' with the 'er' the other way around. Why don't they spell server as 'servre'?

    First of all, it would be 'serveur', not 'servre'. And its use is too new to be one of those words in which the french spelling is retained from the days of the Normans. Incidentally, even in America, we fail to reverse the 'er/re' in some words, consider 'acre', 'massacre', and 'mediocre', so we're not exactly consistent either...

    --
    "Convictions are more dangerous enemies of truth than lies."
  14. Re:Excess Heat by CompMD · · Score: 2, Funny

    "Very little, since you're dealing with very low quality heat."

    And that's why in the really good systems, the only acceptable option is Monster Heat, the finest quality heat available, and its even gold plated.

  15. Nospill valves and other modern water cooling tech by DrYak · · Score: 2, Informative

    No spill (as in "almost insignificant", not as in "not too much, won't empty the whole system, but you better have some towel nearby just in case"), quick disconnect, low resistance valves for watercooled system have been already available for quite some time for enthusiasts.

    (Koolance is an example of compagny producing such thing in the US, Aquatuning is an example of shop selling similar implements in the EU - no links to avoid gratuitous advertising to web spiders, but you can easily google the names).

    Anyway, low conductance liquids are popular in application where spills and leaks aren't easily monitored (see above source). And don't forget that every other blade module is sealed too. So in case of leak you're just spilling... on a sealed container which isn't affected by external liquids anyway.

    As for pressure : Well, uh, no. You would need tremendous pressure if you had to fill the whole rack using 1 single pump. Which would be a single point of failure and is bad.
    The more sensible approach would be each blade module having its own small pump (Laing DDC for the win !!!) for pumping water out of the rack's main tank.

    It's already the scenario used in most rack-cooling situation (see again mentioned sources above). And in case of pump failure, well, only 1 blade module fails. The rest of the rack is immune to it.

    Well I'm sure most /.er have some ricer friend (the kind which custom hand compile gentoo with "-O9999" :-) ) to whom a massive failure of watercooling has happened some time ago. Watercooling safety has evolved since then and it's now much more secure even for simple enthusiast. Now, a company specializing into data-centers has even more possibility to offer safety.

    --
    "Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]