Slashdot Mirror

Liebert made one of these for server room UPSs. We never got one although the salesman tried to get us to buy it. The thought of that wheel sitting in the next room and spinning that fast spooked me. I am not religious but there is no need to constantly tempt fate by working next to that kind of energy day in and day out. I guess it is a good way to store energy but I really dont want one in my backyard, basement or server room. Let's see the explanation for that disaster. Well, we made it through the hurricane and the earthquakes and, nope, the fire did not take us out. However, when that damn flywheel got out of balance due to cheap magnets in the bearings.... Damn thing took out half the rack before going through the roof and into the cafeteria. Just saying, I like fire but I dont carry an incendiary grenade in my back pocket.

You want gear with integrated lights-out-management. Any gear that supports SSH and SNMP should be perfectly usable over a shitty connection.

Most (all?) of the Sun servers come with an embedded ILOM that supports remote KVM through a web browser with Java as well as SSH. The SSH access gives you full out-of-band power control over the server, and can be used to look at system part numbers, power supply voltages, fan speeds, etc. Additionally you can configure SNMP monitoring/traps through the ILOM no matter what OS is running on the box.

We've used the x2200 M2, x4200 M2, and x4540 servers and the ILOM in each of them means I never have to go down to our data center to physically touch a box.

Ironically, some of the HP DL series have integrated out-of-band management called iLO, but they charge an additional few hundred dollars to gain features such as KVM or authentication. I don't like paying extra for features that should just be available out of the box.

The other thing you want is remotely managed online battery power. You want your power to be clean, going through a dual transformer conversion so no matter what kind of crappy power you have at the site, your gear is getting a nice clean voltage. Get something that has a good network-management interface on it. I've used MinuteMan Endeavor, Liebert GXT2, and one from APC that was online, but I can't seem to find it now. Each of these supports SNMP and web-based management.

I empathize with previous posts indicating that you are designing a computer room that happens to have the IT guy's desk in it, as opposed to an office per se.

If you've only got 4 racks of machines to house, depending on the power density a self-contained solution such as the Liebert MCR (mini computer room) may be of interest (APC also has similar products). These are racks with built-in aircon, UPS, & cable management. While they are certainly not cheap, they may be easier to fit into the budget than dealing with electricians & HVAC contractors to set up your computer room with a custom solution providing equivalent facilities. They are also a better investment as you can take them with you the next time you move, add more as necessary anywhere you want, etc.

Also they are nice racks made to show off das blinkenlightz and should be placed so that they are in view of anyone addressing you while seated at your desk if possible (your desk facing the door of the office with your back to the servers seen through a window etc). This may seem frivolous, but impressive cleanly installed racks of servers staring back at the minions as they address you behind your monitors (leave the lights off in the server room unless you're using them) will certainly add weight to your position in conversations. They will soon learn to fear the BOFH :-).

Why call it "Liebert XD" ? What, it's not water ?

Ummm, because it's made by the Liebert Corporation? And it's their "XD" line of products?
Is this the first time you've ever heard of branding?

You might as well ask: "Why call it 'Toyota Camry'? What, it's not a car?"

it's still just liquid cooling, something that existed in the server room long before Michael Dell ever sold a single server machine.

So? Cars have been around a long time, too. That doesn't mean I want to drive a Model T.

Anyway, the news here is not that there is a new HVAC product, but that Dell is going to be selling HVAC systems to datacenter owners.

Also, this system uses a gas/liquid phase-change cycle, and it operates on a different scope (zone and spot cooling -- doesn't get anywhere near the CPU), so it's really not like what most people would think of as liquid cooling systems for computers. It's just a way of getting the cooling closer to the heat source instead of blowing cold air around in ducts, such that your HVAC operates more efficiently.

Sure, this idea has been around for a while (though this system makes some improvements that are especially helpful for datacenter use). The news is that Dell is selling it.

The ZDNet article is, unfortunately, very scant on details when it comes to our XD (short-hand for "eXtreme Density") system. For one, they give the impression that cooling coils are water-based. They're not; they're refrigerant, just like a whole-house air conditioner. The specific coolers they seem to be referring to are the XDV, an 8kW unit which mounts directly to the top of the rack, and/or the XDO, which is a 16kW unit which hangs in the center of a cold aisle. Just like an air conditioner, hot air is sucked in at the top/side, run across the evaporator coil (cools the air, boils the refrigerant), and cold air is then shot downwards towards the front rack face. The hot refrigerant makes its way back to a heat-exchange unit (an XDC chiller or XDP pump), where it dumps the heat into the rejection loop. This could be anything from a building chilled-water system, to roof-mounted drycoolers, or even a subterranean geothermal system. For a better idea of just what the thing is (and to see lots of pretty graphics and thermal images), the family "sales glossy" lives here(PDF).

TileFlow is excellent for most conventional datacenters and has served us well. Best practice in a conventional datacenter is to use TileFlow to ensure positive pressure and appropriate distribution (don't forget to model all your underfloor blockages), set up cold rows and hot rows, and then use a drop ceiling as a plenum with strategically located grates to pull the hot air back into your CRAC units. The only problem, as others have pointed out, is that it's only a 2D model presently and you really need to start modelling your load in 3D when you get past around 12kW/rack.

We've speculated that if we were to start all over again we'd skip the raised floor and do a bi-level drop ceiling with one level being cold air distribution to cold rows and the other level being the hot air return. Let cold air fall and warm air rise, and augment it all with XDO's from Liebert.

We worked very closely with Liebert ( http://www.liebert.com/ ) when we recently rennovated our data center for a major project. The traditional CRAC (Computer Room AC) units supplying air through a raised floor is no longer viable for the modern data center. CRAC units are now used as supplemental cooling, and primarily for humidity control. When you have 1024 1U, dual processor servers producing 320 kW of heat in 1000 sq ft of space, an 18 inch raised floor (with all kinds of crap under it) is not adequate to supply the volume of air needed to cool that much heat in so small a space.

We had intended to use the raised floor to supply air, but Liebert's design analysis gave us a clear indication of why that wasn't going to work. We needed to generate air velocities in excess of 35 MPH under the floor. There were hotspots in the room where negative pressure was created and the air was actually being sucked into the floor rather than being blown out from it. So, we happened to get lucky as Liebert was literally just rolling off the production line their Extreme Density cooling system. The system uses rack mounted heat exchangers (air to refrigerant), each of which can dissipate 8 - 10 kW of heat, and can be tied to a building's chilled water system, or a compressor that can be mounted outside the building.

This system is extremely efficient as it puts the cooling at the rack, where it is needed most. It's far more efficient than the floor based system, although we still use the floor units to manage the humidity levels in the room. The Liebert system has been a work horse. Our racks are producing between 8 - 9 kW under load and we consistently have temperatures between 80 - 95 F in the hot aisle, and a nice 68 - 70 F in the cold aisles. No major failures in two years (two software related things early on; one bad valve in a rack mounted unit).

This is already being done by at least one vendor.

http://www.liebert.com/dynamic/displayproduct.asp? id=1077&cycles=60hz

Liebert cools the Virginia Tech XServe cluster.

From http://www.tcf.vt.edu/faq.html

Q: How much heat does System X generate?
A: Each rack full of equipment generates in excess of 8 kilowatts of heat on a continuous basis and the machine consists of about 40 racks. That's like 240 hair dryers on high constantly in 1000 square feet of space.

Q: How is System X cooled?
A: System X uses a Liebert Extreme Density cooling system that is fed off of a chilled water loop. There are two 125 ton Carrier water chillers that provide roughly 3 million BTUs of cooling capacity. This chilled water loop is heat exchanged in the Liebert XDP units with a R-134A refrigerant loop that is fed to the rack mounted liquid-to-air heat exchangers. We only use about 110 tons of the 250 ton capacity.

Target uses Automated Logic HVAC controls. ALC has had a long-standing relationship with Liebert, and *they're* the "specialized" HVAC guys that do server rooms. I know all this crap because I work for an ALC dealer. I do this stuff for a living.

As for the GP's question about outside air: yes. Outside air is very commonly used for "free" cooling. You have to calculate the enthalpy, which is the potential amount of cooling based on the temperature and humidity of the air inside vs. outside. Basically, if you have low outside air humidity, your enthalpy can go sky high. This is especially true if your indoor humidity is high. It's basically just a calculation of how much surface evaporation is going to take place if you mix these two air sources.

Of course, a proper HVAC unit is going to have decent (and hopefully well-maintained) filters. Which means that those server room units are already using as much outside air "free" cooling as they can.

You can purchase a 15 ton unit that is mounted in the ceiling. I just bought 2 of these.

It's made up of three 5-ton cooling units that can be individually controlled. If you can't vent into the plenum (ceiling), you're looking at putting in a glycol loop up to a condenser on the roof or outside.

Resource-wise, call Liebert and similar companies and have a rep come in and do a dog and pony show. Also check with local contractors, as they frequently know about palleted and sold units that never got installed, that can be purchased for a huge discount.

No, the 'big fan-things' are called XDVs from Liebert (http://www.liebert.com/dynamic/displayproduct.asp ?id=1077&cycles=60hz). The mount on top of the rack.

Better yet, don't use water.

If you look in any well-built data center, you'll find air conditioners that are set up like this. You've got the air conditioner itself inside, which emit heat into a glycol coolant which is then fed to a chiller on the roof (basically a radiator with a fan blowing on it).

Glycol is an excellent thermal conductor, and it's just toxic enough that slime and other germies can't grow in it.

I've deployed a lot of small data centers. Your choices in what to buy are actually easier than you think. You can calculate your power and cooling needs down to the last BTU and KVa, and then you have to buy equipment in a size they make them in! Just go the smaller end of the lines.

You'll end up with a 3 ton or 5 ton air conditioner. Liebert air conditioners can also humidify/dehumidify and heat/cool. There is a market for used Lieberts if you want to save some money. Call your local A/C contractor.

A 3Kva UPS would be a good size unless you want more standby time in which case you could go for a 5 Kva or maybe two 3's for redundancy.

Liebert makes great UPSs, too. The APC Matrix line is a pretty good design because you can hot-swap the batteries yourself.

I've deployed a lot of small data centers. Your choices in what to buy are actually easier than you think. You can calculate your power and cooling needs down to the last BTU and KVa, and then you have to buy equipment in a size they make them in! Just go the smaller end of the lines.

You'll end up with a 3 ton or 5 ton air conditioner. Liebert air conditioners can also humidify/dehumidify and heat/cool. There is a market for used Lieberts if you want to save some money. Call your local A/C contractor.

A 3Kva UPS would be a good size unless you want more standby time in which case you could go for a 5 Kva or maybe two 3's for redundancy.

Liebert makes great UPSs, too. The APC Matrix line is a pretty good design because you can hot-swap the batteries yourself.

Liebert maked a self-contained rack with built in air conditioning and UPS. Details here.

I had a similar problem with equipment strewn about, and being "forced" to have my office in the server room! What I have requested (you know, companies will put off a decision as long as possible so it looks like they're trying to save money) is a "Little Glass House" from Liebert. It is a totally sealed rack enclosure, insulated against noise, has an environmental control unit, backup cooling fan in case power goes out, online UPS system, and you can get an option to exhaust the hot air into the plenum space so that if your boss 'knows' the normal room A/C is enough you don't have to work in the Sahara. Also available are water/smoke/temp detectors, SNMP management, etc. Did I mention it's lockable? Now you can keep the windows people away! (You know, the "Maybe we should just reboot it and see if it helps" people) -Jim