Cooler Servers or Cooler Rooms?

mstansberry writes "Analysts, experts and engineers rumble over which is more important in curbing server heat issues; cooler rooms or cooler servers. And who will be the first vendor to bring water back into the data center?"

Comments

Why not both?

[tquinlan]

Unless you make things so cold as to prevent things from working properly, why not just do both?

Re:Why not both?

[Anonymous Coward]

Cost.

The question isn't whether it's good to keep both cool. The question is, which makes more financial sense? Cooling the whole room? Spending the money to purchase servers with a low heat-to-computation ratio?

Probably a combination. But to say that people should equip every computer with VIA ultra-low-power chips _and_ freeze the server room is silly.

Re:Why not both?

[FidelCatsro]

This is exactly the point , you need a dammed good balance.
If i have to be working on the server localy in some fashion i would rather not be boiling or freezing .In the average working enviroment We cant have it in a refrigerated room as this would bring up alot of working issues and not to mention design of the room itself .on the other hand we don't want a drasticaly complex cooling system that would add another possible avenue for failure to occur.
The best is possibly a nicely air conditioned room with a nice simple cooling system on the server , good airflow and a comfertable working enviroment .

Re:Why not both?

[elgatozorbas]

The question isn't whether it's good to keep both cool. The question is, which makes more financial sense? Cooling the whole room? Spending the money to purchase servers with a low heat-to-computation ratio?

In that case, I would say: cool the room. The room is forever (on this timescale), the servers maybe change every 5 years. Of course, start by NOT choosing the hottest server too, but I would invest in the room.
Also I don't expect room cooling techniques to improve significantly in the next few years. Servers hopefully will.

Re:Why not both?

[ePhil_One]

Also I don't expect room cooling techniques to improve significantly in the next few years.

Why would we need to improve room cooling techniques? We can readily freeze CO2 solid, I'd say we're pretty damn good at making things cold.

Re:Why not both?

[Havenwar]

Agreed.

although if it is the kind of complex where one is to work with the servers at the location... I'd rather keep the air moderatly cool and stick with cooler servers.

Re:Why not both?

[Anonymous Luddite]

>> I would say: cool the room.

I think you're right. That's the way we do it, but - (there's always a but) some cabinets still get damn hot depending on what's in the rack. Sometimes you need to do spot cooling as well, or put in bigger fans to keep the equipment closer to ambient.

I think starting with a cool room is the most cost effective way though - not to mention it makes work "O.K." in August...

Re:Why not both?

[ReeprFlame]

Both need to be done to an extent. The priority should be kept directly to the machine since that is where the heat issues are. Also, cooling a room is a lot more time intensive and expensive. Think about it, you have the entire volume of the room to cool with much of that air being void and useless. Then by time the cool air reaches the CPU and components, it will not be doing much work cooling the system at all. That is why the focus of heat removal should be taken from the components directly, released into the air that is moderately cool [to remove overall heat]. That would be an effective system. BEtter yet [although not very feasible] run water cooling to the systems and put the heat exchanger in another room so you need not cool the room at all...

Re:Why not both?

[FireFury03]

The question isn't whether it's good to keep both cool. The question is, which makes more financial sense? Cooling the whole room? Spending the money to purchase servers with a low heat-to-computation ratio?

You don't need to cool the whole room - you could just cool the cabinets. Most cabinets have doors and sides, an open bottom and fans at the top. So you can blow cold air up the inside of the datacabinet (which is what most datacentres do anyway) and take the air from the top to recycle it with reasonably minimal air (and hence heat) exchange with the rest of the room.

Re:Why not both?

[Anonymous Coward]

You could have both. But to what ratio?

Let's apply the IT paradigm here, what you are going to need is scalability.

What if you get a nice aircon that keeping your current number of servers is at the top end of its limits and suddenly you need to add 20 more servers and a whole bunch of additional racks?

If you have servers with adequate cooling, you could scale up without having to get a super expensive air con unit and not have to replace or upgrade the unit immediately, because the servers can handle a lot of the heat problems themselves.

-SJ53

Re:Why not both?

[banuk]

cool room + hot female boss? mmmm tasty

flamebait I know.. but can't argue with mmmm tasty logic

oh and cooler chips for the servers too

Re:Why not both?

[Frumious Wombat]

I actually did that, so ask again in six months to see if it works.

I bought Mac servers, as for the cycles they put out relatively little heat (at least compared with an Itanium), and then I specified the room had to be able to handle 80K btu of heat, continuously. So, the U. renovated a room, controlled the volume (translation: assistant professors can't have too much space), and put in a recirculating water unit with the chiller out on the roof.

If I could have afforded it, I would have use VT's approach, and cooled the rack itself, as well as having the room turned down to the limits of comfortable for the humans. However, that's for some later point down the line.

I just hope it keeps working when the winter snows settle back in, and start to cover the chiller.

Re:Why not both?

[Fjornir]

Definately. My boss worked out every day so she was always wearing a sports bra. When she walked into the server room they both stood at attention (and so did I...)

Re:Why not both?

[ArmedGeek]

I think the OP meant that you'd not have to upgrade or otherwise change the room cooling.

Re:Why not both?

[Datamonstar]

Damn, man. What company is this at so I can apply?

Re:Why not both?

[Fjornir]

Macrohard.

Re:Why not both?

[LucBorg]

Making the servers more efficient, use less power, and therefore run cooler will probably be better in my opinion. Cooling rooms costs more and with people moving in and out of the rooms, there are major shifts of air, bring warm/hot air in and taking cool air out. This means even more energy has to be used to cool down the room to optimum levels again. The problem is only worse in the summer! The only case for hot servers and cool rooms is probably in the arctic/canada/antarctica. Here its cold enough to just cool the rooms, as even in the summer the ambient air temperature outside is pretty low, and can be lowered to the optimum temperature for the servers, if required, very easily, with little energy being used, and therefore with little cost.

Re:Why not both?

[Sylver Dragon]

I wonder if a more directed cooling might work better. For instance, change the rack/chassis design such that it expects airflow to come from the top and exit the bottom, then duct the A/C right into the top of each rack. While you would still keep the rest of the room cool, it just seems to be wasteful to keep a several hundred square foot room at 60 degrees the whole time when the real goal is to keep the equipment in the racks from baking itself into oblivion.
I also agree with the guy in the article, liquid cooling in the server room is going to happen eventually. I got to see the difference a simple watercooling system made on a P4 3.02GHz Extreme Edition chip, stuffed in the same case with a GeForceFX5950. Even with some serious fans the case was borderline overheating in games. Part of the problem being that the room it was in wasn't kept that cool, and the owner had it in a cabinet in his desk (it is what that cabinet was designed for). He dropped a liquid cooling system into it, and now the thing is always nice and frosty. And even with the jolts and jostling of taking the system to several LAN parties, the liquid cooling system is still leak free and rock solid. His experience has actually made me consider one for my own next system. For a server, where the system usually sits still long enough to collect a measureable amount of dust, water cooling may be a very good choice. If it's installed properly the likelyhood of leaks is low, and the performance can be very good. Heck, I can see it now, our server rooms will eventually have a rack or two devoted entirely to the radiators for the liquid cooling systems of servers, which run hot enough form plasma.

Re:Why not both?

[drgonzo59]

Not just cost but also reliability (in the end also a cost issue). Having water flow among the CPUs, hard drives and all other components is just asking for trouble. In case of a leak it won't be just one damaged CPU, or one memory stick so the system can still compensate and keep going, but the whole box or maybe rack might be ruined.

This doesn't seem like an either or situation or a large research question. A cost and reliability analysis should determine what it better for each individual setup.

Re:Why not both?

[blogeasy]

The traffic flow of air is critical. It only makes sense to cool a room if you are properly allowing the heat to leave the room. Racks should be designed to take the hot air coming from the backs of the units and sending it to the cooler intake. From there, it is cooled and recirculated back to the fronts of the units. This way you make effective use of the cold and hot air masses without using extra energy to cool hot air in a bigger room.

Unfortunately, most of the server rooms that I've been in are small with poor air circulation and it always seems very warm.

Re:Why not both?

[MrPC81]

Cooling a unit generally means exchanging heat out of a machine and dumping it in the room. Tis not efficient when you then have to take that heat out of the room and dump it outside the building. Picture a refrigerator in an air conditioned room. The heat radiates out the back, but then has to be cooled again by the air con. With logic like this, can anyone say brownouts?

Re:Why not both?

[temojen]

I can see it now, our server rooms will eventually have a rack or two devoted entirely to the radiators for the liquid cooling systems of servers, which run hot enough form plasma.

It seems more likely to me that the radiators would be placed outside. I could forsee water cooled racks that come with a centre mounted warm and cool water manifolds plumbed to high flow lines to take all the water to one big radiator outside...

Or probably easier to manage, a 2-4U centre mounted unit with the manifold and pumps for that rack, circulating water through that rack and to/from a central resevoir (55 gallon plastic drum) in the server room, and annother set of pumps and very large pipes to take the water through an outside radiator.

Each (1U, 2U, etc server ) unit could have it's own rear mounted hose attachments (and bleed valve) in a modular fashion so you just hook up a new computer to the warm and cool manifolds, open the valves, bleed the air, and your new unit is cooled. To remove it, just shut the manifold valves, open the bleed valve, put a bucket under the lines, and take the lines off the connectors.

Re:Why not both?

[hey!]

It's pretty simple to figure this out with current day prices. You just crunch the numbers and see what mix of investments win. The place where it gets interesting is the place it always gets interesting: predicting the future. Will energy prices rise, and if so how quickly? If you expect them to double in a hundred years, then you can probably work from present day costs. The net present value of these future savings is nil. If you expect them to double in three years, then you have to do the operating cost calculations differently, as you would if you expected dramatic price drops soon.

Another factor to consider -- what value to your organization does knowing that the price is predictable? Suppose that building and operating an energy efficient data center is within the financial parameters that make sense for your organization, but you could save money by building a gas-guzzler. What is the liklihood that near to mid term energy price rises could make operating the gas guzzler impractical? What would the impact be on other aspects of your business? If you assigned an expected cost to this scenario, does it offset savings?

Finally, there is a ethical/public image issue. Is it acceptable to waste energy? If we are going to be efficient, can we get some secondary benefits out of this?

Re:Why not both?

[IBeatUpNerds]

Agreed. Though I lean a little more toward cooler machines. Buying a cooler and more efficient machine is a one time investment. Cooling an entire room is ongoing and definitely more expensive in the long run.

Re:Why not both?

[AndyCater]

Talking to someone putting in a cluster the other day - air cooled, two racks - 7.5kW or so of heat. He mentioned that IBM were talking, seriously, about water cooling the doors of the racks in order to maximise the effectiveness of chilled air. The consequences of leaks can be left to the imagination - but there's nothing intrinsically wrong in the idea

Re:Why not both?

[misleb]

Probably a combination. But to say that people should equip every computer with VIA ultra-low-power chips _and_ freeze the server room is silly

Was that said? -matthew

Re:Why not both?

[mr_exit]

The reason for keeping the whole room cool is buffer.

In your scenerio if one aircon system fails then that rack will start overheating straight away. if you have a room full of cold air already you have a few extra minutes before you have to think about shutting down a rack or two or turning up the aircon on the remaining units.

Re:Why not both?

[Sagarmatha]

Well said. The breakdown occurs when you look at the airflow required for higher watt density equipment such as blade servers. You could theoretically fill a std. 42U cabinet with 6 of these 7U chassis. Seeing that each chassis (using IBM here as an example) can pull 4kW of power, we could draw 24kW of power per cabinet. In most data center layouts, cabinets generally have one perforated tile in front of them which can supply 300 cubic feet per minute (CFM) based on std. design under-floor static pressure. To maintain an acceptable temperature rise through the server (so the exhaust isn't melting things), we need to supply approx. 120 CFM per kW of load. For 24kW that would be 2880 CFM. Seeing that the cabinet only has one 300 CFM tile in front of it, we have a problem Houston. There are variety of successful air-side strategies to deal with this including supplemental air delivery and exhaust air segregation. Ultimately, to go much higher than 24kW, we may need to use liquid cooling, which has the ability to remove far more heat, but with the liabilities mentioned by others. Cooling the room more doesn't help, as it's really an air delivery issue, regardless of supply temperature. In fact, overcooling is tremendously expensive due to the dehumidification effect of really cold cooling coils. The humidity then has to returned to the air to maintain the 50% RH goal. There are huge energy losses associated with this. Additionally, when the under-floor supply temperature goes below the dew point, some surfaces will as well, and they will have condensation occurring on them. Needless to say, condensation is not good for PC boards. In the short term, separation of the supply air and the exhaust air to avoid mixing is a good strategy. By avoiding mixing, assuming the available air volume is sufficient, high density power consumption can be cooled. A raised computer floor isn't even necessary for this if the air delivery method is designed right.

Re:Why not both?

[Anonymous Coward]

Just put them into the actual duct (the whole thing) and remove the side panels so the air goes through it sideways, then you can put them every few feet. You wouldn't need a special room.

Re:Why not both?

[AmberBlackCat]

In my junior high, all of the computers labs were ice cold in order to make the computers last as long as possible. And in my high school, pretty much every classroom was filled with computers so they basically froze the whole building. Students were wearing their coats all day in school. Students who came from other schools to attend summer school were constantly complaining about the environment. To make things worse, wearing coats in class was against school policy, so people had to choose between freezing and getting in trouble.

In the spring time, we'd all be in short skirts and cutoff shirts and such. The frigid air was killing us and people called us idiots for subjecting ourselves to the torture. But this was normal behavior. We were not the problem. The problem was the building was set up more for the computers than for us.

The moral of this story is get better servers because cooling the whole room affects more than just the servers. I realize there are rooms filled with servers that people rarely go into, but there's got to be some lab technician or network admin who has to spend a lot of time in there.

Re:Why not both?

[wh00dini]

that is a great idea until the a/c unit in rack 3 goes and you go from your nice fridge to a hot house.

Re:Why not both?

[Anonymous Coward]

"In the spring time, we'd all be in short skirts and cutoff shirts and such"

Please let you be speaking of an all girls school...

Re:Why not both?

[sr180]

Heck, I can see it now, our server rooms will eventually have a rack or two devoted entirely to the radiators for the liquid cooling systems of servers, which run hot enough form plasma.

Too many people forget history. These problems have all been solved before. If you look at the Computer Science Divisions of most universities, you will generally find a lake or large pond nearby. Why? Because it was originally designed as a radiator for the cooling of the early computers.

Re:Why not both?

[Doppler00]

I think liquid cooling is a great idea, but they need to first: Standardize!

Have standard hoses, and connectors with quick disconnects. You buy a rack mount 1U server and it would have the standard cooling ports in the back of it. Then just design a central liquid cooling system. You could just put the cooling connections in the wall like you would the power outlets. Then just plug in the server to this connection. This way, you could have the heat transfered directly to an AC unit on top of the building instead of trying to inefficiently just cool the air in the room.

Re:Why not both?

[rocca]

I'd question if it's more efficient though. There is still the same amount of heat energy that needs to be cooled, it is just more concentrated. Ie, instead of having to cool 100 sq feet by 1 degree, you have to cool 10 sq ft by 10 degrees. This of course assumes there is no temperature influence from other forces such as solar energy coming through windows, etc.

Venting through the bottom is typically done to ensure there are no hot-spots created by lack of airflow, not to concentrate the heat exchange.

Re:Why not both?

[dadtenx]

After reading many of the comments and replies I thought a little illumination on the power and cooling issues would be in order. First temperature and humidity. Theoretically, electronic components operate better at colder temperatures. In reality, both temperature and humidity must be kept in a very tight range for proper operation of servers. Too dry and you risk electrostatic discharge. Too humid and you risk condensation on circuit cards causing errors and failures. Second, cooling the room vs. cooling the servers. Regardless of the approach, the heat produced by the heat source must be removed. Most of the energy it takes to cool a rack of servers is used in rejecting this heat, not in cooling the surrounding air. Yes, energy is used by fans to circulate air across cooling coils, but other methods of heat rejection use similar amounts of energy to affect a thermal exchange. One watt of power consumed in a server results in 3.14 BTUs of heat energy that must be removed. It is the density of this power (and thermal) load that dictates the cooling tactics. High density racks loaded at 4kw per rack are typical and can be handled with standard air handling quipment. Very high density cabinets loaded at 10-15 kw per cabinet must use a combination of air and active thermal exchange at the cabinet level to achieve adequate heat removal. Third, AC vs DC to a rack. AC to DC power supplies are not a significant source of heat in the data center. CPU processors are the most significant and that is why multiple processor servers and blades require so much power. Spinning disk arrays in SANs and DASD cabinets are another. There is a growing inventory of switches, routers and servers available with DC power supplies driven by companies that come from the legacy telco world. However the DC suppies expect an input voltage of only 48VDC. A rack of servers requiring 4kw of power at 48VDC would need an electrical circuit of 100amps from a source less than 100 feet away to avoid voltage drop. These constraints result in a lot of wasted copper and higher critical infrastructure failure probabilities...not to mention doing very little to save energy. The reluctance of hardware mfgrs to go back to liquid cooled equipment will mean that we must deal with the growing power density issues with creativity and innovation.

Uhm..

[tewmten]

Why not both?

Aquafina...

[vmcto]

Will probably be the first vendor to bring water into the datacenter... I believe I've seen evidence in some datacenters already.

Re:Aquafina...

[Anonymous Coward]

Yes, and since liquid (molten) sodium chloride is used in some nuclear reactor designs as a heat transfer mechanism, the obvious conclusion would be to use water + sodium chloride to cool the server room! Yup, salt water...just spray it everywhere!
Ooops...gotta go, my toyota truck is in the shop...for some reason, it get's really rusty after the Winter each year....

Re:Aquafina...

[alex4u2nv]

and water sprinklers.
Server #5 in row six is on fire. Quick turn on the sprinklers and kill the fire along w/ the other servers, and causing more electric fires!

BrillianT!

Re:Aquafina...

[_Sharp'r_]

Well, since my last company had most of their servers in a data center room where we had two different floods, I'd say I have a pretty good idea which hosting company will be the first to bring water into the data center ...

The first problem was snow that piled up outside, combined with clogged drains, that led to melting snow coming in through the wall where some pipes entered/exited. Since their layout was power in the floor and networking in the ladder racks, it's actually pretty amazing that a large portion of the power plugs and switches still worked, even while being submerged in 6 inches of water.

So about a year after they had taken care of that issue, a water pipe for a bathroom on the floor above burst, and of course the water came down right in our room in the hosting center. It wasn't so bad until the flourescent lights in the ceiling filled up and started to over flow. We were able to limit the damage by throwing tarps over the tops of all the racks (there goes your cooling effect, though), but we still lost about 100K worth of server and switching equipment.

So yeah, water in the data center? It's been done.

Re:Aquafina...

[Anonymous Coward]

And don't even get me started on Evian - that stuff will turn you gay

... suppose then Adamian turns chicks lesbian?

Re:Aquafina...

[whmac33]

My wife refuses to drink Dasani and Arrowhead. I agree that they suck, and would never pay for either.

Sparklets and Aquafina are the choices we make.

This won't be much of a flame war.

Re:Aquafina...

[ari_j]

Generally, I don't buy bottled water. However, I got into the habit of buying it while I lived in Phoenix, AZ, and Aquafina was what I drank. Arrowhead may advertise a lot, but it just doesn't taste good. And Dasani...is just about as homogenic (is there a real word for "things that make you turn gay"? ;-D) as Evian. Plus they taste bad. And Fiji has to be the highest-priced urine sample available anywhere.

Re:Aquafina...

[Anonymous Coward]

it's actually pretty amazing that a large portion of the power plugs and switches still worked, even while being submerged in 6 inches of water.

I'll agree with you on that...this is a bit more amazing, though...

IBM server survives a month submerged in salt water
http://www.greencracker.com/archives/2004/12/rebir th_of_a_su.html

Re:Aquafina...

[_Sharp'r_]

That's impressive, but it'd be more impressive if it was actually turned on while it was submerged....

Re:Aquafina...

[coolmadsi]

A friend of mine tried to watercool his computer for some reason or another, it would have worked fine but one of his extra components leaked and his computer got a lot of water inside where it shouldn't.

Re:Aquafina...

[Anonymous Coward]

Evian is Naive backwards....

uh, "rumble"

[koreaman]

it sounds like they're having some kind of gang warfare over the topic...what the hell?

Re:uh, "rumble"

[Master_T]

Yeah man.... I got stuck in one of them cool room or cooling system "rumbles". They are hardcore. I lost a buddy in one those... pen wound. I still have a scar from when a Cool Roomer stabbed me with a ISA Modem.

Re:uh, "rumble"

[BarryNorton]

I think the author was making a stab for 'ruminate', but couldn't quite manage...

Re:uh, "rumble"

[Anonymous Coward]

it sounds like they're having some kind of gang warfare over the topic...what the hell?

Clearly you've never seen a meeting between the Unix sysadmins and the facilities managers for an overheating data center.

Re:uh, "rumble"

[hey!]

OT but true. I was talking with a guy who coaches top level international martial arts competitors last week about hitting power. He mentioned those guys who do that stunt where people hits them with a fat sticks and the sticks break.

"What would really hurt, though" he said," is to get hit with a cable."

well I've always wondered this

[Saven Marek]

I've always wondered this. why have duplication of a function in a server across every single server box when it could all be done in the environment. For example all servers get electricity from the server room and all servers get network from the server room so why not all servers get cooling from 10F cooling in the server room.

It makes sense!

Re:well I've always wondered this

[green pizza]

Blade servers are a noble start. Less duplication of power supplies and network gear. I imagine the situation will continue to get better over time.

Duplication is nice in some respects, more redundancy is a big plus. That and you actually have several useful machines when you finally tear it all down. Who's going to buy 3 blades off ebay when they can't afford the backplane to plug 'em into?

Re:well I've always wondered this

[El_Servas]

Duplication is nice in some respects, more redundancy is a big plus.

definitely. More redundancy is more redundant.

Re:well I've always wondered this

[Bastian]

What makes sense is what solution is cheap and reliable.

On one hand, it should take much less juice to cool just a few boxen directly rather than keep an entire room cold enough to keep all those CPU cores cool as well.

On the other hand, high-performance cooling systems inside every box means a lot more points of failure.

On the other hand, if everything in your server room requires a working HVAC to function, you're in trouble if the HVAC goes out - while if the cooling system in one server goes out, you can just swap it with a backup server while you're waiting for repairs.

I'm sure there's a whole lot more pro/con that I could parrot if I had bothered to RTFA. . .

Re:well I've always wondered this

[T-Ranger]

What I have never understood is why servers virtually always have AC power supplies. Yes, you can get NEBS(?) compliant servers that take DC, but this isnt really a general option, but a distinct model line compleatly.

UPSs take AC, turn it to DC, charge their batteries. A sepearate system takes DC from batteries, inverts it and sends out AC. (Good UPSs, anyway. Otherwise they are "battery backups", not uninteruptable) Computer power supplies take AC and distribute DC inside the case. WTF?

Why doesn't APC start selling ATX power supplies? Directly swap out AC powersupplies, have them plug into the DC providing UPS and/or per-rack (or even per-room) powersupplies.

Electrical codes are a BS excuse. Even if you needed verdor specific racks, a DC providing rack is, so far as the fire marshal should be concerned, just a very large blade enclosure, which are clearly acceptable.

I cant beleive that Im the first one to ever come up with this idea. So there must be some problem with it.... Some EE want to explain why this wouldnt work?

Re:well I've always wondered this

[Anonymous Coward]

Shut up, troll.

Re:well I've always wondered this

[MindStalker]

it would work fine, except that you still have to convert the dc into different DC power voltages for the different devices. Another problem also is single point of failure, if your single AC to DC converter failed everything would go down. And your battery backup infact does not convert AC to DC then back again, it has two seperate paths a direct AC path then another path for AC to DC, yes it does have a DC to AC converter, but that is only used during power failure.

Re:well I've always wondered this

[david.given]

Some EE want to explain why this wouldnt work?

I'm not an EE, but it's something I've always wondered about. I don't have a datacentre, but I do have far too many computers: why does my machine room contain about fifteen wall warts, all producing slightly different DC voltages and plugged in to their various appliances via fifteen different non-standard connectors? Why not just have one low-voltage standard and have all these things plug into that?

One possible reason is that (IIRC) power losses vary according to current, not voltage. By increasing the voltage, you can push the same amount of energy down a wire using a smaller current, which limits losses. This is why power lines use very high voltages.

This means that if you produce regulated 5V at one side of your datacentre, by the time it's reached the other side it's not 5V any more. But it should be easy to get round this by producing 6V and having DC regulators; they're very small and extremely efficient these days.

However, I suspect that the main reason why this kind of thing isn't done is inertia. There's so much infrastructure in place for dealing with high-voltage AC supplies that you wouldn't get off the ground using DC.

Re:well I've always wondered this

[Steven Gray (Pulse U]

Data centers function on commodity hardware, and at the end of the day the cost of cooling the room would be bearable, however the cost of replacing all the cooling systems on the servers would not, and would need varied solutions for the thousands of types of boards likely to be in service. It's not a homogenous environment by any stretch of the mind. Additionally, this whole thing makes the bad assumption that just by cooling the room, you cool the components that need it. Hard disks dont function at low temperatures, they' designed for work in typical computing environments. Whats also to say that airflow across critical components like ram and CPU would be sufficient to dissipate the heat at the required rate anyway, you'd still need fans obviously. So, trading in your PSU fans, CPU fans for a room heating system, a hard disk warmer, protection for the ram and PSU, plus fans to circulate the air at a high rate through the case, does not seem like that much of a good idea.

Re:well I've always wondered this

[drinkypoo]

When you use DC power in a data center you run a high voltage (24V, 48V, depending on equipment) and then it's regulated down to all the usual voltages. However, it's a lot easier to transmit AC power over distances. It's not exceptionally lossy to convert AC to DC or vice versa any more.

Re:well I've always wondered this

[vidarlo]

UPSs take AC, turn it to DC, charge their batteries. A sepearate system takes DC from batteries, inverts it and sends out AC. (Good UPSs, anyway. Otherwise they are "battery backups", not uninteruptable) Computer power supplies take AC and distribute DC inside the case. WTF? Why doesn't APC start selling ATX power supplies? Directly swap out AC powersupplies, have them plug into the DC providing UPS and/or per-rack (or even per-room) powersupplies.

Really interesting tought. I've seen some of this (i.e 3com SuperStack III used a _proprietary_ DC solution), but all had their faults. The drawback of using a single huge PSU would be point of failure. It could bring down the entire data center. On the other hand, combining multiple DC sources to drive one unit is trivial. And a single huge transformer is more efficient then 100 small ones... So it'd not only reduce heating, but also reduce power bils, and maybe even decrease cost of servers, since it would not need to include it in the cabinet, which would free up space (in a 1U that means 1-2 extra disks). Heck, no need to have the PSU in the same room. Could have a room/closet for PSU's, which would need as good cooling and humidity stability as the server room.

Re:well I've always wondered this

[windex]

We use -48VDC, and it's a pain in the ass to find power supplies for modern hardware.

Whenever we need something outside of normal ATX, we wind up paying custom development fees.

No one makes DC to DC power supplies that are worth a damn, and the few vendors who do sell them (Sun, IBM, etc) charge an arm and a leg above and beyond what we pay to have them custom engineered.

Re:well I've always wondered this

[birdman17]

why does my machine room contain about fifteen wall warts, all producing slightly different DC voltages

This is a very good question. I recently analyzed my collection of wall warts to see whether I could replace any of them with connections to my outboard ATX power supply, which is powering my audio mixer anyhow. One of the wall warts produce AC (!), and of the other 4, 3 of them produce very odd voltages: 18 V, 15.6V, and 9V. No luck there. But one did produce 5V (for my Netgear switch), so I replaced it with a cable terminated in an ATX connector. Presto - one less wall wart! The others would need a small array of DC-DC convertors/regulators, which I haven't assembled yet. A project for another day. Not counting the AC-producing wall wart... not sure what's up with that one.

Then there's the whole question of the UPS and the inefficient/redundant DC-AC-DC conversions between it and the computer power supplies... argh!

Re:well I've always wondered this

[standbypowerguy]

Because DC can't be transformed easily, like AC. Distributing DC at low voltages requires higher current to achieve the same power (kW), thus a significantly larger wire size. Distributing DC at higher voltages is also inefficient, each end use device would require a DC/DC converter to convert to the lower voltages. The name DC/DC converter is a misnomer, most of them use high frequency sampling (AC) as an intermediate step.

Re:well I've always wondered this

[non-poster]

What I have never understood is why servers virtually always have AC power supplies.

I've wondered that, too. Every time the power is coverted between AC, DC, and the voltage level, there is some loss, so it's less efficient to do all of these conversions. I think having a UPS-oriented power supply would be a Good Thing, where you can hook up some external battery pack for the backup.

At a previous job, we used some Unix machines that were completely fault tolerant, including backup processors, backup network cards, and backup power supplies with batteries. Yes, the enclosure had 2 power supplies, with batteries in the bottom of the enclosure. It was a monster. Then again, it cost $50k-$100k for a couple of hundred MHz, but it ran forever. Or until the software died.

Re:well I've always wondered this

[warpSpeed]

if your single AC to DC converter failed everything would go down

Assuming that you only have one converter. The nice thing about AC to DC conversion is you can have multiple AC converters all feeding the same DC voltage to a single set of conductors to run the DC power out to the machines. The converters can even be out of phase. If the power conversion system is designed right, any one or two converters can fail, be disconnected from the power feed, and the remaining good converters will pick up the slack.

Re:well I've always wondered this

[FireFury03]

What I have never understood is why servers virtually always have AC power supplies.

For low voltages I don't see any problem with DC but AFAICR at higher voltages DC is more dangerous - a shock from an AC supply causes you to let go quickly, a shock from a DC supply (ISTR) causes the muscles in your hand to contract so that you can't let go.

However, these days we have so many low voltage DC systems (even in homes) that running a 12 or 18v DC supply around your office/home/datacentre sounds like a good idea. You still have to convert it to the voltages you need (usually 12v, 5v, 3.3v, and maybe a few others) but I can't help thinking that building a DC-DC converter for these low voltages would probably be cheaper and easier than a full 240v AC switched mode PSU for each device. (low power devices can even get away with using cheapo linear regulators).

Of course I'd still like some power regulation in each device since I don't want a power spike in the low voltage circuit blowing every device.

Re:well I've always wondered this

[Detritus]

Due to the insane current and voltage regulation requirements of today's motherboards, the power supply for the CPU and associated chips has to be physically close and tightly integrated into the motherboard. You can't just pipe in regulated DC voltages from an external power supply directly to the chips on the motherboard. In your typical PC, the power supply (the metal box) provides bulk regulated DC power. Some stuff can run directly from the power supply. Components with demanding power requirements, like the CPU, are powered by dc-to-dc converters on the motherboard. These take DC power from the power supply, convert it to high-frequency AC, and back to regulated DC.

The general rule is that stricter requirements for power supply performance can only be met by decreasing the physical distance between the power supply and the load. The trend towards lower supply voltages and higher currents makes the problem worse.

AC power wiring is cheap and well understood. It doesn't require huge buss bars or custom components. It is the most economical way to distribute electrical energy.

Once you reach the box level, you want to convert the AC to low-voltage DC. Confining the high-voltage AC to the power supply means that the rest of the box doesn't have to deal with the electrical safety issues associated with high-voltage AC. The wiring between the power supply and load is short enough to provide decent quality DC power at a reasonable cost. Those components that require higher quality power can use the DC power from the power supply as the energy source for local dc-to-dc converters.

You could feed the box with -48 VDC like the telephone company does with its hardware. You would still end up with about the same amount of hardware inside the box to provide all of the regulated DC voltages needed to make it work. Cost would increase because of the lower production volumes associated with non-standard power supplies.

In the end, it boils down to economics. DC power distribution costs more money and it doesn't meet the performance requirements of modern hardware. The days of racks full of relays, powered directly from battery banks, are long gone.

Re:well I've always wondered this

[Roadkills-R-Us]

It's partially inertia as someone else noted, but there are other factors.

For one, it's a microcosm of the generalized AC distribution concept. If you want to run X watts through a wire, you can run it at low voltage and high current, which by ohms law insures a hefty voltage drop (WRT the supply voltage), or you can run it at high voltage and low current, which by ohms law means a lower voltage drop, which means it's essentially unchanged. Then you drop it back down at the distribution point (just like they do with substations and transformers near your home or business). This voltage transformation is trivial with AC, but much (perhaps insanely) more complex and expensive with DC.

Which brings up the cost issue. Partially this is also inertia-driven, because the infrastructure is there to produce and transform AC voltages, and to produce equipment that uses it. But there are hidden costs as well. The wires inside a computer to distribute +/-5VDC and +/-12VDC don't have to be that large, because they are short. But you'd either have to run a bunch of wires per computer, or run much thicker wires to distribution points, to handle all that current. Go read a power supply comparison with today's fast chips or high end disk drives to get an idea of the current requirements. A DC panel to run a 6 foot rack would be capable of starting a large, diesel truck or three!

Even at low voltages, high current DC carries its own set of dangers. You'd have to do a lot of education to get people to be careful around it; far too many people would think it's as safe as handling a battery. At least until they were hit in the face with molten metal as they accidentally arc-welded something.

Finally, someone in this thread stated that if you are running off AC and have to switch to batteries, it's not a UPS. You could make a case that, technically, this is true, but in fact, so long as the UPS switches faster than the computer power supllies run out of stored power, it's "uninterruptible enough". The real issue for most of us isn't whether the AC power is technically interrupted, but whether the computers keep running. If your servers' power supplies can't ride through a single, 50Hz or 60Hz cycle problem, you're running too close to the edge.

Re:well I've always wondered this

[Agripa]

Most of these small systems have internal switching regulators anyway and depending on the design should run anywhere from say 8 to 12 Volts. If they are labled as using AC instead of DC, they have an internal bridge rectifier preceeding the voltage regulator and can almost certainly run on 1.4 x the listed votage in DC although there are exceptions where they need to generate a negative voltage for things like RS-232. Older external modems generally have this problem.

Designing all small devices to use one standard input voltage is definately possible. 12V DC would probably be the best choice although 48V DC might be better except for cost.

Re:well I've always wondered this

[Agripa]

Internally, small offline switching power supplies take the 120 volts AC, voltage double it, and rectify it to 340 volts DC. That makes it easy to make dual input power supplies that also accept 240 volts AC which can be directly rectified to 340 volts DC. All of them should accept 340 volts DC with no problem.

The larger power supplies have power factor correction of various types and designs. They should also have no problem with high voltage DC but it would be implementation dependent.

340 volts DC is not something I would want floating around in the same way 120 volts AC is available but as other posters have pointed out, there already exists a 48 volt telecom infrastructure that was designed pretty much for this very problem. Unfortunately, 48 volts DC does not mesh well with 120V/240V AC so dual purpose power supplies are unlikely.

Re:well I've always wondered this

[Anonymous Coward]

Yes, you can get NEBS(?) compliant servers that take DC, but this isnt really a general option, but a distinct model line compleatly.

Dell (and others) offer 48V DC power supplies for most of their rackmount servers.

UPSs take AC, turn it to DC, charge their batteries. A sepearate system takes DC from batteries, inverts it and sends out AC. (Good UPSs, anyway. Otherwise they are "battery backups", not uninteruptable) Computer power supplies take AC and distribute DC inside the case. WTF?

If you need to go any significant distance with low voltage wires, you need fat copper wire to avoid unacceptable voltage drops.

Why doesn't APC start selling ATX power supplies?

The market is very small. If you want a DC powersupply, it's probably for a rackmount server that doesn't use ATX, and you probably would have bought it with 48V DC in the first place.

While 48V DC UPSes are cheaper & simpler than 120V AC, the heat generated by the UPS won't vary much.

Re:well I've always wondered this

[anakin876]

is that AC wall wart for a Shure wireless mic or something?

Re:well I've always wondered this

[dontbgay]

Actually, with AC voltage, when voltage increases current decreases. They can pump much higher voltages to a transformer and step it down. The exact opposite is true for DC current. IANAEE.. but i play one for the military.

Re:well I've always wondered this

[ansaari]

DC transmission can be more efficient. In Northern Quebec, Canada, there is a power development that converts the output from the generators to high voltage DC for the long distance transmission. With AC, the capacitance between the ground and the wires becomes appreciable, with DC the 'capacitor' charges up once, and no energy is lost in the constant recharging (with its attendant losses). However, you're right that the end circuitry is quite a bit more complex.

More details at:

http://www.hydroquebec.com/transenergie/en/reseau/ caracteristiques.html

Re:well I've always wondered this

[Tassach]

Amen to that! It's always boggled my mind why every single electronic gizmo on my desk needs a wall wart running at a different voltage with it's own unique connector. It would make SO much sense to have a standardized connector which provides a common set of voltages.

If everyone in the industry can agree to make internal PC components (Hard drives, CD/DVD rom drives, etC) that use the standard ATX 4-pin molex connector, why the hell can't the industry agree on a standard for small external devices too?

Hell, even a single VENDOR can't make up it's mind what to use -- I've got 3 Linksys devices sitting on my desk (router, WAP, and switch) and they all use mutually incompatible power supplies. You'd think that they'd standardize on a single PS type unit to streamline inventory and overhead.

Re:well I've always wondered this

[kilodelta]

When you consider that circuit switched phone carriers have been doing this for years you then understand that the fire codes are baloney.

Phone switching and networking gear use -48VDC. If you've ever gotten to tour a switching facility the battery area is a treat.

Many of those fire codes were developed during the intial roll out of electrical power. More than likely much of the resistance to it being widespread is because with DC, the current is constant, there isn't even a fraction of a break. AC at least gives you a very short period of time to 'break' from the connection.

Good design and practices would elminate the perceived dangers but I still think you'd catch a lot of static for it from authorities.

Re:well I've always wondered this

[Anonymous Coward]

I think it's a good idea but you should also be aware of the current demands of a modern PC. If your core is drawing 100W at 3.3V that's over 30A per CPU. Even at 12V that's still close to 10A. A decent server room might well need several 300A busses and would probably still need some circuitry to replace the PSU at each CPU (isolation, conditioning and protection).

If you've never had the opportunity to see a telcom -48VDC system, you should try. They're pretty impressive beasts.

Re:well I've always wondered this

[birdman17]

is that AC wall wart for a Shure wireless mic or something?

No, nothing that fancy, it's just for my DSL modem - a basic SpeedStream 5360. I guess the modem might need AC power for its telephone line interface circuitry, or something.

Re:well I've always wondered this

[T-Ranger]

I think your missing the point of my question/discussion. The question is why is it a pain in the ass to find DC power supplies?

Re:well I've always wondered this

[drinkypoo]

IT costs more to provide more voltages in the wall wart, so you produce the highest voltage you need to reliably generate the voltage you actually wanted inside your machine, and generate all the other voltages from that. What you suggest has already happened, but only for a short range of voltages, and it is available at radio shack. For higher-voltage applications, it's just not in the cards. Personally I would just be happy if manufacturers would generally stick to one voltage and plug across their products. Why are some HP printers 28V, and some 32V? Printers that were sold concurrently, I might add. It would make much more sense to just use 32V for all of them to begin with, because you have less loss and can use a smaller wire from wart to printer, which is also cheaper.

Re:well I've always wondered this

[lcsjk]

Well, I am an EE and I have spent more years designing power supplies and backup systems than I care to talk about. As a power supply and system's designer, I also have spent a lot of time designing cooling systems at the board level and at the system level.

1. The wall warts are there so that international safety codes can be met easily. They provide the necessary safety isolation between the 120/220 volts from the wall and the device they are powering. It takes a lot of time , money and testing to meet these international safety standards, and if you build the power supply inside your equipment, then you deal with having high voltage inside your enclosure, and a nightmare of conditions you have to meet and test.

2. Once you have selected a wall wart to provide the "safety low voltage" of less than 40.2 volts peak, you can re-regulate to lower voltages as needed. And since many of these wall warts use switching technology, the efficiency can be quite high (80-90 percent). Those that use only a transformer for safety isolation have 95-99 percent effeciency. Yes, one large wall wart could do the job of the multi set of small ones, but that would mean someone would have to design a custom power supply for those products, and that gets back to the original problem

3. The computer power supply provides this same function. That is why it is sealed, and has only low voltage outputs. Even the power entry plug and fuse is part of the power supply. Look on the label and you will see all kinds of international safety logos. The processor power of 3.3 volts and other similar voltages is conditioned directly at the processor since there are unique design problems that occur as a processor changes from 100 percent usage to sleep mode.

Addressing the cooling issue: (I will try to use simple explanations for those who do not design in this area.)

For all ICs, the faster it goes, the hotter it gets. If a processor can be slowed or stopped when it is not doing anything, the power can be greatly decreased. However, as the article states, most people prefer to get better performance instead of being concerned with heat.

The problem is the same whether you are discussing the silicon chip inside a semiconductor package or discussing the server box inside a room. The power generated by the chip has to be removed from the room to avoid overheating. Assume a processor chip dissipates 200 watts. The chip designer has to find a way to move 200 watts from the silicon chip to the metal or ceramic surface of the package. The board designer has to find a way to move the 200 watts from the package surface to a heat sink. The system (box) designer has to find a way to move the 200 watts from the heatsink to the surrounding air and get the heat out of the box and into the room air. The building designer has to find a way to move the 200 watts from the room to the outside. As you can see, it is not a matter of room cooling vs processor cooling. Each person in the design sequence has to remove heat. The only person who can really make a difference is the processor designer who can shut down to lower power at any time the full speed is not needed. Secondarily, the programmers can tell the processor when the program needs only slow speed, such as when one is typing or filling in values on a spread sheet or database and thus decrease the initial power. However, even if the processor is running slow speed, the other designers in the heat chain still have to design for maximum heat removal unless the processor can tell the rest of the system its actual power used.
Now, give yourself 50 of these processors in one large room and you are generating 10,000 watts of heat. That's like having seven of the 1500 watt bathroom heaters running continuously.
Since your overall power supply is only 80 percent efficient, the total power jumps up to 12000 watts. Now add in the power for the hard drives, fans, monitors, etc and you start getting really warm air. That's how it works! Now how do you solve it?

Re:well I've always wondered this

[skintigh2]

Well, it may not be 5V at the other side of the data center I doubt it would be much less than 4.99V. Besides, it's not like the power supply in the computer is accurate to more than a few percent.

I have often wondered how much electricity would be saved if you removed all those 60%-80% efficient (82% at high load is the best I have ever seen - low load is much worse) power supplies and the two conversions in a UPS and replaced it with one industrial unit. You could even locate the unit outdoors and cut down a huge chunk of your cooling bill. Another poster mentioned some companies do convert their datacenters to DC, but I have never seen it.

And have you ever looked at how many adapters you have around your house? They are still using power even if nothing is hooked up or turned on. It's called a "phantom load." As electronics become more popular along with trendy low-voltage (DC) lighting, it may actually become economical to have a single converter in your house and a DC jack for all your gadgets, phones, lights, computers, LCD monitors, etc.

All that would require is every industry on Earth working together and deciding on one voltage and one standard jack, how hard could that be?

Re:well I've always wondered this

[Richy_T]

However, most everything will/should be internally regulated anyway and it's quite possible your 9V item would be quite happily powereed from the 12V supply. Not that I'm recommending trying it :D

Rich

Re:well I've always wondered this

[Phroggy]

If you're doing custom engineering anyway, maybe you should look into mass-producing them to undercut the competition?

Re:well I've always wondered this

[Infinite Entropy]

AC to DC switching power supplies are generally 80 to 90 percent efficient. I don't think it would be hard to get DC transmission more efficient than that.

Re:well I've always wondered this

[Anonymous Coward]

..this wasnt flamebait...but it could be Redundant..

Re:well I've always wondered this

[njh]

8GA cable (heavy car cable) 0.00252ohms/m
assume 10 m between powersupply and computer, 200W power gives 2V drop. So you're wasting about 30% of the input power.

There's a reason that cars are moving to 42V!

250V DC makes more sense technically, but it is far more lethal than 250V AC.

Re:well I've always wondered this

[njh]

Just be careful of ground loops - the device in question may rely on the independant supply to avoid noise problems. Ethernet is fine, but usb, audio and firewire are not isolated and make have trouble. High efficiency wall warts are quite doable, there just isn't any demand. Personally I would like the power block separate from the plug (so they don't get in each other's way) and designed to stack or clip together.

Re:well I've always wondered this

[njh]

Well you need to go back to school. Power is volts*amps (ignoring things like PF for now) for both AC and DC. The only problem was that DC transformers are very hard to build. But we don't care any more because we can build efficient DC->AC generators in each power supply, and tune the AC for conversion, rather than transmission. DC is superior to AC for transmission (hence the high voltage interconnects).

Higher voltages mean lower current for both AC and DC.

Re:well I've always wondered this

[njh]

The same is true of AC. The technology is called 'grid interactive', and we should be developing it more as it allows for things like co-generation and distributed generation.

You can buy solar panels with a single, standard wall plug which, when you plug in starts pumping energy back into the grid. Perfectly safe too - you can unplug it and stick your tongue on the plug!

Re:well I've always wondered this

[dbIII]

When your UPS dies you can still run AC power in from somewhere else, or if you need to run something on a bench in another room etc. If something relies on a specific infrastructure you need a backup of that infrastructure - in this case other DC sockets somewhere. If you lay out the capital cost to do that you can then save on losses from converting the power mutiple times.

Your UPS will die someday, batteries don't last very long before they need to be replaced.

Re:well I've always wondered this

[inKubus]

The chip designer has to find a way to move 200 watts from the silicon chip to the metal or ceramic surface of the package. The board designer has to find a way to move the 200 watts from the package surface to a heat sink. The system (box) designer has to find a way to move the 200 watts from the heatsink to the surrounding air and get the heat out of the box and into the room air. The building designer has to find a way to move the 200 watts from the room to the outside.

Then the air outside has to find away to absorb the 200 watts of heat by moving up into the clouds. Water in the air comes with it and condenses and it falls as a drop of rain into the lake; the lake runs down thru the dam which spins the turbines, which spins the generators, which energize the tranmission lines, which brings 200 watts to the building--

Rinse and repeat. Ahh, there.

Re:well I've always wondered this

[devilspgd]

Yes and no. Redundancy is fine if the machines are actually redundant.

However, if you give me an option between having a server with a single power supply or a shared power supply between a dozen serves, all other factors being equal, I'll go for the shared one every time.

Why?

Well, if something goes wrong, they have 12x the incentive to fix it *now*

Re:well I've always wondered this

[devilspgd]

Any idea if these panels are cost effective (vs the cost of the panel itself)?

Re:well I've always wondered this

[devilspgd]

Sure, but if they could settle on a couple standard voltages (5V and 12V anybody?), you could buy one power supply and connect it to all your third party junk.

It would even let manufacturers cheap out by not providing a supply at all, so you'd think they'd be all over this.

Re:well I've always wondered this

[devilspgd]

What do you think will be more reliable, a thousand $50 power supplies (one in each server), or $50,000 spent on building a redundant power solution to power the entire data center?

You're already building around a number of potential single points of failure (Everything from the power in from the grid to the UPS(es) to the wire from the UPS to each servers' UPS)

Re:well I've always wondered this

[devilspgd]

Sure but you can have multiple UPSes, plus a backup transformer that will convert AC from the grid to DC.

Re:well I've always wondered this

[Idarubicin]

For low voltages I don't see any problem with DC but AFAICR at higher voltages DC is more dangerous - a shock from an AC supply causes you to let go quickly, a shock from a DC supply (ISTR) causes the muscles in your hand to contract so that you can't let go.

That's partly true. An electric current passing through a muscle will cause it to contract. Since AC passes through zero voltage many times per second (120 in North America) there's a brief interval where the muscle may relax and the person may be able to let go--or at least his grip will get limp enough that he'll fall off the wire. DC's constant flow of current will tend to prevent a relaxation of the muscles--there will be no opportunity to release.

On the other hand, AC is much more likely to cause fibrillation of the heart. All those little pulses of contraction seem to be much better suited to causing the heart to flutter--not in a good way--compared to the steady paralysis of DC shock. If you kill the power quickly, then a heart that's received a DC shock stands a substantially better chance of spontaneously resuming a normal beat.

Now for an anecdote--I don't recommend trying this at home. I had a physics teacher who once told me that if I had to check to see if a wire was live using my own hands, I should use the back of my hand to do it. Just brush lightly. If there's a current flow, then the induced contraction of muscles in the hand and arm should pull the arm away from the wire...instead of tightly clenching the cable, which is what happens when reaching in the usual way.

Re:well I've always wondered this

[njh]

They certainly are for small numbers of panels, which is what interested me for a remote power system. For larger systems I expect there is a break point where you are better off with a larger inverter. The big advantages of per cell inverters are: that you can scale the system as you grow; panels don't have to be located near each other, only near existing wiring; shaded cells don't drag the whole string down; the inverter can be sized optimally for the panel; and the inverter can implement peak power point tracking more effectively. I expect that being closer to mobile/cell phone tech with the lower voltages would be helpful.

Ideally we would get the tech to the point where each cell (or cell stack) generated its own 400V DC output, these could then be collected and fed into the grid at the panel level. It may be possible to build a significant port of the circuit in the same die as the panel (may also be a bad idea - IANASE).

I expect that with higher production rates the cost of the inverter would disappear in the noise (a cheap 100W inverter costs $20 here, compared with around $1k for a 100W panel)

Re:well I've always wondered this

[drinkypoo]

Modern electronics need higher voltages to avoid a costly wall wart, because the devices themselves use more than 12V, especially inkjet printers. (Laser printers consume enough power during fusing to need a real power supply, but they also use high voltages.) You would need 3, 3.3, 5, 7, 9, 12, etc etc because manufacturers would like to make their devices as simple as possible and adding a DC-DC power supply that can run off a range of voltages is simple and cheap but not so cheap across umpteen zillion units.

Re:well I've always wondered this

[devilspgd]

The vast majority of my bricks are between 5V and 12V, many being either 5V or 9V.

Some devices obviously couldn't use a central power supply, printers being a good example. Fine. None of my printers have a brick anyway, my laser printers both use standard PC power cables and my inkjet has an external power supply that takes a PC power cable input, so it doesn't block an entire power bar.

It's not the end of the world if a couple devices still have special needs, but why should I need a seperate brick for each device which fits into the common range?

As much as DC-DC power supplies might raise the price slightly, it would be offset by the back that the manufacturer doesn't need to supply a brick.

I'd be more then willing to spend the $150 I just spent on a computer power supply on a magic box that would provide everything from from 3v to 12v with standardized cables if it would remove the rats next of of power bars with 2-3 items connected to each.

Heck, if they'd just standardize on one size of plug for each voltage, those of us who want to invest in a central power supply could do so. Those that don't could stay with the manufacturer supplied bricks.

Re:well I've always wondered this

[plover]

I'd be more then willing to spend the $150 I just spent on a computer power supply on a magic box that would provide everything from from 3v to 12v with standardized cables if it would remove the rats next of of power bars with 2-3 items connected to each.

I'd rather have USB commercially standardize on the higher voltage and power option that IBM developed for cash register systems. +5/+12 VDC (and even a single +24VDC station for a printer) VDC, 1000mA (or more). What's better is the plugs are backwards compatible with USB 1.0 standards (they simply have a second socket mounted above the first, and it contains the power conductors in a ground-first connector design.) Then I wouldn't have any wall warts or power cables at all -- I'd draw all of my many peripherals power from the PC's power supply.

A quick glance under my desk shows at least 12(!) wall warts and power bricks, not counting any devices that take 120 VAC directly, such as the computer and monitor. That's 12 power cables that I'd be happy to do without, and I could ditch three power strips.

Yup, over half these stupid USB peripherals take wall warts. The hubs take wall warts. The printers and scanners, tuner and speakers, cradle and charging station -- all those warts and wires would just be gone. That would be a dream. And the technology is here, right now -- but nobody's using it yet. Bummer.

Re:well I've always wondered this

[devilspgd]

In theory, I agree. In practice, I'm be afraid of sticking too many devices on USB power simply because I don't trust cheap motherboard manufacturers to be able to handle the load. There is also a significant risk of a cheaply made device shorting out and taking out the motherboard as well as every other connected device.

That being said, it does drive me nuts how many USB devices have their own brick supply too. In fact, other then my cordless phone, I think every brick attached to a power bar on the floor beside my PC is connected to a device connected via USB to my system.

One other concern with having USB supplied power -- This isn't practical for all devices, many devices need power when the system is off too. My cordless mouse (rechargable), Palm, PPC and digital camera all would need power when the system is off. I'd be annoyed if my SB Extigy turned off with the system too, it's nice having it passthrough audio from line-in to my speakers even when my PC is off.

Also consider that even if power was available when the system is off, many devices really should turn themselves off when the system is off. It would be nice to cut power to these devices, but that would mean adding yet another pin (one for always-on power, one for switched power)

Now admittedly many systems (mine included), still provide USB power when the system is off. However, looking at my power supply's load limits, the power supplied when the power supply is off is minimal, and the hardware mentioned earlier would go over my power supply's standby maximums.

This could be resolved as part of the specification by requiring another connector from the power supply to the motherboard (to show the motherboard that the power supply is capable of supplying higher demand then the current ATX standby power). If this new connector wasn't connected then the motherboard would only supply power via USB when the system is running. As a workaround, some external USB hubs could have power connectors that would supply downstream devices with power even when the system is off.

All of this is a lot more hassle then standardizing the connections used by bricks connecting to external hardware. If each connector size corresponded to a specific voltage and minimum amp rating, bricks would become interchangable, and it would become feasible to buy a single power supply that would feed all of your power hungry gadgets without dinking around with sizing the power connector.

Re:well I've always wondered this

[plover]

If you're willing to adapt to a different standard of power (a single brick-style connector), why are you unwilling to assume that a PC power supply isn't similarly adaptable?

There is already USB support for power signalling. Many of today's peripherals shut themselves off when the system is powering down, even though they're attached through a powered USB hub (especially true of POS devices such as barcode scanners and mag stripe readers.) "Extra pins" are not required.

It's completely possible to make this change without requiring motherboard changes or software in the PC or BIOS. First, remember that the "Powered USB" connectors have additional hardware requirements (new shell shape) and therefore those connectors also have additional cable requirements (to deliver the power.) Now, imagine the power supply delivering those new wires also being an upstream hub on the USB bus. It could watch the downstream "current requirement" messages, and deliver the requested amount of current -- standby or operational.

As a matter of fact this exact same functionality could be added to an existing PC (with no internal changes) simply by adding an externally powered hub that provides the new Powered USB connectors. It would require its own power supply (of course) but it could provide all the power with a single transformer, rather than a dozen individual wall warts. Think about that: an 8-port hub you just add to your current machine that delivers power to Powered USB devices. That would be one way to drive the part to market in advance of Powered USB peripherals becoming commonly available.

I'm a big fan of the "one wire" solution. Anything that means I don't have to find another power strip, or cable-tie another handful of black wire makes it that much cleaner, and that much simpler for everyone. While a "standardized brick" makes a lot of sense, I'd jump on the chance to go with a "no brick at all" solution.

Cooler Rooms

[forlornhope]

I like cooler rooms. Especially for a large number of servers. Its more efficient.

Re:Cooler Rooms

[Thrakkerzog]

That is, until the AC breaks and all of the servers melt into slag.

Re:Cooler Rooms

[Valdrax]

By what metric? If you choose power and cooling costs, then cooler servers wins. If you have servers that consume less power, you don't have to pay to cool them back down. It seems that cooler machines would be more efficient. Furthermore, targetted, specialized cooling solutions can be made more power efficient than general purpose room air-cooling solutions.

Re:Cooler Rooms

[ePhil_One]

That is, until the AC breaks and all of the servers melt into slag.

Thats why we actually have 2 A/C units working in parallel, one goes down the other stays up. Unfortunately I know from experience a failed server room will stabilize around 140F once the A/C goes out, which will cook some equipment. You know there's a problem when the walls to your server room are warm to the touch.

Re:Cooler Rooms

[Thrakkerzog]

We actually had both AC units crap out in the same afternoon. Really hot day. :-)

Re:Cooler Rooms

[ePhil_One]

We actually had both AC units crap out in the same afternoon. Really hot day. :-)

Strangely, our problem was triggered by an especially cold day, the night cold triggered a second mode for the condenser that was meant to prevent "overcooling", instead they shut down entirely. Took a few visits to fix too because when they arrived it was warm enough the mode was enabled, and the system thus worked fine...

Err, well, both?

[Pants75]

The servers are the heat source and the cool room air it the cooling mechanism? Yes?

So take your pick. To make the servers cooler, either buy new more efficient servers or buy a whacking great air con unit.

Since the servers are the things that actually do the work, I'd just get feck off servers and a feck off air-con unit to keep it all happy!

Everyones a winner!

Re:Err, well, both?

[kfg]

"Everyones a winner!"

Indeed. It's your treat.

Think about it.

KFG

Re:Err, well, both?

[Pants75]

I'll take 10!

If you already have either the machines or the air con, and can't afford to replace the existing items, then that entirely dictates which side of this equation is your you.

Re:Err, well, both?

[FireFury03]

The servers are the heat source and the cool room air it the cooling mechanism? Yes?

Alternatively you could water cool all the servers through a central water cooling system for the entire datacentre - Then use a central (set of) heat pumps to cool that heated water down again. Or take cues from the power stations and cool the water by evaporation in cooling towers. Sounds more efficient than cooling entire (probably badly insulated) rooms down anyway.

Cray still has water cooling!

[green pizza]

Unlike most companies that are considering going back to water cooling, Cray has always used water cooling for their big iron. In fact, the only air cooled Crays are the lower end or smaller configured systems.

All hail the Cray X1E !

Re:Cray still has water cooling!

[1zenerdiode]

I thought Cray's originally used Fluorinert(tm), which is definitely *not* water... *spark* *spark* *fizzle*

All hail non-conductive fluorochemicals!

Re:Cray still has water cooling!

[green pizza]

You're right. I guess I should have said "liquid cooling".

Like the original Cray 1, the current Cray X1E uses Fluorinert to cool the system itself (check out the swanky video of the Fluorinert vapor jets on cray.com!). There's also a loop of some sort of refrigerant between the heat exchanger indoors and the cooling tower outdoors.

Re:Cray still has water cooling!

[timeOday]

How about distilled water?

Re:Cray still has water cooling!

[Anonymous Coward]

the problem is it ends up getting contaminated and starting to become a rather better conductor

Re:Cray still has water cooling!

[Anonymous Coward]

Corrosion

Re:Cray still has water cooling!

[drinkypoo]

You could solve this problem by using a reverse osmosis filter and a pump as part of the cooling system. Problem is, your pump has to produce 40 psi for the RO filter to work. Still, it would clean up the water nicely. It will only work for very large servers, but you don't get too much bigger than a cray (at least in a single package). I suspect the flourinert was used because it carried more heat.

Re:Cray still has water cooling!

[digidave]

OT: At the Toronto Science Centre there is a working television submerged in a clear fluorochemical. It gets a lot of awkward gazes by passers by who assume liquids should cause sparks/shorts.

Hell, even though I know what a fluorochemical is I still think it's very cool :) And a great way to keep dust off the TV!

Re:Cray still has water cooling!

[Detritus]

You need a water treatment plant to keep it clean and pure. They do this for water-cooled transmitter tubes in high-power radio transmitters, like those used in shortwave broadcasting.

Re:Cray still has water cooling!

[Anonymous Coward]

Not quite. It actually has somewhat worse heat conductance and capacity than water.

The problem: yes, you can keep water clean. But you won't prevent it from dissolving the computer...

This is because water is an excellent solvent for almost all polar substances and mediocre for others. Perfluorocarbons (Fluorinert(tm)) are great solvents for fluorinated polymers (Teflon(tm) is reported to soak in PFCs, which is something quite spectacular as Teflon doesn't normally soak up anything). And they are worthless solvents for almost everything else.

When your machines have to work for a long time, you don't want them dissolving, right? Of course, PFCs have another problems: they are excellent solvents for oxygen :)

Re:Cray still has water cooling!

[Tassach]

It will only work for very large servers

You could easily have a standard for water/liquid cooling of standard rackmount gear. Really all you need to specify is standard dimensions for the intake and outlet connectors and a flow rate / pressure range -- after that, the specifics of the internal plumbing are up to the vendor. The external cooling system provides the pressure, flow, cooling, and filtering according to spec.

Re:Cray still has water cooling!

[drinkypoo]

deionized water is not corrosive. I suggest that the cooling system use water which is RO filtered or distilled to remove chlorine (the waste water can be used to run toilets) and then deionized and sent out to the heat sources. Heat returned to the system can be used to [help] run the distillation process.

Re:Cray still has water cooling!

[Anonymous Coward]

yeah until it gets out and starts mixing with dirt etc

and once its leaked out of the pipe system its no longer getting the frequent cleaning.

Virginia Tech's Super Mac Used....

[sammykrupa]

Lots of A/C. (more where that came from).

Both

[turtled]

I agree, both solutions would help. Our room is a nice cool 62.5. Best condtions to work in!

Cooler rooms also keep others out... we get a lot of, its so cold, and they leave. That's golden =)

Re:Both

[green pizza]

I agree, both solutions would help. Our room is a nice cool 62.5. Best condtions to work in!

We keep ours at 73 degrees, about 2 degrees warmer than the rest of the building. We did the 60 degree thing for awhile, but it required quite a bit more electricity to maintain that temp. The servers work fine at 80 degrees, but 73 is more comfortable and provides a little more cushion.

Re:Both

[Anonymous Coward]

Translation: We keep the room super cold in case any perky womenfolk come in, but they often take one look at our hideous geek visage and leave.

Re:Both

[cjmnews]

Computer rooms are not meant for human habitation.

The noise levels are above what is healthy, the temperatures are too cool, the air is too dry, and your not allowed to drink while in there.

Overall it is not the best place for someone to spend hours at a time.

I am currently letting management here know that spending the day in that environment is not acceptable to me. They are trying to remove the secure network access to the development environment, which I obviously oppose.

We'll see how it goes. Who knows maybe I'll transfer out.

Mabe a mix?

[burgeswe]

Right now I'm just starting up the IT department where I work, and I've noticed a substantial difference in merely moving the servers from one room to another. of course, my new it room is the old boiler overflow room, and I've heard it rumored that it used to fill up with steam... :S

Re:Mabe a mix?

[TykeClone]

I just moved my servers from a room near the boilers to a new server room with a dedicated air conditioner.

In the old room, I couldn't keep the servers cool in winter when the boilers were on. Summer wasn't so bad because I could leech off of the main air conditioners to get some extra cooling. I did have a small window unit air conditioner dumping heat from the small server room into a larger room, but it just couldn't keep up.

I guess that what I'm saying is "Good luck with that" :)

Re:Mabe a mix?

[burgeswe]

I came in a few weeks ago and found my office full of water from where the water main into the boiler had cracked and formed a small lake in my office.
"**laughs** oh, I know, we'll stick the IT guy in the lake"
you have any suggestions on how to better protect equipment in those type situations?

Re:Mabe a mix?

[TykeClone]

My old server room was separated from the boiler by an old foundation - no worries about boilers leaking. I had (and now have better!) shelving set up so that the servers were at least 1 foot off of the ground - it is very nice to be able to get around to the back side of the shelving to connect cables and such (I didn't have that in the old room, I do in the new room).

Re:Mabe a mix?

[jaavaaguru]

I have that problem in my room at the moment (it's about Spring here - still cold outside). I'm not looking forward to what it'll be in the summer.

I had a Playstation 2 in my rack for a while and it pushed the temp up by about 8 degrees C. Nasty.

Re:Mabe a mix?

[TykeClone]

In my old server room, the temperature problems were inversely related to the outside temperature. When it was cold outside, I couldn't cool the room because of its proximity to the boilers. When it was hot outside, the server room must have been close to the beginning of an AC run, because I got plenty of cool air when it was running.

The new server room has a dedicated AC unit and is working much, much better than the old room.

Re:Mabe a mix?

[jaavaaguru]

I was talking about my bedroom. This *is* Slashdot after all :-)

Re:Mabe a mix?

[TykeClone]

I was talking about my bedroom. This *is* Slashdot after all :-)

I was typing at work so was in the wrong frame of mind.

What about cold countries

[rescendent]

Does this mean Alaska, north Canada, Greenland and Antartica are soon going to be popular server centres? Saves on air con...

Re:What about cold countries

[Ironsides]

Does this mean Alaska, north Canada, Greenland and Antartica are soon going to be popular server centres? Saves on air con...

[sarcasm]But that would promote Global warming, the melting of the Polar Icecaps, the Greenland Glacier and other glaciers! [/sarcasm]

Seriously, I can honestly see someone arguing that point.

Re:What about cold countries

[rescendent]

I suppose you could have them water cooled as well and heat your installations water... Then your saving on your heating bills.

Re:What about cold countries

[LWATCDR]

Actually back in the Dot Com days some company was talking about putting a server center in Alaska. The reasons?
1. Cheap power. Use the abundant natural gas that is too expensive to ship out of Alaska.
2. Security. Hey It is in middle of freaking no where!
3. Yes Cheap cooling. Just vent it to the outside.
I do not think it ever took off because of the cost of connecting to the back bone. But it was an idea.
Maybe Inuvik will be the new server center capital? How much bandwidth do they have up there?

As a matter of fact here is the Slashdot story http://slashdot.org/article.pl?sid=01/05/14/159258 &tid=95

Re:What about cold countries

[denis-The-menace]

Alas no.
The price of electricity is not high enough for using the the cold outdoors to make chiled water during the winter and then switch back to refrigeration for summer.
But electicity prices are starting to go up...

Re:What about cold countries

[TykeClone]

Maybe Inuvik will be the new server center capital? How much bandwidth do they have up there?

How many backup tapes can you fit on a dogsled?

Re:What about cold countries

[Ironsides]

Maybe Inuvik will be the new server center capital? How much bandwidth do they have up there?

Having to have looked this up recently, almost none. lines are very expensive to most places in the state and I do not think any backbones go through there. They are generally limited in most places to modems and in some cases 128k DSL at best. This is from when I had to look up internet connections for some businesses up there, and may not reflect some places in Alaska.

Re:What about cold countries

[LWATCDR]

Inuvik is not in Alaska it is in Canada in the North West Territories. From what I have read they do have broadband :) Now do they have enough band width for a good slashdotting? Who knows?

Re:What about cold countries

[Ironsides]

Inuvik is not in Alaska it is in Canada in the North West Territories.

Sorry, since you mentioned Alaska earlier in the post, I thought you were still refering to Alaska. My bad.

Re:What about cold countries

[LWATCDR]

Sorry about that. The story was about setting up a data center in Alaska.
Actually Inuvik may be getting a big pipe soon in more ways than one. I hear that Canada is thinking of building a gas pipeline from Inuvik down to the US. It seems that there is a lot of gas and oil up there. When they run the pipeline they are planning on putting in a big fiber optic line. So Inuvik could have cheap power, cheap cooling, and a big fat pipe. The only things that would keep it from being a good place for a server farm is the cost of transportation, food, construction, and personal. That it is in the middle of FREAKING NOWHERE and is FROZEN SOLID a good part of the year :)

Re:What about cold countries

[Ironsides]

When they run the pipeline they are planning on putting in a big fiber optic line. So Inuvik could have cheap power, cheap cooling, and a big fat pipe. The only things that would keep it from being a good place for a server farm is the cost of transportation, food, construction, and personal.

Sounds like it would be a great place to put a datacenter for disaster recovery. Only problem is that since it is so far north, it won't have any access to the geostationary satelites (angle of sight to the dish above the horizon would be very small, if it could even see it). And yes, satelite access helps for some companies where it is cheaper to rent some uplink time (or temporarily repoint a dish) rather than use an OC-3 year round.

As for the fibre, if they do it hopefully the will put in a nice ig trunk of, say, a thousand strands at least? It is a rather insane amount of possible bandwidth, but could later be used for distribution along that route. (FYI, maximum theoretical bandiwdth of a single fibre strand is 20Terrabits, using all available optical frequencies)

Re:What about cold countries

[LWATCDR]

"As for the fibre, if they do it hopefully the will put in a nice ig trunk of, say, a thousand strands at least? It is a rather insane amount of possible bandwidth, but could later be used for distribution along that route."

I suggest you take a look at what is between Inuvik and US :) ahref=http://www.inuvik.ca/tourism/maps.htmlhttp:/ /www.inuvik.ca/tourism/maps.html>
That isn't a lot of people there to use that much bandwidth. I would guess that with a 1000 strand line a 100 gigabits a person would not be out of line. Man I hope one of them is hosting a torrent I want :)
I think they use a satellite feed now for Internet access ahref=http://www.newnorth.ca/http://www.newnorth.c a/>
Frankly I think the cost of construction would be the killer. You have only a few months a year to work plus the permafrost to deal with. There is a road but it is still a long way from anything and with the gas prices going up I just do not how practical it would be.
Still I can see a pretty domed city with lights for the winter, green houses for fresh produce, and usiversal wifi. A cool techtopian vision to be sure.

got them all beat

[Anonymous Coward]

And who will be the first vendor to bring water back into the data center?

Hah! We've got them all beat. We keep a water cooler in the server room.

If you have cooler servers...

[havaloc]

...you won't need as much cooling in the room. Easy enough. This will save a ton of money in the long run, not to mention the environment and all that.

Re:If you have cooler servers...

[ari_j]

That depends entirely on how you get cooler servers. You could set up a complete liquid cooling system per server and have it be less efficient than just cooling the room that they're all in. Implementation details make or break the assumption you've stated.

Re:If you have cooler servers...

[Anonymous Coward]

It's all relative to how you get cooler servers. Do you pack more fans into each case? Do you use a water cooling system? I all instances heat is expelled where cold it created. Cooling the inside of the case just dumps warm air outside the case(into the room.) Pumps are needs for water cooling. They generate heat. Follow? If the air in the room is cool, the intake into the server cases/rack enclosures will be cooler to begin with. My vote is to cool the room.

Re:If you have cooler servers...

[Illserve]

The heat has to go somewhere, it doesn't magically disappear just because you put fans in the case.

More to the point, more fans mean more power, more power means more heat.

Cooling the room lets you focus your efforts on one part of the system that is external to the room.

On the other hand, HVAC systems break all the time and it would suck to lose a million dollars of equipment every time it happened.

Re:If you have cooler servers...

[Bongzilla]

how about having the systems exhaust their heat outside of the room (similar to a laundry dryer)

I dunno why you have all this gear heating up the datacenter, when it would behoove you to keep that room cool since you have expensive gear in there that tends to perform better in a cool environment!

Another similar but more specialized problem is information pollution on the networks, and even buses, used by extremely high performace machines. I've been involved in the design of systems in which performance was such an issue, that they could provide no security features of their own. Meanwhile, non performance machines with many non enterprise user apps were placed on the same networks, using the same disk arrays!

This is provably an analogous problem but I don't have the time to go into it here and I'm feeling lazy.

Re:If you have cooler servers...

[UnknowingFool]

This will save a ton of money in the long run, not to mention the environment and all that.

Nottt ttttto mention cccccomffffort fffffor tttthose of ussss whooo have tttto wwwwork in the datttta-centtters.

Re:If you have cooler servers...

[imroy]

Um... we're talking about servers/CPUs that produce less heat in the first place. Not simply putting more fans in them.
Look at the 1.xGHz G4's in the Mac mini. I have a 750MHz Athlon that needs a pretty good heatsink and fan, while the mini gets by on one little fan.

Re:If you have cooler servers...

[SpongeBobLinuxPants]

Managers with true data centers probably would not go for a 16-way VIA database server... A faster server saves the company more money with productivity than lower air conditioning costs would.

Outside air?

[jargoone]

Maybe my ignorance is showing here, but does any installation use outside air for cooling? It seems that it would make sense in places that have cold winters (like here in the midwest).

Re:Outside air?

[green pizza]

Maybe my ignorance is showing here, but does any installation use outside air for cooling? It seems that it would make sense in places that have cold winters (like here in the midwest).
You'd need a lot of filtering and/or humidity control to make that a realistic option. Better yet to make use of outside air temperature. Which is exactly what your heatpump loop or your AC cooling tower is for.

Re:Outside air?

[iso]

I'm no expert, but I would assume that it's probably too unpredictable, and too humid

Re:Outside air?

[Anonymous Coward]

We used to do exactly this in an ops building belonging to the company I worked for in 1997. The server room was cut off from the rest of the heating system in the building, with piped cold air from outside.

It took 8 months until the first servers started dying from the intense corrosion & pitting on equipment closest to the air outlets. We were bringing in air that while it was ice cold, was unfiltered and brought pollution in from 2 storeys above street level, and I dare say more moisture than the air conditioned recycled worker-breathable air.

Filters fixed the problem.

Re:Outside air?

[Anonymous Coward]

Especially in the colder areas of the world it'd be criminal to waste the heat by pumping it outside (up to a certain point, anyway)

You could be heating buildings or a greenhouse with it, after all. Or making steam to pipe heat. Maybe even turning generators? Not sure what the step-down of the efficiency of it all is.

Apparently A/C is only 1/3rd efficient... but as you're going to be losing that anyway, might just look at the output heat.

Re:Outside air?

[milgr]

Last year, the AC died for one of my company's smaller labs. We connected some portable AC units, set up fans, removed the glass from a large window on this 10F day, and shut down about 1/2 the servers. This kept the lab temperature to about 95F. It was also a short term solution.

Re:Outside air?

[onyxruby]

Live in Minnesota here. We have air conditioners for server rooms that are designed to be used in the middle of January. HVAC contractors know what a server room is, even if they don't have the foggiest about the contents. It takes specialized HVAC to do it, but you better believe those server rooms are air conditioned in the heart of winter. On the plus side, it's usually easy to get an HVAC contractor to work on one, after all who else uses an air conditioner when it's 20 below?

When I worked at Target we had specialized monitoring equipment that notified the same people that handle burglar alarms if a server room got to be too hot. It was written directly in the contract we had with the HVAC co's that only 911 call centers and Hospital Emergency rooms could be prioritized over one of our server rooms.

Re:Outside air?

[NixieBunny]

There are things called economizers that do this task. We have one in our radiotelescope cooling system. In fact, it's just been converted to proportional control (was on/off) to improve the temperature stability. It works great in the wintertime, but then the mountain has rather dry air to start with; that's why it has telescopes!

Re:Outside air?

[Mattintosh]

Target uses Automated Logic HVAC controls. ALC has had a long-standing relationship with Liebert, and *they're* the "specialized" HVAC guys that do server rooms. I know all this crap because I work for an ALC dealer. I do this stuff for a living.

As for the GP's question about outside air: yes. Outside air is very commonly used for "free" cooling. You have to calculate the enthalpy, which is the potential amount of cooling based on the temperature and humidity of the air inside vs. outside. Basically, if you have low outside air humidity, your enthalpy can go sky high. This is especially true if your indoor humidity is high. It's basically just a calculation of how much surface evaporation is going to take place if you mix these two air sources.

Of course, a proper HVAC unit is going to have decent (and hopefully well-maintained) filters. Which means that those server room units are already using as much outside air "free" cooling as they can.

Re:Outside air?

[onyxruby]

I used to do this for a living, it's where I got my break in IT. I was one of the people that worked in the control center monitoring all of the equipment. When I worked there the level of preventive maintenence they performed (coils washed etc) was staggering. The amount of money they saved by doing all of this was also staggering.

The stores are all computer controlled for their HVAC and lighting. I would spend my day working with HVAC contractors and electricians and remotely monitoring the stores. We had server room air conditioners in North Dakota and Minnesota that we had running in January, using as much free air as possible of course. I still probably know more about commercial HVAC than home.

Re:Outside air?

[Anonymous Coward]

Outside air is already done, a basic economizer approach with a low dewpoint lockout is all you need. Very efficient in temperate climates like Silicon Valley. Datacenters run 24/7- for most of the year you don't need an AC unit banging away at 1 AM, just bring in outside air.

Actual humidity requirements are pretty lax for server equipment, with the main problem being very dry winter air. In really cold climates, a cooling tower can make cold water evaporatively (very efficiently - less than 0.06 kW/ton typically versus 1-2 kW/ton for standard unit cooling) and feed coils.

So, there are a few control quirks to be dealt with, low humidity lockout and such, but straight outside air for cooling can and is used. Where it can't be, a waterside economizer/'strainer cycle'/cooling tower system can do the trick (works great for cleanrooms too where filtration actually is a problem).

Re:Outside air?

[ispland]

Use fresh, already cool outside air? In HVAC terms, you are referring to what's called an "economizer". These are not uncommon (and will save money for your typical building. Adding the necessary ducting, filtering and control hardware will add to the intitial cost, however.

For a datacenter, economizer options are not popular. Why? As a general rule, outside air is generally considered too dirty and too humid to be worth using on valuable data center equipment.

Re:Outside air?

[Anonymous Coward]

I know a number of smaller installations that do. At the very smallest scale, one smoker friend has reversed all his fans and backed the computer up against a fresh air duct: it actually goes a long way towards keeping his apt livable especially in the winter.

As another poster pointed out, air conditioning for larger datacentres is as much or more a matter of keeping the air clean and slightly humid as keeping it cool. In those cases, closed loops make more sense.

Re:Outside air?

[adamruck]

PLEASE MOD PARENT UP

I was just thinking the same thing. Well... we used some sort of resource to create the electricty that is causing heat, why not use the heat to create some more electricity.

Certainly this wouldn't be something that you do at home. However for situations like huge data centers with racks that are putting off 50-90K BTUS, times however many racks there are, that adds up to an ass load of heat. Why not pipe all that heat off to a generator sitting on the roof?

Re:Outside air?

[Anonymous Coward]

Parent poster here.

The problem with that is the thermodynamic losses are always going to add up to 100% for the same reason that voltage always drops to 0 by the end of a circuit. What produces energy is the temperature differential, not the presence of heat itself.

Simply reconverting the heat into electricity doesn't make the sense that you would imagine, since you are still left with the problem of how to get rid of all this buildup of heat on the other end.

However, in situations like steam systems, greenhouse and building heat, road de-icing, etc you are using either an inherently lossy system (greenhouse or building, for example, which during winter apparently always need x+1 heat, where x is the heat being pumped in) or a system with a thermal gradient built into it already, like the problem of melting large quantities of snow.

Re:Outside air?

[themusicgod1]

Not sure about institutions but me & my roommates have left our windows open through most of the winter. My room was about room-tempurature, and my roommate's room was about 40 degrees Celcius the whole winter, just from the heat of our computers alone. Now that we don't have the -60 degree celcius winter to cool our computers I've had to turn half my computers off just to keep from turning my room into a completely uninhabitable place.

Re:Outside air?

[identity0]

This reminds me of what my high school's compuer lab did.

The room had a very high ceiling where all the heat would rise to, and when it got too hot we could activate large fans set high on the outside wall that would push air *out* of the room, instead of in. This pushed the hot air out, and new cooler air would come in from the rest of the building. There were metal shutters that would close when the fan was not in use, to keep outside air and humidity from coming in.

All in all, it was pretty effective, though a bit loud. Of course, this was a lab of PCs and Macs, not a datacenter.

Liquid Oxygen Anyone?

[dink353]

That might keep the odd CPU or two cool for a while...

Re:Liquid Oxygen Anyone?

[P-Nuts]

I realise this was a joke, as obviously liquid oxygen is going to be a fire hazard. But you can't go pouring liquid nitrogen on either, as you'll have problems with frost forming from moisture in the atmosphere.

inevitability breeds contempt

[icebrrrg]

"Roger Schmidt, chief thermodynamics engineer at IBM, [recently] admitted that, while everyone knows servers are one day going to be water-cooled, no one wants to be first, believing that if their competitors still claim they are fine with air cooling, the guy who goes to water cooling will rapidly drop back in sales until others admit it is necessary."

you know, some times the market actually rewards innovation. tough to believe, i know, and this isn't innovation, it's common sense, but mfg's are afraid of this? come on, people, the technocenti have been doing this for their home servers for a long, long time, let's bring it into the corporate world.

Re:inevitability breeds contempt

[trentblase]

Manufacturers are worried about raising the price point. Water cooling is actually pretty pricy, so you have to measure the (potential) overclocking gain against the cost of throwing another blade in there. If you really need raw serial performance, as another poster pointed out, Cray has always used liquid cooling.

Re:inevitability breeds contempt

[Anonymous Coward]

you know, some times the market actually rewards innovation. tough to believe, i know, and this isn't innovation, it's common sense, but mfg's are afraid of this? come on, people, the technocenti have been doing this for their home servers for a long, long time, let's bring it into the corporate world.

IBM's mainframes have been water cooled since the 70s. Afterall, they did invent the technique!

Re:inevitability breeds contempt

[scsirob]

.. Care to explain how going back to a cooling system from 30 years ago qualifies as innovation?!?

Seriously, with the greenhouse effect and everything, I think preventing heat from being generated is a double-edged sword. Cut down on energy heating the servers up, so you can also cut down on cooling the remaining heat.

Re:inevitability breeds contempt

[Anonymous Coward]

come on, people, the technocenti have been doing this for their home servers for a long, long time

I don't understand. Is technocenti the plural of ricer?

Re:inevitability breeds contempt

[non-poster]

Regardless of cooling method, everything is still generating the amount of heat. If we can reduce the amount of heat generated by having more efficient components, we'll be better off.

Re:inevitability breeds contempt

[alc6379]

the technocenti have been doing this for their home servers for a long, long time, let's bring it into the corporate world.

I don't particularly like this mindset. No offense to the parent poster, but I don't agree that just because some tech-savvy home user can successfully get something going it means that it should be used in a production environment. I'm probably just as bad a "ricer" as your next individual, but that doesn't mean I'd consider doing a Stage 1 Gentoo install for all of my datacenter servers...

Re:inevitability breeds contempt

[DerekLyons]

"Roger Schmidt, chief thermodynamics engineer at IBM, [recently] admitted that, while everyone knows servers are one day going to be water-cooled, no one wants to be first, believing that if their competitors still claim they are fine with air cooling, the guy who goes to water cooling will rapidly drop back in sales until others admit it is necessary."

you know, some times the market actually rewards innovation. tough to believe, i know, and this isn't innovation, it's common sense, but mfg's are afraid of this? come on, people, the technocenti have been doing this for their home servers for a long, long time, let's bring it into the corporate world.

The technosenti are generally cooling a single machine in a single location. Cooling any significant number of machines (even if they are all in one rack) is a very complex process with many opportunities per machine to screw things up. Then there are the extra labor and maintenance costs, then large (and worse yet, upfront) capital costs of the system at the datacenter level.

The market rarely rewards something that so increases the cost of the system upfront without providing an upfront performance boost.

So everyone is playing chicken?

[jcuffe]

while everyone knows servers are one day going to be water-cooled, no one wants to be first, believing that if their competitors still claim they are fine with air cooling, the guy who goes to water cooling will rapidly drop back in sales until others admit it is necessary

So everybody knows that it's necessary, but they're just waiting for the other guy to do it first so that they don't have to take any risks? Sounds familiar to me.

No brainer for me...

[Eil]

Ideally, you should have a cool server and and cool room. The two work in combination. If you have a hot room, then the server isn't going to be able to cool itself very well even with the best heatsinks and well-placed fans. Yes, you could use water cooling, but there are other important bits inside of a machine besides whatever the water touches. But a cool room does no good if your servers aren't setup with proper cooling themselves.

Re:No brainer for me...

[Anonymous Coward]

"If you have a hot room, then the server isn't going to be able to cool itself very well even with the best heatsinks and well-placed fans."

Tell me about it. We have a 64-node cluster here at the university and after moving into the new building, the physical plant decided that 95 degrees F was alright because the computers could cool themselves....

Fast-forward... we now have a 13-node cluster, a bunch of melted motherboards, and a physical plant that says it wasn't their fault...

Already done.

[nickco3]

There is already water in the datacentre where I work. The site is a converted leisure centre, and has a water-sprinkler fire system. The first whiff of smoke in that place and the entire server room is toast. A Halon system is regarded as too expensive. Seriously.

Re:Already done.

[IckySplat]

Actually most new Data Centres are using water.
Halon and it's replacements are actually bad for the hardware.
These days Data Centres use de-mineralised water.
Safer for the gear (Once you've dried it out)

Re:Already done.

[C_Kode]

Our NOC uses dry pipe water sprinklers. The pipes are pressurized and empty. If a fire starts they do not open. It has to get hot enough to melt a release valve. If the release valve melts, then the water passes into the pipes and only then can be released by personal into the room.

Re:Already done.

[D3]

Can't use halon for a lot of reasons. Halon is corrosive to the components, dangerous if you have people around, and systems cannot be recharged once discharged. This is why it makes sense to have a real emergency backup plan to run your critical systems off-site in the event of a problem in your data center.
Also, with water and proper electrical controls, you can shut down the servers quick with one big switch and leave them off until they dry out. You won't lose 100% of the equipment and insurance will easily cover what you do lose.
Of course, you could use a dual system with FM200 to slow the initial spread and give time to shut down clean, then use water a few seconds later when no power is moving through the systems. Much better long term.

Re:Already done.

[jaavaaguru]

Pressurized with what? Air?

Re:Already done.

[par38lamp]

Actually, the way a dry pipe, or more commonly called "Pre-action" works, the pipes are pressurized with air.

The pressure is monitored with a fire alarm panel along with smoke/heat/fire detectors. Even if a sprinkler head melts, there usually has to be a second detection method in alarm to fill the pipes with water.

Other dection methods include standard smoke and heat detectors, "fire print" detectors that can analyze the characteristics of a flame, and active air sampling piped detection systems that use a laser beam and particle discriminators (much quicker than passive smoke detectors).

Liebert makes racks cooled with water already. The XDWR and XDWP lines. I saw them at an Emerson Network Power expo in Ohio last year.

Halon was phased out primarilary because it was ozone depleting and became VERY expensive to refill upon release. FM-200 and Intergen are available still.

First thing..

[Anti Frozt]

That comes to mind is that it will probably be vastly cheaper to cool a rackmount specifically than to lower the ambient temperature of an entire room to the point that it has the same effect. However, I'm not entirely sure how well this scales to large server farms and multiple rackmounts.

I think the best option would be to look at having the hardware produce less heat in the first place. This would definitely simplify the rumbling these engineers are engaged in.

Re:First thing..

[infolib]

it will probably be vastly cheaper to cool a rackmount specifically than to lower the ambient temperature of an entire room to the point that it has the same effect.

That really depends on how well isolated your room is. Energy flows into the room as electricity (which the servers convert to heat) and heat from the surrounding rooms. If you've got powerful servers and a well isolated room, you might not win much by isolating even better. (Which is the point in cooling only the rack).

Re:First thing..

[myov]

I thought APC had a system which put racks back to back, sealed the backs and then cooled the exhaust air. The idea being that you're not cooling the entire room, just the hot air where it's being generated.

Of course, I can't find the link now.

Cooler servers, definitely

[AtariAmarok]

Cooler servers, definitely. If you have cooler rooms, people from all over cubicledom will gravitate to the room during the dog days of August, followed by the inevitable "what does this knob do?" and secretive "what happens if I start switching around these funny phone cables in the back of the black box?".

The fewer rubes lounging by the server towers, the better.

Re:Cooler servers, definitely

[cdcarter]

We have cooler room, we have lock...

Re:Cooler servers, definitely

[Anonymous Luddite]

>> If you have cooler rooms, people from all over cubicledom will gravitate

you're right. People *do* know the servers are 72 degrees all day every day. That's why we lock our server rooms. If you do get more than 4 feet in the door and you aren't supposed to be in there, you'll get some friendly directions..

Re:Cooler servers, definitely

[Tesen]

Cooler servers, definitely. If you have cooler rooms, people from all over cubicledom will gravitate to the room during the dog days of August, followed by the inevitable "what does this knob do?" and secretive "what happens if I start switching around these funny phone cables in the back of the black box?".

Haven't you guys ever heard of physical security? Why on earth would you maintain any kind of server room where the everyday user can gain access? Auto-locking doors behind you, big clue by fours to wave at people approaching you as you walk in.

It is a dismissal offense to allow non-administrators in to our data centers where I work.

Tes.

Water cooling, pah!

[hazee]

Water cooling? Pah! Why not take a leaf out of Seymour Cray's book - build a sodding great swimming pool, fill it with non-conductive freon, then just lob the whole computer in.

Also has the added benefit that you can see at a glance which processors are working the hardest by looking to see which are producing the most bubbles.

Wonder if you could introduce fish into the tank and make a feature of it? If you could find any freon-breathing fish, that is...

Re:Water cooling, pah!

[LiENUS]

Isn't freon liquid at room temperature? Your swimming pool wouldn't last too long, you'd either have to preassurize the system (not good for equipment) or refill it every couple of minutes.

Re:Water cooling, pah!

[hazee]

Isn't freon liquid at room temperature?

Err... that's the point. Wouldn't be much of a swimming pool otherwise, would it?

Besides, without liquid in the tank, you'd have to get hold of levitating fish, which I suspect would be even harder than finding simple freon-breathing ones.

Re:Water cooling, pah!

[Anonymous Coward]

Submersive cooling has been used in some high-end servers for years. There is a non-conductive liquid from 3M called Fluorinert, which can cost up to $3000/L, and is used just for this purpose.

Re:Water cooling, pah!

[Quiet_Desperation]

If you could find any freon-breathing fish, that is...

You might also want them to be non-excreting.

Someone with true Kung-Fu would build a computer out of the fish.

Re:Water cooling, pah!

[hairykrishna]

The old Crays used a freon like flurocarbon which was liquid at room temperature. As these are the same fluids which were proposed for liquid breathing systems (like in the the Abyss) your swimming pool should be able to support air-breathing life (hamsters?)never mind fish - so long as you oxygenated it of course. It's about $400 a gallon. An olympic sized pool contains ~2 million litres so it would cost ~160 million dollars to fill unfortunately.

Re:Water cooling, pah!

[LiENUS]

Im sorry I didnt mean liquid, I meant gas. Freon evaporates at a really low temperature. Well that and freon is illegal to use now, r12 was replaced with r134a in cars some time ago and houses now use r22.

Re:Water cooling, pah!

[TheHawke]

He was probably thinking of R-11 which is liquid at slightly below room temp, or Fluorinert. R-11 which contains a shitpot of the so-called "ozone killer" chlorine, was banned immediately when the fed signed the clean air act. The AC techs used to use R-11 to clean electronics and calibrate their equipment with it. Great stuff.

On a side note,i'm surprised that the IT dept have not made a attempt to either duct chilled air directly into the racks, or tap the chilled water system that their buildings use to keep their rooms cool. I mean, thats 35 degree water circulating through those pipes, you know? Even a simple titanium heat exchanger would be able to tap a large amount of heat from the systems that use H2O coolers.

I would not dare try to tap directly into the building's chill water supply and route that water into any system. That stuff's nasty, got small chunks of dirt in it, and it would probably eat up the lines, save for tygon and some other hardy plastics. Or worse, the pressure surges might cause a fitting to pop loose! Why do you think they use IRON to plumb the chill water systems?
'sides, the pumps used to circulate the water are 5-20HP 3phase pumps and i'm not about to try to monkey with the head pressures those monsters dish out.

Re:Water cooling, pah!

[DJCF]

I like it - I don't need no stinking fish, I'll take a terranium in my supercomputer!

Re:Water cooling, pah!

[artemis67]

There was a story on Slashdot a few years back about a guy who got a non-conducting liquid (I think he used mineral oil), cooled it and submerged his motherboard in it. It worked great -- for about half an hour. The problem was that, on a stock motherboard, there are lots of pieces that had to be plugged together (plug the video card into the AGP slot, plug the power into the motherboard, etc), and the oil eventually seeped in and coated all of the contact surfaces, disrupting the connections.

Moral of the story was, to do submersive cooling, EVERY connection has to be soldered down.

seems to me...

[justkarl]

that you should stop the problem where it starts. Cool the servers, then the room won't get hot(duh).

Re:seems to me...

[bill_mcgonigle]

that you should stop the problem where it starts. Cool the servers, then the room won't get hot(duh).

Easy to say, but are you willing to give up fast servers for cool servers? We don't have the technology to make fast and cool microchips.

Re:seems to me...

[Peyna]

Cool the servers, then the room won't get hot

So long as you're pumping the heat that you're drawing off of the servers outside that makes sense. But otherwise, the room will get very warm, because all the heat from the servers is just being pumped out into the air. Thus the servers will have to work harder to keep cooler and be fighting against each other in a way. You need to get the heat out of the server room at some point in time.

It's like taking a window air conditioner and setting it on the floor and keeping all your windows closed. The area right in front of the air conditioner might be cool, but you're going to end up heating the room up overall.

Obvious choice

[Black Jack Hyde]

Cooler beer, of course. Mmm, beer.

Water in the Data Center

[eericson]

I had a vendor bring water into my data center once.....

JUST ONCE.

Re:Water in the Data Center

[C_Kode]

Just once? I have a Slip and Slide between the racks!

Cooler rooms are great until...

[MasterOfCeremonies]

...a major problem happens and you have to spend four hours straight in there freezing your tits off.

Re:Cooler rooms are great until...

[Ruud Althuizen]

Ever heard of climate CONTROL

Swiftech

[eander315]

Swiftech is my guess as the first who will offer widespread, professional watercooling solutions for 1U rack-mount water cooling solutions using the Laing DDC pump, rebadged as the MCP350. I don't think any of the other big players in that industry currently have the products or expertise to pull it off in the near future.

Re:Swiftech

[GKevK]

It would be interesting if there were just a standard developed for a water cooling connection... just like power and network. The machine room would get more comfortable for humans both from a temperature and sound perspective.

Re:Swiftech

[green pizza]

Swiftech is my guess as the first who will offer widespread, professional watercooling solutions for 1U rack-mount water cooling solutions using the Laing DDC pump, rebadged as the MCP350. I don't think any of the other big players in that industry currently have the products or expertise to pull it off in the near future.

Yeah, I bet Dell or HP can't figure out how to do that.

*rolls eyes*

Actually, given some of the recent decisions made by HP, they probably couldn't!

Is cooling the answer?

[$RANDOMLUSER]

How about better instruction sets?
Which uses more power: 20 micro-ops or 400 micro-ops?

No way, not in my shop

[doc_traig]

The sign on the door clearly states, "No Food or Drink". Of course, shirts are still optional.

Re:No way, not in my shop

[Anonymous Coward]

The sign on the door clearly states, "No Food or Drink". Of course, shirts are still optional.

See, we don't need cooler servers or cooler rooms, we just need cooler admins.

Re:No way, not in my shop

[rob_squared]

"Of course, shirts are still optional."

-For pre-approvied female employees only.

Many processors for cooling

[kabbor]

The first thing I thought was - Yes, half the heat per processor x 4 chips = double the heat, right?
Then I recalled that the reason for those blindingly fast chips is so that they can get through their present job and react in a timely manner to the next task.
If you've got another chip or two waiting, then you can take your time, can't you!
(I'm also sure that power usage could be shown to be exponential, so we will make gains then.)
Hey, and who needs water? There's a few of those non-wetting liquids that we could use. Flow them around the die itself for maximum heat transfer!
Lots of fun.
(Until $RANDOM decides a glycol antifreeze would make a better chioce! Then EVEN MORE fun!)

Re:Many processors for cooling

[green pizza]

Hey, and who needs water? There's a few of those non-wetting liquids that we could use. Flow them around the die itself for maximum heat transfer!

That's what Cray does. They spray the stuff directly onto the dice to utilize evaporative cooling.

http://www.idexpositivepumpcare.com/images/micropu mp/microelectronic/moww_et_collage.jpg

No one...

[AndyCap]

Since rittal is already there.
PDF flyer here: here

Does look like a neat way to keep your beowulf cool. :-P

Cooler servers...

[Aphrika]

From experience of aircon failing/breaking.

At least if a server fails it's one unit that'll either get hot or shutdown which means a high degree of business continuity.

When your aircon goes down you're in a whole world or hurt. Ours went in a powercut, yet the servers stayed on because of the UPSes - hence the room temperature rose and the alarms went off. Nothing damaged, but it made us realise that it's important to have both, otherwise your redundancy and failover plans expand into the world of facilities and operations, rather than staying within the IT realm.

More to the point...

[Anonymous Coward]

..when will vendors STOP taking the water ?

My anecdotal data using HD temps:

[ender-]

Lets see here. I have a server [Duron 1200] in a datacenter that's kept fairly cool. [Not as cool as it should be, it's a shitty hosting company, but I can't complain because I'm not paying for it]. The HD temperatures in that server run around 54C.

I have another server [P4 1.8Ghz] sitting in a spare bedroom of my house. The temperature in that room is usually about 80F (27C) even with the AC going. It gets even warmer if I'm in there gaming on my desktop machine during the summer. The HD temps in that server are around 33C.

The difference between the two cases is the fans. In my home server there are 4 fans blowing directly over the HD's using air coming in from the front, as well as an exhaust fan directly over the CPU, an exhaust fan in the back plus the PS fan. In the datacenter server, there's just the PS fan [and MAYBE an exhaust fan in the back, I don't remember].

So in my experience, the cooling in the server itself is more important than the ambient temp. All the cool ambient air in the world isn't going to help if the server case becomes a small insulated pocket of hot air. With that said it's important to give both as much cooling capability as possible.

Ender-

Re:My anecdotal data using HD temps:

[Anonymous Coward]

In my home server there are 4 fans blowing directly over the HD's using air coming in from the front, as well as an exhaust fan directly over the CPU, an exhaust fan in the back plus the PS fan.

Holy crap, isn't that loud?! Loud computers suck.

Re:My anecdotal data using HD temps:

[ender-]

Holy crap, isn't that loud?! Loud computers suck.

Heh, yeah it's a bit loud. But as its purpose is as a fileserver, it's more important to keep the hard drives cool than to keep it quiet.

Of course, my game machine isn't exactly silent either, with the CPU fan, GPU Fan, Chipset fan, MB exhaust fan [part of the motherboard, exhuasts next to the PS/2 ports], PS fan, rear exhaust fan and side exhaust fan. It's not as bad as it you might expect, as ALL the fans are variable based on the temperature. It runs fairly cool with an internal ambient tempurature usually around 100F at Max load [according to the thermometer built into the case with a sensor sitting between the CPU & GPU.], the CPU temp is usually around 35-40C tops [Athlon64 2800+]

Ender-

Re:My anecdotal data using HD temps:

[dbIII]

Heh, yeah it's a bit loud. But as its purpose is as a fileserver, it's more important to keep the hard drives cool than to keep it quiet.

I had four 1U servers arrive earlier than expected two weeks before a move, but at a time when all the CPU power available was never enough. All four ended up piled on a crate in a couple of metres from the corner of a conference room so the air blasting out the back wouldn't wreck the vertical blinds. The guy working at a table next to them had to put up with the noise, but since he was using them himself he could turn off the ones he needed, and actually hear what stage his work was up to as the fans ramped up for the CPU intensive bits.

Cool servers are nice for when you find out the AC is on one phase and the servers are on another, and you have to shut down what you can, lift the ceiling tiles and bring in industrial fans to get some air moving.

the CPU temp is usually around 35-40C tops [Athlon64 2800+]

For a few months I could only run my amd64 for more than a couple of hours at night with a pedestal fan pointed at the open case, since ambient was 35-40C. Water cooling looked good at that point, but condensation is not a trivial problem to solve in a high humidity situation.

Not really

[Anonymous Coward]

Cray has always used water cooling for their big iron

Actually not technically true. They don't use "water", the older ones anyway used a liquid called fluorinert by 3M to do their cooling. In this way the components could be submersed in the fluid (which you obviously can't do with water since it's conductive). You could also get T3E configs that were air cooled (though you could get more processors in the liquid cooled versions)

Re:Not really

[hey]

components could be submersed in the fluid....
cool!

Re:Not really

[javamann]

Actually pure water is not conductive. By measuring the resistance is one way to tell how clear your water is. (Wafer FAB background, sorry)

Re:Not really

[Anonymous Coward]

yes, but distilled water is also *really good* at dissolving things which are submerged in it. so it wouldn't stay pure for long...

A/C and commercial solution worked well

[matth1jd]

At the university I used to work at during my undergrad our servers were cooled not only by 2 large A/C units in the room, but by large fans on the rear of the racks whose exhaust went directly out of the room.

IIRC this was a solution from APC. All together it effectively kept the room at right around 55 - 60 degrees.

People used to wonder why I'd go to work in jeans and long sleeved shirt in the summer.

It's not a colder room, it's air circulation

[pcguru19]

The costs for improving data centers to provide more or colder air is more than just building out more square feet of data center space.

Just because HP is sells a 42U rack doesn't mean you have to cram blades into all 42Us. It's cheaper to spread the heat load across a larger area than to figure out how to put 1500 CFM out of a floor tile so the top of a blade rack gets air.

There are studies by the uptime institute that say that 50% of hardware failures happen in the top 25% of rack space because the top of the rack doesn't get any air from the floor tiles and it cycles exhaust from the rack or ambient air for cooling.

We just put in the latest blade rack from HP. 4 50 amp circuits(2 for redundancy) for a 4 square foot space is beyond silly. That's more service and electrical consumption than a 1500 square foot home after you eliminate the two circuits for redundancy.

Re:It's not a colder room, it's air circulation

[MadMorf]

Just because HP is sells a 42U rack doesn't mean you have to cram blades into all 42Us. It's cheaper to spread the heat load across a larger area than to figure out how to put 1500 CFM out of a floor tile so the top of a blade rack gets air.

Agreed, to a point.

Google did a study about Datacenter Power Density a couple of years ago and IIRC, concluded that Blades were not the most cost efficient solution (for them) because of the increased environmental conditioning required...

Wish I could find that link...

direct connect

[Anonymous Coward]

I wonder if the industry will come out with a standard direct AC to case connection anytime - seems to me to be more efficient than cooling the whole freakin' room.

experiments

[ArnIIe]

It would be interesting to see some stats (if available) on what happens to the servers heat, when a room temperature drops in incremental stages. Also I have seen some server farms that have fans inside the cabinets for extra cooling.

Cooler servers!

[Brian Stretch]

See the power consumption chart on this page. Buy the right CPUs and heat is much less of a problem. (Yes, I know, PowerPC is better in this regard, but if you want to run x86...)

I believe...

[pkx]

...the Cisco CR-1 (known internally as the HFR - Huge F'in Router) uses water cooling.

Save the globe: don't slashdot !

[AtariAmarok]

"[sarcasm]But that would promote Global warming, the melting of the Polar Icecaps, the Greenland Glacier and other glaciers! [/sarcasm] Seriously, I can honestly see someone arguing that point."

"Hey! Did you know that when you slashdotted that server near the Ross Ice Shelf, you caused 2 icebergs to calve? You insensitve clod!!!!"

A Wash

[mattmentecky]

I think it is safe to say it is a wash.
If you kept servers cooler, then you wouldnt need the room to necessarily be as cool, so it is just a shift in energy consumption one way.
If you kept the room cooler, the ambient temperature would need to be much cooler to have the same affects of cooling directly on the server, so it is a shift in power consumption the other way.

Energy efficiency and Hosting- Host NORTH !

[cbelt3]

OK, here's a concept.
If data center location isn't such a problem as long as we have high speed data lines, locate the data center someplace nice and cold.
Like Manitoba, or Minnesota, or Alaska, or Siberia. Heat the work area with the flow from the data center.
Hire your local population to maintain the data center.
Profit !

Re:Energy efficiency and Hosting- Host NORTH !

[Anonymous Coward]

You'd need plenty of filtering, first of all. Secondly...

Reposted from my post above:

Especially in the colder areas of the world it'd be criminal to waste the heat by pumping it outside (up to a certain point, anyway)

You could be heating buildings or a greenhouse with it, after all. Or making steam to pipe heat. Maybe even turning generators? Not sure what the step-down of the efficiency of it all is.

Apparently A/C is only 1/3rd efficient... but as you're going to be losing that anyway, might just look at the output heat.

Re:Energy efficiency and Hosting- Host NORTH !

[infolib]

Yeah, but then you'd have to shift your computing to southern Chile during the summer. Put it on a mountain instead.

Re:Energy efficiency and Hosting- Host NORTH !

[Detritus]

Hard disk drives don't like high altitudes.

Re:Energy efficiency and Hosting- Host NORTH !

[freeweed]

Heh.

Temperatures go well over 100F in Manitoba in the summer. For 2-3 months straight, some years. Believe me, Canada is not the arctic paradise some people think. It's one of the places where you need to invest heavily in BOTH heating and cooling of a datacentre.

It was always fun to show my house and car to people from more tropical climes. The car has a plug in for the block heater (absolutely required when it's -30 out), and A/C for the summer. The house has A/C, and a high-efficiency furnace for the winter.

What really gets them, however, is the drink machines located outside: they run small electric heaters during the winter. Yup, we need to warm up our Cokes here :)

Re:Energy efficiency and Hosting- Host NORTH !

[infolib]

Weird - I didn't know. Why?

Re:Energy efficiency and Hosting- Host NORTH !

[Detritus]

I think it is mostly due to the poor heat transfer properties of the thinner atmosphere. The head may also be more susceptible to head crashes due to the smaller air cushion provided by the thinner atmosphere.

I read about this being a real problem at many of the new astronomical observatories that have been constructed at high altitudes.

Cooling is not an option

[onyxruby]

Water cooled servers have been out for a little while by some vendors. You can find rack mount water cooled gear pretty easily. Too much damage is done too quickly when you don't have cooling. I have worked in environments where if a server room was allowed to get up to 74 F /23.3 C and an HVAC contractor wasn't already on the way there would be hell to pay.

There really isn't a question of if it will become widespread. Overclocking sites have had more than a few visits from Intel and AMD over the years. It's an inevitable problem with an inevitable solution. The only question is how long until water cooling becomes more popular. Heat needs have had people clamoring for Pentium M processors for rack mount gear for a while as well. It's a reasonably speed CPU that handles heat fairly well. It would work very nicely in rack mount gear, but motherboards that will take one are fairly rare.

As for server rooms, they will continue to be air conditioned for as long as all of your server room equipment is in their. Even if you found a magical solution for servers you still have RAID arrays, switches, routers and the like all in the same room. Server rooms are well known by HVAC people as requiring cooling. Most HVAC vendors will prioritize a failed server room HVAC over anything but medical. They know damn well that anybody that has an air conditioner designed to work in the middle of January in Minnesota or North Dakota isn't using the cooling for comfort.

Re:Cooling is not an option

[russellh]

Yeah. you could put the datacenter underwater in the north atlantic. powered by tidal generators. Wanted: experienced system and network administrator. Open water / extreme conditions diving experience a must.

water already there

[Anonymous Coward]

last I checked, Cray's were water cooled out of the box.

Heat the great white north

[AtariAmarok]

"Like Manitoba, or Minnesota, or Alaska, or Siberia. Heat the work area with the flow from the data center. Hire your local population to maintain the data center" P? You really like Innuit women in bikini's, don't you?

So what is the best temp for the room

[arctuniol]

Currently my server room is sitting around 67-70 degrees. I am curious what the best temp for a server room should be at.

Re:So what is the best temp for the room

[Tjoppen]

Would that be kelvin, Celsius or Fahrenheit?

The first vendor

[nigham]

... will probably be supplying Intel chips. Strangely, Intel's had far more heat-dissipation issues than AMD in the recent past; probably because they concentrated a bit too much on clock speed?

Re:The first vendor

[tomstdenis]

Well the third revision of Prescott [E0] is much better from what I read [and will find out today when I pick up the box ;-)].

What gave AMD a kick from K7 to K8 was the lower voltage [1.1-1.5v instead of 1.6v], probably more efficient transistors [e.g. less leakage], oh, and an integrated heat spreader...

Instead of having the only contact be a tiny little die you spread it out before you even touch the HSF.

You can still get several Ghz components and not have too much temp. It's about spacing. If you cluster them you're gonna burst into flames.

Just because the CPU is running at X Hz doesn't mean every discrete component is running at that speed. For instance the ALUs on the P4 actually work at 2x the clock rate... so a 3Ghz P4 has 6Ghz components. The scheduling buffers of any OOE processor probably work at more than clock rate, etc...

Intel's problem with Prescott is they wanted to jump into 0.09um before it was stable enough. My last P4 [which I sold to a friend] was a 2.8Ghz Northwood. It idled at 35C and ran "busy" at 52C. When I get my prescott tonight I'll see how it compares..

Tom

freee cooooling

[dingDaShan]

Other than some high pings, there is no reason to not just make every data center in the artic circle. The cooling would be free, and then we wouldn't have to argue about what costs less.

Re:freee cooooling

[Anonymous Coward]

Yes, but it would only be a temporary solution:

http://www.awitness.org/journal/melt_ice_cap_glaci er.html

Re:freee cooooling

[dingDaShan]

Well first off, I was JUST JOKING, and secondly, I don't consider a solution that lasts (only) 50 years to be temporary in this computer age.

More efficient (ie less heat output) servers

[GAATTC]

Isn't cooling the room vs cooling the server a bit like trying to solve our energy problms by drilling for oil in Alaska vs spending $400 billion a year securing our oil interests in the middle east. Neither one is a good long term solution, while reducing our consumption helps the whole problem go away. Perhaps the same argument applies to servers. If they are more efficient and throw out less heat, then it is much easier to deal with any heat that is produced.

cooler servers...

[Anonymous Coward]

cooler servers are better. you reduce the power usage from having to cool the room and the servers themselves; this saves your company money. it is also better for the environment since you need less electrical generation (less coal burning, etc.) as well as less need for whatever nasties may be involved in modern HVAC (freon, etc.).

Simple Solution really....

[gosand]

1. Open server farm in the Northwest Territories

2. Open the windows

3. Profit!!!

Reminds me of an amusing anecdote

[Raul654]

About 4 years ago, I was touring the US National Supercomputing Center in San Diego. One of the supercomputers had a clear plexiglass side where you could see inside, and it had running water and even a waterfall. Mind you, this 'water' was running directly over the electronic components. So the guy doing the tour said that it wasn't really water, but a chemical compound similiar to water, but very nonconductive. He tells us that it costs $10,000 per barrel, and that he always gets questions about what happens if you drink it. "Well, we're not sure what happens if you drink it, but we figure one of two things will happen. It could be toxic, and you drink it and die. Or, it could be nontoxic, and when our finicial guys found out you were drinking their $10,000-a-barrell water, they'll kill you."

Cool The Room

[Anonymous Coward]

Those who've done HVAC know that heat doesn't rise - it goes to where the temp is significantly less [read: cold].

Cooling the server is worthless if the temp outside the box is marginally cooler than the temp inside. Besides circulation, ambient heat will impact server internals.

Heat Problems and Misconceptions

[standbypowerguy]

Data centers with heat problems usually fall into three categories; those with inadequate cooling capacity, those with inadequate cooling distribution, and those with unrealistic equipment densities.

However, I often find people have misconceptions, they think they have a heat problem, but in reality they do not. One must measure the air temperature at the inlet to the servers, not the exhaust. If the inlet air meets the manufacturer's specifications, there is no problem, despite the fact that it's uncomfortably hot in the exhaust aisle.

"Hot spots" can often be corrected by rebalancing, which is the science of redirecting the supply air proportionately to the heat loads in the space. Any good maintenance firm that knows data centers will offer rebalancing services.

If you really do have a heat load problem, e.g. more load than capacity, as evidenced by excessive temperatures throughout the space, consult a mechanical engineer that specialzes in data centers.

space vs. capacity

[Aslan72]

To me this is an issue of space vs. capacity. Cooling systems for servers probably don't run cheap and you can't just magically make a cooler server; 15,000 rpm drives clustered together in tight quarters and sitting on top of each other and dozens of servers crammed into close spaces make for hot rooms no matter what you do.

Cooler servers would make for more of a foot print. Somehow, I don't think people can just magically make a server room bigger.

--pete

Powersupplies

[iamthemoog]

In a rack of 1U units, does each 1U slab take 240volt (or 115 or whatever) each, and have its own PSU?

I've often though it might be nicer if there could be one power supply for a whole room of PCs for example. This could be placed somewhere outside of the room and cooled there, with appropriate UPS backup too.

12 and 5 volt lines then feed each PC - no noisy or hot PSU per machine... Peace and quiet, and a bit cooler too...

Re:Powersupplies

[Harassed]

Yep. Each 1U server takes its own 110/220v power input - in fact, most 1U servers now have redundant PSUs so actually take two power feeds.

DC power is an option - IBM for instance has a NEBS compliant DC-powered server in its xSeries range - the xSeries 343

In fact IBM also have a blade offering which is DC powered, the BladeCenter T

These are primarily designed for the telecoms market which I understand already is big into DC-powered racks.

Re:Powersupplies

[evilviper]

12 and 5 volt lines then feed each PC - no noisy or hot PSU per machine... Peace and quiet, and a bit cooler too...

I know it sounds simple and easy, but it's far from it...

First off, devices run on watts, and watts=volts*amps. As you lower the voltage, you have to increase the current accordingly. The higher you raise the current, the fatter the wires have to be to avoid MELTING. If you look at the wires comming off the 12VOLT battery in you car/truck, you'll see that they are HUGE relative to the 120/240Volt power cords that normally plug-in to computers. You would need absolutely enormous copper wires going to the racks, and you'd need MANY of them to supply each different level of voltage.

Additionally, the longer the wire the electricity is running through, the more power you will lose, and the more the voltage will drop. Generators designed to supply 120V normally output 130V or more, because the line losses from just 20feet of extension cord will drop that down to about 120V. A 10 volt drop isn't too significant with 120V, but it would be disaterous even with 12V.

And that's not to mention the voltage drops due to power spikes. When you turn-on almost any electrical device, you will quickly see the voltage drop if you are monitoring it. On 120V lines, this can be about 5-10V. This problem has already shown-up prominently in computers with SCSI drives. They all have a jumper setting so that they will each power-up a few seconds after one-another, so they don't overload your 12V power-supply. It's hard to imagine scaling this problem up to server-room sizes. You'd have to power-up one computer at a time (per-each-large-power-supply) and wait about a minute between each one to give every drive a chance to spin-up. I don't even want to think about it...

Now that we've got those practically insurmountable stubling-blocks covered, let's talk about how little it helps you... It's certainly still going to be just as noisy, because where the PS used-to be, there will now be a large fan or two in it's place to help exhaust hot-air from the case. Besides, server-rooms will never be quiet, because the fans are by FAR drown-out by noisy hard drives. And you're not any better off in the cooling department, because you still have to cool the power supplies just as much, it's just in a different room, where it can be cooled seperately.



I think the cooling problem is rather simple, and I've got a solution... Raised-floor cooling is a poor solution... Dropped-ceiling heat exhaust is a much better one.

By attaching ducts from each exhaust fan, you can direct most of the exhaust heat out of the building. Hook the other end of the duct to a powerful exhaust fan to help pull the heated air out. This doesn't cause each system's temp to drop at all, what it does is prevent heated air from being mixed with the extremely cool ambient air, and you only need about half the cooling capacity as a result. In fact, if you're in a cold area, you could even do without air conditioning units, and just use powerful fans (much lower-power usage than air cooling).

Cooler servers first, cooler rooms second

[dafz1]

We're currently going through re-evauating our cooling needs in our server room. The answer we came up with is we have to buy a bigger a/c unit.

Unfortunately, a couple times per year, the chilled water to our a/c unit gets shut off, and our servers are left to fry. The better answer is to have machines which run cooler. If they lose a/c, they won't fry. However, replacing clusters isn't cheap...and I don't think most people think, "which one's going to run the coolest?" when they are going to buy one.

Does anyone have a link to a page that has grossly generalized heat numbers on certain processor families in certain case configurations(I realize these numbers aren't going to be anywhere near exact, but it would be a starting point)?

Re:Cooler servers first, cooler rooms second

[El]

Unfortunately, a couple times per year, the chilled water to our a/c unit gets shut off, and our servers are left to fry. Here's an idea: make the moron that shut off the water pay for replacing the servers!

Re:Cooler servers first, cooler rooms second

[dafz1]

Obligatory Ghostbusters quote:

Stantz: Everything was fine until dickless here turned off the power.

Mayor: Is this true?

Venkman: Yes, Your Honor, this man has no dick.

Hmmm...

[biglig2]

PHB: Dear god, that server is actually red hot!

SA: Yes, but notice that the room is lovely and cool.

PHB: That's all right then. By the way, what's delaying that upgrade to Windows 2003?

SA: Every time we put the CD in the drive it melts. We think it's going to be fixed in the next service pack.

Re:Hmmm...

[MikeBabcock]

Waiting for heat resistent DVD-3 to be released is yet another reason for the delay of Windows 2012 ;-).

I'm not an EE, but ...

[Anonymous Coward]

You're implying that DC-based power connections will be less hot -- generating less thermal output -- than AC power supplies?

The problem is that the AC/DC circuitry is a complex beast that can adapt to increased / decreased load within the powered circuits, provide for spike and some limited sag remediation, etc. By taking that circuitry out of the local box and moving into a centralized location, you'd still have to have the dedicated components PER box to do the 'power thing'.
Further, where would you locate this massive multi-plexed power supply? Meaning, you'd just throw it on the roof? Yah, right.

How about you need to cool it like any other electronic component? Now, add up the costs of cooling THAT, and the cost of the paradigm shift redesign and expensive intial production concepts for this, and you now know why this would never make it.

Re:Ma Bell has been doing this for years

[jhines]

Telecom equipment runs off -48VDC, and the phone company uses big batteries as their UPS.

It exists, it just is expensive.

RTFA

[Valdrax]

I wonder if the industry will come out with a standard direct AC to case connection anytime - seems to me to be more efficient than cooling the whole freakin' room.

From the article:

Calibrated vectored cooling (CVC) is an example. CVC optimizes the path of cooled air flow through the system, allowing servers to use fewer fans and less power. It directly channels refrigerated air through the hottest parts of the server. IBM recently offered CVC for its xSeries and blades. CVC technology for blades had allowed IBM to launch the first Xeon-based blade product.

SGI had to do this at NASA

[freelunch]

When SGI added BX2 nodes to NASA's Columbia system, the standard air cooling was inadequate. They were forced to do a quick cooling change that added water cooling. Some would call the change a kludge.

More detail on the change, and cooling in general, can be found in this interview with the SGI designers who dealt with the problem.

Ummm, can I say "DUH!" ?

[Anonymous Coward]

Heat transfer. If the room's air is warmer, then the air's warmth will transfer to the colder things (the servers). We want the opposite to be happening here. Colder air so the servers' heat transfers to the air.

SGI Altix already water-cooling

[Cyrano de Maniac]

In an interesting hybrid between system cooling and room cooling, we read at http://www.linuxelectrons.com/article.php/20041028 100608909:

For the first time on a product carrying the SGI brand, the new Altix system also can be installed with an optional water-cooled door, which provides system cooling capabilities in addition to the Altix system's existing air-cooled architecture. Already in use at NASA, the water-cooled door option is ideal for environments in which high-density systems are deployed in close proximity to other large-scale equipment.

Anecdotal stories from onsite indicate that the cooling is so good that users/admins want to wear warm clothing if they are in the server room for extended periods.

Why not neither? Remove the power supplies.

[mollog]

If they would redesign server racks so that the DC power for the motherboards was brought in from outside the server room, they could probably;

Reduce power consumption

Reduce heat in the server room

Improve reliability

Re:Why not neither? Remove the power supplies.

[timster]

I don't think the power supplies contribute a really major portion of the heat to servers these days. It's all about disks and processors.

As for power consumption, I don't see how converting the power outside the rack uses less power than converting it inside the rack. And it won't improve reliability since each server will still need a power supply, it will just be a DC-DC one. I don't think you can (reasonably) run a 500-watt power line at 12 volts. Not to mention that you need more than one voltage.

Re:Why not neither? Remove the power supplies.

[Iphtashu Fitz]

Converting to DC can help a lot in big datacenters if you have a lot of hardware. UPS's run exclusively off DC. (remember, they're basically just chains of car batteries daisy-chained together) The datacenters lose power & generate heat in the conversion from AC to DC and back to AC. They're always happy to avoid that second step if possible. And if you happen to have hardware located in a datacenter where telcos have equipment you're likely to find a huge DC infrastructure already in place since a lot of telco equipment runs on DC.

Personally I think BOTH the power & the cooling needs to be addressed. I've worked in datacenters where cabinets are filled with 30+ 1U servers. Not only is it a royal pain in the ass to deal with all the power cabling for 30 individual servers but the heat blowing out the back of those cabinets is enough to melt the polar ice caps...

I've also worked on blade servers like IBM's BladeCenter. Their next generation of blades will require even more power than it currently does. Trying to convince a datacenter to run 4 208 volt feeds to handle just a pair of BladeCenters (28 blades) is like pulling teeth. They can't comprehend that much power in such a small footprint. A rack full of BladeCenters could easily require 8 208 volt feeds, whereas a rack full of 1U's may only need 3 or 4 110 volt feeds.

Re:Why not neither? Remove the power supplies.

[fdawg]

And then on an unusually humid day, as you're walking past this DC infrastructure, you witness first hand why the power company uses AC.

I can almost smell the burnt flesh.

Re:Why not neither? Remove the power supplies.

[JVert]

I now pronounce you as unqualified to operate telephone. Nobody is going to buy your laptop with the AC/DC transformar built into the case.

But yes, I belive it is absolutly impossible to run 500 watts at 12 volts for any application. I hand crank my car for this very reason.

Re:Why not neither? Remove the power supplies.

[Iphtashu Fitz]

And then on an unusually humid day, as you're walking past this DC infrastructure, you witness first hand why the power company uses AC.

If it gets that humid inside one of these types of datacenters then all that DC current is the least of your problems. It means that so many of the HVAC's have died that the ambient temperature is well above acceptable levels and servers are probably frying in their own heat.

Re:Why not neither? Remove the power supplies.

[timster]

Heh, I'm certainly entirely unqualified to operate telephone infrastructure. I'll grant you that though I'm not sure what relevance it has.

As for a laptop -- well, we were discussing servers, I thought. Servers are different from laptops, obviously. However, I do happen to have a laptop in my filing cabinet that does in fact have its transformer built in to the case -- Compaq made laptops like that, years ago. They are a little too bulky, though it is a nice convenience that you can use a standard power cord.

I think you misunderstood me about volts -- of course it's entirely possible to run 500 watts at 12 volts, and I know how a car battery works. But it would be a bad idea to try to run power to servers at 12 volts -- we are talking some seriously thick power cables.

That's why when people run DC to servers it is at a higher voltage, and in any case the servers still need power supplies to provide the different voltages for various components. And yeah, there are reasons to do that, but they don't solve heat problems by any stretch. The OP was suggesting running 12VDC directly to the *motherboard*, which I still say is impractical.

Re:Why not neither? Remove the power supplies.

[Matt_R]

I have an old Compaq 486 laptop that has the transformer built in.

Re:Why not neither? Remove the power supplies.

[Critical Facilities]

UPS's do not run exclusively off of DC. You are correct that they convert AC to DC, then route it through the battery strings, then invert it back to AC current. While this does generate some heat, it is NOTHING compared to the server racks. I've worked in datacenter environments for several years now, and I can say that one of the biggest foes to efficient cooling is poor space planning.

I've never seen people so difficult to communicate with as hardware planning people. You would be amazed at how much better a computer room gets cooled when the computer equipment gets installed properly in a "hot aisle/cold aisle" configuration. Also, vendors and hardware folks don't like to have things pointed out that they're not doing, like making sure not to install a top discharge cabinet on the edge of a cold aisle right next to a front intake cabinet, or installing plenums inside the cabinets as some vendors recommend.

A combination of good space/hardware planning as well as honesty and communication in determining potential heat loads are probably the 2 biggest factors in keeping a computer space cold, IMHO. No one's being helped by just guessing as what a rack full of SunFire servers is going to put out in terms of heat, find out from the manufacturer. And don't feel that your engineering staff is trying to tell you how to do your job or piss you off by letting you know that a rack you've installed is disrupting airflow. We're all in this together, remember?

Re:Why not neither? Remove the power supplies.

[Anonymous Coward]

"I don't think you can (reasonably) run a 500-watt power line at 12 volts"

Umm..better lookup OHMs law....and lookup step up/step down transformers...

Re:Why not neither? Remove the power supplies.

[timster]

Try looking up "reasonable", and take a look at a 50-amp power cord.

Ecobay!

[limabone]

Sanmina-SCI (yes I work for them, not in sales though) has a very nice solution for cooling:

http://www.ecobay.com/catalogs/enc_catalog/ecobay. html

The A/C company brought our water

[Roadkills-R-Us]

Bob was changing backup tapes when something caught his eye at his feet. Looking through te holes in the cooling tile in the raised floor, something was moving, like a bundle of shiny snakes. Looking closer, we had 1/2" of water down there!

We spent several hours with a tiny shop vac (we need a bigger one!) emptying the water and being thankful Bob had seen it before it got high enough to get into the power conduits.

An A/C unit drain pan had a clogged drain, so the sump pump couldn't carry the water away. Whoever had the units installed had purchased water alarms, but *they had never been hooked up*. Now *that* was a brilliant move.

We now have water alarms down there.

Meanwhile, the room stays about 70 degrees, and the servers stay comfy, as do we. I like it that way,

Re:The A/C company brought our water

[Ubergrendle]

I can top that...

Heat exchange liquid coolers pumps are mounted on the top of the datacentre intended to cool about 6 fully loaded MVS systems in the late 1980s.

Oh, did I mention this was in Canada?

Christmas eve... -40C outside...liquid gell in the coolers begins to congeal in the exchangers on the roof, soon becoming a frozen block of ice. So guess what happens? Yes, 6 MVS systems fail in a cascade over about 20 minutes due to overheating... pages go out to datacentre staff like gangbusters on Christmas eve. Imagine at 11pm at night trying to explain to the bank president (CIO was out of town) why you're having overheating problems on the coldest day of the year!

Re:The A/C company brought our water

[Gigabit Switchman]

Disclaimer: Yes, I work for a company that makes temperature, humidity, and leak sensing equipment for IT. Sensatronics

I think trying to convince server/CPU vendors to produce cooler products is futile... so long as the server room cooling is available, it makes sense to take advantage of it if it gets you more computing power per cubic foot/per U. If it were economically THAT much cheaper not to provide so much cooling, then hosting providers/rackspace sellers would probably offer reduced-price space for low-power servers. Does anyone know of a provider that gives discounts (or conversely charges fees on the other end) for low-heat colocation space? I haven't seen it.

On the disaster story side, I still remember watching the IT guys at a previous employer wander by the window to the server room periodically and check the thermometer in the window... now that I work for a company that sells cheap networked devices to do the checking automagically (and alarms on overtemp) I wonder how any IT department can justify something like that... it seems potentially disastrous, yet it was a large company I worked for, with (theoretically) experienced IT guys. Does anyone else see silliness like this where they work?

Have always been there

[Anonymous Coward]

I worked once in a Unisys shop. Their mainframe CPUs have been water-cooled for something like 20 years. [IBM's mainframes are also water-cooled, but IBM mainframes are much more common than Unisys mainframes.]

Currently they are moving all applications from green-screen to HTTP(generate new HTML-based screen definitions, recompile and save on mainframe web server). Once it's done they'll have one of the fastest web servers in the world.

So it seems that the world is returning to centralized mainframes inside data centers. Like one guy said, "I don't know what kind of computer we'll be using in 20 years, but it's color will be blue."

it's called an "on line" UPS...

[SuperBanana]

. And your battery backup infact does not convert AC to DC then back again, it has two seperate paths a direct AC path then another path for AC to DC, yes it does have a DC to AC converter, but that is only used during power failure.

Many UPS's DO convert to DC and back again- they're called "on line" UPS's, and they are by far the dominant type of UPS.

Of course, you'd know this if you had any datacenter/IT experience, instead of being a college student.

Re:it's called an "on line" UPS...

[MindStalker]

Sorry your right, been a while sense I've really looked into it. I'm more of a it works for me kinda guy. BTW I do have datacenter experience, kinda scary hu? :)

Just too HOT! - Geekcorps challenge!

[Thrazzle]

You guys are just too spoiled. :)

Take a look at what the hardworking boys and girls at GeekCorps are doing and put your skilz where your mouth is!

Geekcorps Mali Heat Sink Contest [top]

3/1/2005 - 6/1/2005 | contest-at-mali.geekcorps.org | Link
Geekcorps Mali has a problem, a hot problem. We are installing computers in many community radio stations across Mali, West Africa home of the famous Timbuktu and the Sahara Desert, and as you might imagine, it's hot.
http://www.geekcorps.org/default.asp?l1i=2&l2i=119 &l3i=453#453
http://www.geekcorps.org

Can someone tell me what's wrong with...

[pulse2600]

...having a data center with completely open air racks/systems? No panels or doors on racks, and eliminate or greatly reduce the case of each server itself. Run some sort of ducts or conduit between each set of racks that directs air onto the boards of each server from the room's cooling system. I'm sure it would reduce the cost of manufacturing each server and rack, and the machines get all the cold air from controlling the temperature of the room and air gently blowing onto the components themselves via the network of ducts distributed throughout each rack.

Re:Can someone tell me what's wrong with...

[Anonymous Coward]

You'd actually be amazed. I fried a Sun system a couple of years ago by leaving the side plate off it. In a 68 degree room. Next to an upvent in the floor.

And more recently, I was witness to an HPaq 360 that nuked itself because some took a failed drive out, and left the empty space there instead of putting in the drive spacer. The cooling on most systems is a very well engineered system, and it doesn't take very much "modification" like leaving open cases to damage the balance of air flow that keeps systems running.

Re:Can someone tell me what's wrong with...

[Detritus]

For starters, you just destroyed the EMI/RFI protection and certification of all your systems.

With a properly designed equipment room, the rack is part of the ducting for the cold air. It enters the rack through a set of louvers in the floor.

Re:Can someone tell me what's wrong with...

[Anonymous Coward]

Given it's druthers, air will follow the path of least resistance. Unless you actually force the air into those little spaces (between the boards or the fins of the heatsinks) the air will simply go around them. The only reason that passive finned heatsinks work at all is that they generate convection by heating the air between the fins.

This is actually the biggest problem with the whole "cool room" argument: they're using a bonfire solution to a sweater problem. Ultimately these guys are not trying to remove the heat from a bunch of 16"x19"x6" pieces of equipment but from a similar number of 0.5"x0.5"x0.2" components. If the CPUs weren't so darn small, the problems would be relatively easy to solve.

Re:Can someone tell me what's wrong with...

[pulse2600]

You'd actually be amazed. I fried a Sun system a couple of years ago by leaving the side plate off it. In a 68 degree room. Next to an upvent in the floor.

But that's not what I'm trying to suggest...I'm thinking more along the lines of an "air irrigation" system where air is aimed directly into or at server components. Who cares about airflow if a constant supply of forced cool air is being shot at or into the components/boards/etc? Would the system still fry if I had a hose running from an HVAC system right into the server?

and that voltage loss = ?

[SuperBanana]

This means that if you produce regulated 5V at one side of your datacentre, by the time it's reached the other side it's not 5V any more. But it should be easy to get round this by producing 6V and having DC regulators; they're very small and extremely efficient these days.

...aaaaaand where do you think that energy goes?

[DING] "Heat, Alex" "Correct, for $100."

...aaaaaand what do you think that energy loss thanks to high current means?

[DING] "Efficiency less than a modern AC->DC power supply" "Correct, for $200."

Anyone particpating in the "DC versus AC" discussion would do well to pick up a history book and read about Westinghouse and Edison. There's a reason we use A/C everywhere except for very short hauls. Modern switching power supplies are very efficient and still the best choice for this sort of stuff.

Re:and that voltage loss = ?

[kesuki]

However, the argument for putting the battery backup directly into an power supply, in unaffected by this statement.... also, having an atx supply with a DC input, and an external UPS with DC output with a cord shorter than 10 meters also has minimal issues.
Also, by having the conversion process take place in the UPS you are shifting 20% of the heat generated by a modern PC away from the enclosure, and putting it into an external device. (compare devices like the PS1 and PS2 which have an internal converter, and early models had serious overheating issues, and other consoles that decided to place the conversion in an external device, even if that converter is on the console connection side, it removes heat from inside the console, and lets it dissipate outside the console.)

Re:and that voltage loss = ?

[cirisme]

>>[DING] "Heat, Alex" "Correct, for $100."

Incorrect, you didn't phrase your answer in the form of a question. :)

Re:and that voltage loss = ?

[Noehre]

You'll note that we now use high-voltage DC power lines in some places. High-voltage DC-DC conversion has made great advances since the time of Westinghouse and Edison.

It would not be difficult, I imagine, to run 100V DC throughout a home or datacenter. From what I can gather, DC-DC converters in this range have efficiencies around 90%. Even good PC power supplies only have 80-85% efficiencies (besides Antec's fanless 90%+ effienciency fanless supply).

Re:and that voltage loss = ?

[gnuman99]

There's a reason we use A/C everywhere except for very short hauls.

Well, except for the 1,000,000V@3600A DC line. http://www.hydro.mb.ca/our_facilities/ts_nelson.sh tml

Reason? RF losses.

Re:and that voltage loss = ?

[drinkypoo]

DC sucks any time you do not have very high voltage. However, if it's not very long, it sucks less. The higher the voltage, the further you can run DC without significant loss.

Re:and that voltage loss = ?

[njh]

What the hell are you talking about? DC carries more power for a given conductor size and peak voltage (hence RMS). The only significant advantage of AC (besides historical reasons) is the ability to change voltage to current with a simple transformer. As consumer devices are tending towards switchmode power supplies (which generate their own AC on demand and at a much more suitable frequency) DC is actually more useful as it avoids requiring power factor correction.

Almost everything in your house would be better served with DC - lights (compact fluoros rectify the current and produce their own higher freq AC, e.g.), heating, motors, computers, etc.

DC at high voltages are much harder to deal with than AC because you need very high voltage transistors and diodes to deal with them. AC only requires a suitable transformer and isolation. DC is used because it avoids inductive and capacitive losses.

Re:and that voltage loss = ?

[inKubus]

Yeah, we ARE talking about rack stuff here. It'd wouldn't be a big deal to have one power supply per rack, top mounted, and then a 3U or 6U of battery in the bottom and a wiring harness that hooks up to each server. but then you'd have a blade system.

One PSU to connect them all

[Anonymous Coward]

I'm thinking of running external power jacks off of my computer's PSU and running some of my external devices off of that, mostly external disk drives. It will be interesting to wire up custom connectors to do all this though.

Both

[Anonymous Coward]

Honestly the best way to do it would be both.
You could use rackmounts that are encased in a reciever for heatpipes ... have those heatpipes run to a place where water can flow through and cool'em down (AWAY FROM THE EQUIPMENT). This is a very vague and rough description of what I'm imagining--but you get the idea.
if ($_SERVERTEMP eq 'cool') {
$_ROOMTEMP = 'cool';
}
else {
print 'Wow, Slashdot just about got me fired for typing this'\n;
}

water ot air conditionning

[chrisnewbie]

I dont think water is an option for cooling servers, water damage would be too costly at every level unless it can never spill out (never say never).Out of control air conditioning will what....lower the temperature under 10 degrees at the most unless you really miscalculated the B.T.U's of the machine itslef.

Re:water ot air conditionning

[chrisnewbie]

For sure cooling at cpu level is way better than room temperature but what if!!!i know it's an all together different game but a friend of mine tried cooling his own p.c. at home and he flooded his basement. i prefer installing 4HP fan on the side of my computer....keeps the dust out too! :)hehehe

Re:water ot air conditionning

[ckaminski]

I can attest to water from a broken AC condensor flooding our data center. Luckily there wasn't a rack directly underneath it, or we could have had a serious issue (from simple shorts to outright fire), and we only had water extinguishers, how nasty would THAT have been?

We actually had a cascade failure as one unit broke, then another as the system tried to pick up the slack. Now we had two puddles. Luckily this was on a weeknight, so we caught it before it could sit for a weekend. Putting liquid cooling in servers isn't all that nasty an idea.

Coffee Cooling/Brewing?

[sickmtbnutcase]

You think with enough coffee-pounding geeks around, we could figure out how to engineer a water-cooling solution that would plumb the heated water through the workplace's coffee makers. It would keep the systems cool, and the coffee addicts happy!

APC agrees with you

[Roadkills-R-Us]

This is part of the basis for their InfraStruxure (or however they spell it). It looks like a great idea, but since we're a startup without 1999 type funding, we'll have to wait a while on trying it.

I have no relationship with APC other than beinga very happy user of their UPSes, both at home and at work (everything from outlet protection to Symmetra 4KVA systems, looking forward to the day we can try the bigger stuff).

Re:APC agrees with you

[Thundersnatch]

...but since we're a startup without 1999 type funding...

If you have 1999-type funding, you can afford just about anything. Do you have a bunch of those neato Aeron chairs? An open floorplan layout with no cubicles and a bunch of water fountains and shit? And a Foosball table in your main datacenter?

If not, you don't really have 1999-type funding ;-)

Re:APC agrees with you

[Thundersnatch]

Of course, after posting, I realize that I misread "without 199 type funding" as "with 1999 type funding". So please ignore me.

ETH Zurich used to re-sell their heat

[toby]

Heat from the large data centre at ETH used to be piped and sold to nearby users. I think the practice has been discontinued but it was a neat idea.

er..

[Anonymous Coward]

Since the server is the thing that is affected by the temperature, the server is the thing i want to keep cool. The room can be at a hundred degrees but, as long as the sever is cool, i'm happy. To keep the server cool, however, i'm willing to cool the room.

Easier to transform?

[bradleyland]

Isn't AC current also much more easily/efficiently transformed when compared to pulsed DC? The reversal of current flow causes a field inversion in the transformer, resulting in more efficient transformation?

I know not of what I speak. I'm just spitting out my best recollection stuff that I remember from working someone who knew a lot more than me.

Water-chilled Cabinets Already In Use for Blades

[miller60]

Liebert and Sanmina have been selling blade server cabinets that use chilled water for at least three years. Vendors and data center operators have been wrestling with the heat loads generated by blade servers since 2001, and the dilemma of how to cool high-density "hot spots" has caused many tech companies to wait on buying blades to replace their larger servers. That's changing now, driven by the need to save costs with more efficient use of data center space.

The industry has taken a two-pronged approach. Equipment vendors have been developing cabinets with built-in cooling, while design consultants try to reconfigure raised-floor data center space to circulate air more efficiently. The problem usually isn't cooling the air, but directing the cooled air through the cabinet properly.

There was an excellent discussion of this problem last year at Data Center World in Las Vegas. As enterprises finally start to consolidate servers and adopt blade serves (which were overhyped for years), many are finding their data centers simply aren't designed to handle the "hot spots" created by cabinets chock full of blades. Facilities with lower ceilings are particularly difficult to reconfigure. The additional cooling demand usually means higher raised floors, which leaves less space to properly recirculate the air above the cabinets. Some data center engineers refer to this as "irreversibility" - data center design problems that can't be corrected in the physical space available. This was less of an issue a few years back, when there was tons of decent quality data center space available for a song from bankrupt telcos and colo companies. But companies who built their own data centers just before blades became the rage are finding this a problem.

And the winner is !!!

[eheldreth]

Apple, at least with thier powermac and powerbook(yes a water cooled laptop) line of computers. Apple is to start wattercooling the upper end systems.

IBM

[Hoonis]

IBM has mentioned that they are working on water cooling.. a solution described to me as "a large radiator that replaces the back door of the rack".

I thought... eeeek!

gonna have to quit

[Rize]

Some of us work in the same room as this junk you know. I already wear a coat all year long. If it gets any colder I fucking quit!

Even with

[Anonymous Coward]

Then there is the anti-fire supression problem:

A data center I am very familiar with, was conducting an annual test of the fire system in the data center. Unfortunately, those who were most familiar with the integreation of the system had been let go in a previously layoff. When the test was started, the UPS's were not bypassed thus causing the whole place (Datacenter and administrative/managment offices)to go dark except for emergency lights.

Thankfully, the fire supression system di not kick in or we would have had a rather tropically humid server farm.

Bringing up hundreds of Wintel/unix systems and a mainframe and a rather large Teradata system was not a task I would wish on any Data Center Manager (nor the engineers who had to fix the stuff that wouldn't be coaxed back online). [sigh]

Re:Even with

[devilspgd]

Then there is the anti-fire supression problem

I believe that would be a "fire suppression system" or an "anti-fire system" -- You don't really need an anti-fire suppression system, you could save yourself a lot of money and not bother installing a fire suppression system in the first place.

Why not water-cooled racks?

[Samrobb]

Build a standard rack with hollow piping, and pipe cold water through the rack itself. Use it as a huge heat sink for the servers - heck, you could probably figure out a way to make a direct connection from the rack to the CPUs with a strip of copper or aluminum. At the very least, you end up with the servers at least partially in contact with a big, cold, water-cooled hunk of metal. Get additional eco-points by using the now-warmed water as input for the building's hot water heater.

Whoops! You say that it's already been done? Dang. Just when I thought I was smart...

Re:Why not water-cooled racks?

[njh]

Wouldn't heatpipes be a lighter and less leaksome solution?

Re:Why not water-cooled racks?

[Samrobb]

Heatpipes? First I've ever heard of the term... what are they?

Re:Why not water-cooled racks?

[njh]

I asked google nicely and it said:

http://www.heatpipe.com/heatpipes.htm

They're really cool aren't they! :)

Sort of like vapourphase but without any energy input. Combine with something like coolchips and you have a really nifty new area in the ancient science of thermodynamics.

Cool the Room...

[ImaFraud]

I would say cool the room. Because, without a cooled room your, the fans inside the pc are just pushing around hot air.

A much cheaper alternative...

[mosel-saar-ruwer]

The question is, which makes more financial sense?

From TFA:

Advances in server technology may give data center managers a little room to breathe. Innovations from chip technology to server construction are improving hardware's capability to deal with high-density environments. According to Gordon Haff, analyst with Nashua, N.H.-based Illuminata, the days of using traditional server cooling methods are over. Companies going forward with dense server environments will need to use every little piece to get closer to its goals. Upshot: A cooler servers start with the processor.

First of all, I wouldn't call the idea of packing more and more blades in a hotter and hotter core some kind of an "advance", but we'll let that one slide.

The obvious answer here is to move the CPUs farther apart. But moving them further apart requires more square footage, and square footage is astronomical in places like San Jose and Manhattan. Therefore the following solution starts to make a whole lot of sense:

Homegrown
IT outsourcing options sprout up across rural America.

Welcome to the ever-so-nascent world of rural IT sourcing. Both large and small companies are tapping into a highly skilled but often underemployed IT workforce in lower-cost rural areas--frequently as an alternative to shipping work overseas.

http://www.computerworld.com/printthis/2005/0,4814 ,100632,00.html

And in the middle of nowhere, square footage is essentially free.

the solution is legion

[RapmasterT]

This problem has been a forfront issue for me in my job for the last couple of years, so understand that nothing I have to say on it is based on speculation, or an article in a tech mag, it's from the real world.

Heat sinks, air flow, etc does NOT reduce the thermal load of a datacenter, it only improves the efficiency of a single server dumping it's waste heat into the space. THAT is the problem to deal with. Personally I can't afford to give a rats ass about one box's cooling, if there's over a thousand more that all have the same needs. If it doesn't help the datacenter as a whole, it's just trivia.

APC makes some cool air handling cabinets that are sealed and vented to draw in cool air, and vent the hot directly into ducting. very slick, but still that does NOT change the thermal loading of the datacenter.

What to do with the waste heat is the #1 problem, and really the only problem in this discussion. fans and heat sinks are fine to get the waste heat out of the servers, but it has to go someplace.

In a traditional HVAC CRAC environment, hot air is chilled in the unit, waste heat is exchanged into a water supply, which flows out of the room to a chiller unit that pumps the heat out into the external air.

A very interesting alternative I've seen gaining popularity is to use ambient cooling. What that means is if you want to keep your datacenter below "x" degrees, then any time the outside ambient air tempature is below that, you simply open a window. It's a little more complicated, but that's the gist, draw in outside air that's cooler, blow out the hot air. In some locations (Seattle for example) the chillers actually only have to run less than 1% of the time. It's obviously less effective in hot locations, but eliminating the cycle of re-cooling your own hot air is a huge win both for cost and efficiency.

The other alternative is better air handling (ducting, raised floor, etc) combined with better cooling, combined with more chiller water supply. The latter is typically the bottleneck.

water cooling in the datacenter only makes sense IF the heated water is vented to outside the datacenter. A central piping system that goes through the heatsink on each server, and vents outside would be great, but complex and dangerous far beyond the benefits. Water cooling that is inedepentant on each server is just a more efficient way to get waste heat out into the datacenter, and hasn't solved anything.

And the answer is.......

[Vroom_Vroom]

(note - I'm an engineer who does this every day, well on the days i can't avoid it anyway)

No-one will. It's like the question "When will Linux take over the desktop"? (sorry couldn't resist).

Cooling datacenters will always be a hassle. They're always packed into a corner of the building (uber heat build up)and no forethought is given to expansion - so when double the amount of IT is thrown in or the upgraded kit spews up twice the heat load the coolign system splutters.

The one true way of cooling is raised floor cooling, great for cable management and very expandable (move a rack, move a floor tile!)

The comment re the need for liquid cooling is probably based on faulty logic, i mean you can cool air down fairly low and air doesn't leak over equipment or need expensive liquid to run.

Cooler servers

[TrevorB]

Because rarely is the AC ever plugged into the UPS (takes too much power) and most server rooms die during a power failure not due to the UPS running out of power, but because the room overheats and the servers all shut down.

Server rooms can turn into tropical saunas pretty fast. During a power failure we have to get into the office in 40 minutes to start powering down less important servers (try telling management that *all* the servers aren't mission critical, or worse yet, getting them to fork out $$$$$ for a bigger UPS)

Re:Cooler servers

[par38lamp]

In a properly designed data center, the computer are on UPS, and the HVAC is backed up with an emergency generator. Sure, they HVAC will go down for a short while, but once the generator has started, the load it trasferred via an ATS to the genset. Sometimes only half of the HVAC units are on emergency power to provide minimal cooling.

HVAC typically = Liebert precision air conditioning, complete with humidification/de-humidification.

We had water cooled servers

[Anonymous Coward]

But not on purpose. A tornado hit our data center. Shortly thereafter we conducted an experiment with water cooling. Results were not positive.

Measuring Efficency.

[rssrss]

I assume that the question of whether it is better to use cooler chips or to spend more on cooling equipment is one that can be anwered with some degree of objectivity.

There is a computational task which needs to be done. It can be measured in instructions processed. The chips have an up front cost, and it costs so much for the electricity to run them. If mechanical cooling is necessary, the coolers have an upfront cost and also need electricity to run them.

Once you have these facts in hand, all that is required is to calculate the costs over a specified time frame, and present value the cash flows.

IIRC, somebody from Google gave a speech/interview a couple of years ago (which was duly reported on /.), in which he said that he wished chip makers would pay more attention to power usage than speed because his electricity bill was getting way too big.

Note: Raul654 (453029) "Reminds me of an amusing anecdote." I assume the machine in question was the one described in the comments above titled "Cray still has water cooling!"

Cool the racks instead...

[Otto]

Of the many server rooms I've been in, the most effective cooling I've seen has been to enclose the racks into sealed cabinets (adding a cheapish layer of physical security as well, by locking the things) and then piping cooled air directly into the top of the cabinets.

If you buy your own racks to put gear in, then getting these things is easy, if you buy whole racks from a vendor with gear in it already (custom systems type of thing), then the thing comes in a cabinet which usually has some kind of a fan/vent arrangement on top. Rip that off, attach some ducting straight up to the ducts running across the rows, and voila, cool air flows straight down and out the bottom.

All you need is to build your room with several ducts running across the ceiling, with removable plates every so often. The AC system pushes air into that, which then goes directly into the racks. You don't even need to cool the room really, since the air coming out of the racks gets cool enough to keep the room itself cool. The servers in the racks stay at fairly chilly temp in there. Only downside is when you need to open one, you're hit in the face with this freezing air pouring out of the rack. :)

Re:Cool the racks instead...

[illtud]

Of the many server rooms I've been in, the most effective cooling I've seen has been to enclose the racks into sealed cabinets (adding a cheapish layer of physical security as well, by locking the things) and then piping cooled air directly into the top of the cabinets.

You sure? Cool air in the *top*? All the ones I've seen (and all the rack equipment manufacturers accessories) pull cool air from under the raised floor and pull it *up* through the rack. This is because the hot air your systems are exhausting is already rising, and pulling the cool air up and exhausting at the top makes a lot more sense!

Re:Cool the racks instead...

[Otto]

You sure? Cool air in the *top*? All the ones I've seen (and all the rack equipment manufacturers accessories) pull cool air from under the raised floor and pull it *up* through the rack. This is because the hot air your systems are exhausting is already rising, and pulling the cool air up and exhausting at the top makes a lot more sense!

You might be right, but I know what I saw and it worked incredibly well.

Sucking the hot air out the top would certainly work, but feeding cool air into the bottom is not a particularly easy thing to do for most racks. You'd basically need to cool the whole room then, and that seems cost less effective than cooling the racks only.

Computers become obsolete

[EraserMouseMan]

Duh. So invest your cooling resources in the room. Then when you need to replace your hardware you don't have to throw away your cooling investment with it. Unless you don't plan to use the same room forever. Then invest in cooling the box so you can take your cooling investment with you (and leave the room behind). So, large Datacenters should cool the room. Small businesses should cool the box. There.

Re:Ma Bell has been doing this for years

[El]

Telecom equipment runs off -48VDC Why not just use +48VDC and switch the red and black wires? (Yes, I do know there is such a thing as a chasis ground.)

Apple XServes

[plsuh]

It's kinda interesting to read about this -- one of the big points in selling XServes has been that they run cooler than Xeon-based 1U servers. You can actually pack a rack full of 42 XServes safely, which you can't do with many other 1U servers. In many cases with other 1U servers you have to leave every other or every third space open so that the servers don't fry themselves. This makes the XServe a lot cheaper as you need 33% or 50% less rack space.

--Paul

Re:Apple XServes

[bani]

apple quotes their g5s at 55W.

however AMD is now producing low power opterons which match this.

Cooler Servers for sure

[sk8king]

We need to do everything to reduce the power required for all our electronic gear. More powerful servers [computational wise] require more power [electricity wise] which then requires more power [electricity wise] to power the air conditioning. If we could get server that somehow consumed less power [a lot less] we would win on two fronts.

Re:Cooler Servers for sure

[SlothB77]

Forget SUV's, our servers are supposedly causing global warming! Now this kind of thinking is what really stifles innovation - we have to bend over backward to make less powerful servers to appease some phenomenon that is not really happening.

Of course it's like pulling teeth...

[PornMaster]

Most internet data centers are not equipped with the cooling to handle customers with racks full of blades. A rack of HP BL40p blades puts out 55000 BTUs. A tier 1 data center in which I've worked was designed to cool 5000 BTUs per rack. While blades are pushed as a way to save space by increasing computing density, the amount of cooling per square foot of data center space, unless it has specifically been designed for blades, is rarely sufficient for cooling them. The aforementioned data center has more power available than it can cool. When sales gets its way (as it almost always does), more power is delivered to customer per square foot than the data center was designed for, then the room warms up, and everyone bitches.

It is right and proper for a data center to make it difficult to allow a customer to go that far out of the specs that the data center can support, or it'll negatively impact the other customers in the room. If they can't meet your needs, it's better to look elsewhere than to go that far out of engineering spec.

Answer

[nickirelan]

"Both" - IRKLE

Done both.

[CBob]

Worked in datacenters w/air & watercooled mainframes. IBM 9121 & 3083, an elderly Tandem NSII, S38's, AS/400's, most of the VAX/Alpha family & assorted blades/pizza boxes.

The water cooled units also sat in cool rooms because they shared them w/their air cooled DASD. The air cooled systems, need the same cool rooms.

My current entrapment has 2x 50kva UPS and 2x BIG Liebert air units in the room w/all the air cooled servers running from a cobwebby PowerStation to a badly configured Alpha cluster & finishing up the the Sharks & the b80 & P-series.

It's 65 degrees F in there & loud. I miss water cooling, it did make things a little less earful.

HA! It's obvious...

[Anonymous Coward]

I can't believe no one has said...

In soviet russia, server room cools you!

Big Deal?

[Jozer99]

I have had water in my basement server room for weeks, since the heavy spring rains have started. My servers are plently cool, as a result.

Re:Ma Bell has been doing this for years

[fdawg]

Telco equipment that you're referring to is for end to end comm. The current requirements is almost nil. A server requires significantly more power than the ringer on your telephone.

Brainless

[mstansberry]

Funny thing, when I posted this story, I'd posted the "track changes" version. So basically, my most-read story so far was full of typos.

High density ( 52 KW per rack )

[fuzzy1]

The only effective way to cool 52 KW racks
is with a cooler in front of the rack -
This unit requires a 15 ton capacity coil,
but can be lined up side by side. Each rack stays cool.

Smaller units work the same way -For example- put a 4 ton coil in front of a rack of 40 opterons - cold air in the front regardless of the room temperature. 80 opterons / 8 ton coil The computers provide the fan - no crac unit needed -

Size the cooler for the rack, hook it to the cold water, - repeat as needed.

This adds 8 to 14 inches to rack depth - and has no hot spots.

email has changed - rcbondsr@gmail.com

For license information - University of Washington Technology Transfer.

Bigger computers - bigger coil - one rack at a time I can cool any installation you have !

I am going to roll out the first installation this summer.

From my patent application -

Background of the Invention
[003]Computers are sometimes cooled by cooling the air in the room in which the computers are located. A typical cooling system cools air and moves it through the room with the devices in the room that need to be cooled. When air is used as the cooling medium, variations in airflow occur, particularly when the heat density rises in a region of the equipment room, or when the absolute heat load approaches the maximum load that the air can handle. In an effort to solve resulting problems, systems have been made in which the devices that heat up are placed inside of a closure and the air inside the enclosure is cooled. These systems have been found to be inadequate when the heat density is above about 8 Kw. None of the existing systems are able to effectively operate in an environment in which the heat density is between about 20 to about 40 Kw. Yet, manufacturers are starting to make computer equipment in which that much power exists in the system. Currently, when the heat density is high, the systems are provided with greater floor space and larger air handlers and chillers. This approach has led to the creation of "hot spots" in the equipment. The known systems fail when the power level raises to about 400 watts per square foot, or when the cooling requirements vary substantially in a given space.
[004]When airflow in a single rack approaches about 3,000 cubic feet per minute, and an aisle of about 20 racks approaches 52,000 cubic feet per minute, the conventional systems cannot handle the airflow in a computer room of conventional size. The use of larger rooms is expensive and they are still subject to the airflow problems that are created. These problems include the creation of "hot spots" which are regions in the room that are not sufficiently cooled and in which the devices that generate the heat are adversely affected by the heat. There is a need for a cooling system that avoids the problems of the prior art systems and which eliminates the "hot spots". A principal object of the present invention is to fulfill this need.

ABSTRACT
A device that in use generates heat is positioned within an enclosed space that includes an ambient air inlet, an outlet and an air mover for moving ambient air through the space from the ambient air inlet to the outlet. A cooler comprising coils and passage ways defined by and between the coils to which ambient air moves from the inlet of the cooler to the outlet of the cooler. Position in the cooler with its outlet in register with the ambient air inlet for the enclosed space. Using the cooler to cool the ambient air that is immediately forwardly of the ambient air inlet for the enclosed space. Using an air mover in the enclosed space for moving the cooled ambient air into the ambient air inlet, through the enclosed space, and out from the outlet of the enclosed space.

rcb

already vendors

[Anonymous Coward]

This is already being done by at least one vendor.

http://www.liebert.com/dynamic/displayproduct.asp? id=1077&cycles=60hz

Liebert cools the Virginia Tech XServe cluster.

From http://www.tcf.vt.edu/faq.html

Q: How much heat does System X generate?
A: Each rack full of equipment generates in excess of 8 kilowatts of heat on a continuous basis and the machine consists of about 40 racks. That's like 240 hair dryers on high constantly in 1000 square feet of space.

Q: How is System X cooled?
A: System X uses a Liebert Extreme Density cooling system that is fed off of a chilled water loop. There are two 125 ton Carrier water chillers that provide roughly 3 million BTUs of cooling capacity. This chilled water loop is heat exchanged in the Liebert XDP units with a R-134A refrigerant loop that is fed to the rack mounted liquid-to-air heat exchangers. We only use about 110 tons of the 250 ton capacity.

Re:Ma Bell has been doing this for years

[gnuman99]

Yeah! Touch the chasis, it zaps you and reboots the box :)

Re:Ma Bell has been doing this for years

[Anonymous Coward]

Telco equipment that you're referring to is for end to end comm. The current requirements is almost nil.

Don't forget the switching equipment. Powering the ringer doesn't mean anything if your switching equipment isn't powered.

Effect of cooler rooms on people

[oncee]

Cooler rooms sounds wonderful until you spend 12 hours working in one.

http://oncee.blogspot.com/

Obligatory history lesson

[emurphy42]

Crunchly meets the Water-Powered Computer:

73-06-04 (glitch)
73-07-24 (overflow bit)
73-07-29 (bug)
73-10-31 (ad-hockery)
74-08-18 (water MIPS)
74-12-25 (number-crunching)
74-12-29 (winged comments)
75-10-04 (bit bucket)
76-02-14 (washing machine)
5678 (flush)
76-07-18 (core dump)

Re:Aquafina... Cow-Worker Story

[tengu1sd]

I walked into our server room after lunch one day. Raised floor from the IBM system that got scrapped years ago, a/c and generally nicely laid out. The a/c drain had backed up and one of our help desk guys was standing in the puddle with a mop and 220 power cables. I had to shout at this guy to drop everything and get away. After a modestly quick shutdown and power shutoff at the breaker panel we started to dry things out. I still don't believe that someone would stand on power cables to mop up a spill. Definate Darwin candidate here.

This guy was a sleazy tomcat'ing sort too. No way was I going to risk CPR with that clown.

A rather embarrasing story...

[wandazulu]

But explains why I'm all for cooler rooms, and not water cooled machines.

I worked in an office building on the floor right above one of the largest mainframes in the world. The machine was water cooled from the same set of pipes that supplied water to the bathrooms on that side of the building. For reasons not explained very well to me, there could not be a bathroom on that side of the floor because water pressure would be too low, on account of the computer using it all.

Now:

Guess what side I worked on.

Guess when I discovered my stomach didn't handle extremely spicy food.

Guess where the bathroom was on my floor.

Guess what happened on the way to get to it.

So improving cooling technology to prevent that "situation" from occuring again is a good thing to me (I suppose I should just not eat spicy food, but where's the fun in that?)

Re:Cooler servers, definitely (No, hotter servers)

[Anonymous Coward]

You need hotter servers. It's easier to get rid of the heat if it's hotter. If the machines could take 140 F say, you'd hardly need A/C (just filtering).
A bit hard on the personal though...

SGI already did it at NASA with the altix cluster

[YeeHarr]

http://www.serverwatch.com/hreviews/article.php/34 36001

a search on google for nasa altix water back will give you lots of hits.

Also we have one for eval in our data center from IBM.

So it's already here really.

The question isn't which vendor will do it, it's which customers want to put in the water cooling infrastructure needed.

Green Computing

[AutumnLeaf]

Cooler servers and cooler rooms are the answer, ideally with cooler rooms being the result of cooler servers.

The issue of computing with respect to its impact on the environment hasn't gotten too much notice. On a bedroom-level scale the biggest concern is heat. Data-centers with tens of thousands of 1-U systems and fileservers packed into 40-machine-per-tiles densities can become a very taxing drain on the power infrastructure. When you think about what's being consumed on the other end of the wire to generate that electricity, at that scale green computing becomes something to think about.

Re:Ma Bell has been doing this for years

[bani]

actually no. cisco makes heavy duty routers which take 48VDC. they eat lots of current, easily more than your typical server.

for example the as5800 and mgx 8200 have options to run off 48vdc. the mgx 8200 power requirements state 1000W !

Central ventilation?

[Java Ape]

I've read several good posts dealing with the potential heat/energy savings of centralizing AC/DC conversion. I have a somewhat related question -- Why don't server rooms use a central vaccuum or blower? Our server rooms are filled with nice racks, which are filled with all the usual equipment, most of which is trending toward single or double-height units with lots of little high-speed fans. The noise puts me into a stupor every time I go in there. It seems like it would be MUCH more efficient to put a single commercial fan outside, and pipe either presssurized air or vaccuum to the individual units. This would also provide the advantage of being easy to monitor. After all, how many of you have lost equipment because a couple of those stupid little fans quit working? There are probably 300 of those little things going in our large server room -- I never notice when one dies unless the equipment it's attached to throws a warning . . . Any comments?

Solution...

[Anonymous Coward]

Outsourcing to Siberia

Re:Ma Bell has been doing this for years

[kent_eh]

Telco equipment that you're referring to is for end to end comm. The current requirements is almost nil

I'm sitting in the control room of a cellular telephone switching office right now. If I slide over to the power plant monitor... Hmmm let's see, we are currently drawing 540 amps thru the -48V system, and 180 amps thru the +24V system. Almost nil, indeed.

While most of that is being used to power the telephone switches and cellular radios, there are also a few racks of Cisco and Juniper networking gear also being fed directly from the -48V.

So, to the grandparent poster, it can be done, and it is being done.

Some are using water!

[fred9653]

The people at the Tier 1 LHC computing center GridKA in germany are already using a water-cooled components. from: http://lcg.web.cern.ch/LCG/peb/MoU/IWR-Rep-GridKa0 022-v1%201-LCGPlanningPhaseII-160704.htm "GridKa developed an efficient, modular cooling concept based on closed water-cooled cabinets. ..."

water in the datacenter....

[Anonymous Coward]

The company I work for had water in the datacenter... not for cooling but dripping from the celing right into our EMC disk array. Man was that a mess...

Re:Water-chilled Cabinets Already In Use for Blade

[CommonModeNoise]

IMHO the essential problem is not getting cold in but heat out. In most data center cooling schemes the exhaust hot air is allowed to mix with the cool room air; thermodynamically, this is a big mstake. What is lacking is simply a set of hot air ducts that can remove the hot exhaust air from each rack and expel it from the building (it will almost always be hotter than the outside air, so it makes more sense to cool ambient external air with heat pumps than to recool the exhaust air). If one designs one's data center this way, one does not even require a raised floor, since the entire room can now act as the cold air supply plenum.

Halon Alternatives, Redux

[nemowho]

The Processor newspaper/trade rag had an article the other month on future/developing datacenter fire supression goops and gases. A brief, but interesting read: http://firegoops.notlong.com/ (The processor URL length was ungodly) An amusing sidenote is that when I was in high school, I had a mad-scientist/BBS SysOp friend who was hell-bent on developing a Halon-based theft deterent system for his car. Once the thief had hopped in and began to drive... (Shudder)

Be aware of APC recalls, though!

[nemowho]

I am a longtime (satisfied) user of APC products as well, but have made it a habit of checking for recalls on their products recently. Overheating/miswired units number in the hundeds of thousands, so take a minute and run the model numbers of your new APC gear past Google. *zap!*

Re:Ma Bell has been doing this for years

[Anonymous Coward]

Any general purpose power supply that grounds one of its outputs internally is not worth buying. The GP is quite correct.

Bring water back?

[AntiGenX]

For some of us, water never left the datacenter... I know of a few Amdahl mainframes that are still in use in large datacenters. They always required deionized water.

No Need for Cooling...

[badford]

If we had data centers at the north and south poles and long lined circuits back to the 'interior'.

Coolrooms...

[beav007]

We keep our server room at 16C all year round. If the A/C fails, the room ends up in the mid 30's within an hour or so. We occasionally have condensation problems, but in general, the air-con is enough to keep a lid on the temperature...

No matter what you say...

[Anonymous Coward]

cooler rooms do not address the issue of heat disapation in the core semiconductors themselves. Anybody that has studied milspec 217 recognizes the knee that exist at about 100 degrees C: semiconductors that rise to that temperature roughly double the failure rate.

Now, ideally, core temps always track ambient. The problem always comes when the equipment cooling the ambient fails. Server farms that have ambients rise may suffer failures weeks or even months later making it very tough to track cause and effect.

Someone else here put it as "...people should equip every computer with VIA ultra-low-power chips _and_ freeze the server room is silly." I say, "gee, are you dgoing for reliability or performance?" If you want 6 9's reliability, then your only choice is exactly that: "VIA ultra-low-power chips _and_ freeze the server room". If you want performance, then you accept the tradeoff in reliability and let core temps rise.

Sure, you can achieve higher reliability figures with redundacy, but that is a null game, and you reach a limit where every additional redundant unit reduces reliability more than the redundacy enhances it!

Look, the real universe is a series of tradeoffs: no single answer, be it water cooling or redundacy or lower power disapation satisfies every need. If there was one single answer, then we wouldn't need engineers to design optimal solutions!

Move to Canada or a far-north Blue State

[bob frost]

Up toward the 49th parallel and beyond, we can sure use that heat 7-8 months of the year. Why not just locate there and save cooling bills--and recycle the heat to save on heating bills. Simple!

Re:Ma Bell has been doing this for years

[Detritus]

Corrosion (see electrochemistry).

Re:Ma Bell has been doing this for years

[MegaHyster]

Yeah just reverse the polarity of the warp coil...

Oh never mind...

it is not more of cooler rooms

[john_uy]

we are operating a datacenter that can accommodate 36 equipment racks. we have been caught off hand with the new hardware because it eats practically double the power and generate double the heat compared to the previous generation of servers.

i guess getting a cooler server is not quite an option now as servers are getting more dense. we have blade servers installed and it just fills in half of the rack due to the power requirements needed. it eats up a lot of power. you must also have standby power in case power supply fails, imagine a power supply eating 2000watts each and you cannot put more load in each circuit as it will cause a cascaded tripping of the electrical circuit (as one of your power supply fails, all the load transfer to another circuit instead of load balanced, and after the transfer, it will become overloaded and that circuit will trip as well causing load to be transferred again for other circuits until all of your circuits will trip, depending in your design.) our main redundancy is placing it in a different rack with different circuits so tripping will not generate a 100% downtime.

the most important design for removing heat from servers is directing all the cold air into the intake of the servers. we have an experiment where we decreased the temperature of the ambient air to 17 degrees celcius retaining the existing air flow and 20 degrees celcius but directing the air flow to the intake of equipment. the output air of the equipment became cooler, the sensors detected a lower processor, chassis temperature than the 17 degrees ambient. this is much better for the equipment.

we don't do raise floors by the way so we just adjusted the vents from the ducts to where the air will flow to.