Slashdot Mirror
The Risks and Rewards of Warmer Data Centers
1sockchuck writes "The risks and rewards of raising the temperature in the data center were debated last week in several new studies based on real-world testing in Silicon Valley facilities. The verdict: companies can indeed save big money on power costs by running warmer. Cisco Systems expects to save $2 million a year by raising the temperature in its San Jose research labs. But nudge the thermostat too high, and the energy savings can evaporate in a flurry of server fan activity. The new studies added some practical guidance on a trend that has become a hot topic as companies focus on rising power bills in the data center."
If rubbing frozen dirt in my crotch is wrong, I don't want to be right.
1. Get a thermostat you can control with a computer
2. Give the computer inputs of temperature and energy use, and output of heating/cooling
3. Write a program to minimize energy use (genetic algorithm?)
4. Profit!!
Possible problem: do we need to factor in some increased wear & tear on the machines for higher temperatures? That would complicate things.
No they didn't - what they did do is figure out that increased temperature is not correlated to higher failure rates - the failure rates don't magically decrease as it gets hotter.
Here's the link for your review: http://hardware.slashdot.org/story/07/02/18/0420247/Google-Releases-Paper-on-Disk-Reliability
I see you've really thought this one through... A warehouse full of servers that need regular maintenance filled with liquid nitrogen is sure to lower costs.
"I always assume Psychology students are hiding in the bushes"
The studies were not long enough to constitute a very in-depth analysis. It would have to be a multi-month, or up to a year to analyze all the effects of raising temperatures.
For example, little was considered with:
1) Mechanical Part wear (increased fan wear, component wear due to heat)
2) Employee discomfort (80 degree server room?)
3) Part failure*
*If existing cooling solutions had issues, it would be a shorter time between the issue and additional problems since you have cut your window by ~15 degrees.
It's all fun and games till someone divides by 0. Then it's hilarious.
It is true that if you are producing X BTUs of heat inside the room, then to maintain temperature, you have to pump that much heat out. However, the efficiency of this heat transfer depends on the temperature difference between the inside and the outside. To the extent you want to force air (or any other heat transfer medium) that is already colder than outside to dump energy into air (or other medium) that is warmer, that will cost you energy.
Also, too cold, and you will invite condensation. In your hypothetical scenario, you'd need to run some pretty powerful air conditioning to prevent condensation from forming everywhere.
For starters, people sweat and computers do not. So, airflow helps cool people by increasing evaporation, in addition to direct thermal transfer. Even when you think you aren't sweating, your skin is still moist and evaporative cooling still works.
Unless someone invents a CPU swamp cooler, that's just not happening on a computer. You do need airflow to keep the hot air from remaining close to the hot component (this can be convection or forced), but you don't get that extra... let's call it "wind chill" effect that humans feel.
And yet the temperature here measured in F gets negative every winter. And where I previously lived it got above 100F every summer (and it also does where I am now, but only a day or three each year).
But in both those places a temperature of 0C was the freezing point of water, and 100C the boiling point. Yes that 100C one isn't so useful in terms of daily temperature, the 0C is though since whether water will freeze or not is the main transition point in daily temperature.
I'm less concerned with the fine-tuning of the environment for servers than I am with getting the basics right. How many bad server room implementations have you seen?
I'm sitting in one. We used to have a half-dozen built-for-the-purpose Liebert units scattered around the periphery of the room. The space was properly designed and the hardware maintained whatever temp and humidity we chose to set. They were expensive to run and maintain but they did their job and did it right.
About seven years ago, the bean-counting powers-that-be pronounced them "too expensive" and had them ripped out. The replacement central system pumps cold air under the raised floor from one central point. Theoretically, it could work. In practice, it was too humid in here the first day.
And the first week, month, and year. We complained. We did simple things to demonstate to upper management and building management that it was too humid in here, things like storing a box of envelopes in the middle of the room for a week and showing management that they had sealed themselves due to excessive humidity.
We were, in every case, rebuffed.
A few weeks ago, a contractor working on phone lines under the floor complained about the mold. *HE* got listened to. Preliminary studies show both penicillin (relatively harmless) and black (not so harmless) mold in high concentrations. Lift a floor tile near the air input and there's a nice thick coat of fluffy, fuzzy mold on everything. There's mold behind the sheetrock that sometimes bleeds through when the walls sweat. They brought in dehumidifiers that are pulling more than 30 gallons of water out of the air every day. The incoming air, depending on who's doing the measuring, is at 75% to 90% humidity. According to the first independent tester who came in, "Essentially, it's raining" under our floor at the intake.
And the areas where condensation is *supposed* to happen and drain away? Those areas are bone dry.
IOW, our whole system was designed and installed without our input and over our objections by idiots who had no idea what they were doing.
So, my fellow server room denizens, please keep this in mind - When people (especially management types) show up with studies that support the view that the way the environment is controlled in your server room can be altered to save money, be afraid. Be very afraid. It doesn't matter how good the basic research is or how artfully it could be employed to save money without causing problems, by the time the PHBs get ahold of it, it'll be perverted into an excuse to totally screw things up.
If there is a failure of AC ... that is, either Air Conditioning OR Alternating Current, you can see a rapid rise in temperature. With all the systems powered off, the latent heat inside the equipment, which is much higher than the room temperature, emerges and raises the room temperature rapidly. And if the equipment is still powered (via UPS when the power fails), the rise is much faster.
In a large data center I once worked at, with 8 mainframes and 1800 servers, power to the entire building failed after several ups and downs in the first minute. The power company was able to tell us within 20 minutes that it looked like a "several hours" outage. We didn't have the UPS capacity for that long, so we started a massive shutdown. Fortunately it was all automated and the last servers finished their current jobs and powered off in another 20 minutes. In that 40 minutes, the server room, normally kept around 17C, was up to a whopping 33C. And even with everything powered off, it peaked at 38C after another 20 minutes. If it weren't so dark in there I think some people would have been starting a sauna.
We had about 40 hard drive failures and 12 power supply failures coming back up that evening. And one of the mainframes had some issues.
now we need to go OSS in diesel cars
Condensation happens on surfaces colder than surrounding air. If you have computers which are warmer than your cooling air, it would not be a problem.
Extreme Programming - Redundant Array of Inexpensive Developers
You mean like Crays used to have ?
The problems with water are numerous: leaks, evaporation, rust/corrosion, dead/weak pumps, fungus/algae, even just the weight of all that water can cause big problems and complicate room layouts.
Air is easy. A fan is a simple device: it either spins, or it doesn't. A compressor is also rather simple. Having fewer failure modes in a system makes it easier to monitor and maintain.
You also can't just "dispose of the hot water". It's not like you can leave the cold faucet open, and piss the hot water out as waste. Water cooling systems are closed loops. You cool your own water via radiators, which themselves are either passively or actively cooled with fans and peltiers. You could recirculate the hot water through the building and recycle the heat, but for most datacenters you'd still have a huge thermal surplus that needs to be dissipated. Heat doesn't just vanish because you have water, it only allows you to move it faster.
-Billco, Fnarg.com
I think we're spending way too much time trying to 'cool' things that do not, in fact, need to be cooler than outside. Nowhere on earth is so hot that servers won't run, unless you've built a server room over an active volcano or something.
All we actually need to do is remove the heat from the servers to the air, and then keep swapping the air with the outside.
Which happens automatically if you let heat out the top and air in the bottom. Even if you have to condition the incoming air to remove moisture, that's cheaper than actually 'cooling' AC. So the second part, replacing the room air, is easy.
As for the first, I've always wondered why they don't use chimney-like devices to generate wind naturally and send it though server racks, instead of fans. I think all the heat in a server room could actually, on exit, suck incoming air in fast enough to cool computers if it actually hit the right places on the way in.
Heck, this would apply anyway. Instead of having AC vent into server rooms, why not have AC vent into server racks? Hook up the damn AC to the fan vent on each server, blow cold air straight in. The room itself could end not cold at all.
If corporations are people, aren't stockholders guilty of slavery?
Nowhere on earth is so hot that servers won't run, unless you've built a server room over an active volcano or something.
Given a sufficiently powerful fan, then yes.
All we actually need to do is remove the heat from the servers to the air, and then keep swapping the air with the outside.
Which becomes more difficult the higher the ambient air temperature becomes. Heat transfer is proportional to heat delta, so the closer the air temperature is to the heat sink temperature, the more air you need to blow to remove the same amount of heat. Eventually, the amount of electricity you are spending blowing air over the heat sinks is greater than the savings of using less AC.
This was half the point of the article -- you can save a lot of money by raising server room temperatures, but eventually (at a temperature well below outdoor ambient around here) you actually start to lose money due to all the extra fan activity.
Which happens automatically if you let heat out the top and air in the bottom.
Yes but much too slowly to be of use. Convection is also proportional to temperature difference. By the time your server room temperature is enough higher than outside temperature to create significant airflow, your servers are toast.
As for the first, I've always wondered why they don't use chimney-like devices to generate wind naturally and send it though server racks, instead of fans.
Go ahead and try it. A lot of cases already have ducting that funnels air directly from outside the case to the CPU. A few more pieces of cardboard, a hole and chimney in the top of your case, and you should be ready to remove the fan and see what convection can do for you. Sneak preview: unless you've specifically picked components that can run off passive cooling, you'll be in the market for a new one. Especially if you live in a hot place and turn off your AC for this experiment.
While its conceivable to have an effective server room based entirely off of low-power chips that require no active cooling, space is still a major concern in the server room. The desire for greater compute density is directly fighting against using a large number of low-power chips spread out. Thus performance/watt becomes a major metric for the server room, because they want the most performance for a fixed amount of space and thus cooling.
why not have AC vent into server racks?
That's actually a good idea, and a lot of places do it.
The enemies of Democracy are