Emergency Cooling with Limited Power?

Nos. asks: "I work for a small webhosting company (I'm on leave from my Gov't job) and we've started having some power problems. Actually someone managed to blow out the substation powering the area so we're piggy backing off another one, and they're slowly powering more and more things off. Elevators, lights, etc. are gone. Since the building we work in has a few IT company's working it in, they're trying as hard as they can to keep the A/C running, but its not looking good. As such, the possibility exists that our server room could get very warm, very quickly. Since we've already powered off everything that's not essential, we're starting to look at ways to keep the room cool without using a lot of power. Generators an small A/C units are a last resort as it would mean holes in the walls. The only thing we've been able to come up with is dry ice and some small fans to circulate the air. Of course this is happening as we're heading in to a week of over 30C days. Does Slashdot have any ideas?"

Comments

Keep ordering icebergs off Ebay :-)

[Gruturo]

No, seriously, I don't have any exceptional cooling method to suggest, so I'd focus on reducing heat production instead of dissipating.

1) Power off every non-essential item (You say you've already done it, but have a second look at what's REALLY essential. Got 2 firewalls in cluster configuration? Keep only one! Pull out that hot-swappable hard drive from your raid-1 array! - Warning: will have a long-term impact to your uptime)
2) Ventilation. As long as you're not in Saudi Arabia, air from outside is cooler than what the server room would be without air conditioning.
3) People! Humans give off a lot of bodily heat (Matrix jokes apart). Keep people off the server room unless it's really necessary
4) Lighting - Use compact fluorescent instead of incandescent (they run much cooler, too) and turn them off when it's not needed
5) Shadow - An incredibly effective way of bringing down room temperature by as much as 10 degrees. Might not apply to you, but if you are in a very exposed side of the building, or under the roof, you might benefit greatly from it.
6) (Illegal in many countries) Cooling with running water. Extremely effective, but a huge waste of water
7) (a bit extreme) Replace the less loaded and less critical servers with a couple laptops you might have lying around. I'm writing from a 1.6Ghz Centrino laptop with 512MB DDR - it's a lot more powerful than some of the servers I have at work. (and laptops tend to be terribly stable).
Its power supply is rated 65W!
8) - If all else fails, decentralization. Put the remaining servers farther apart (the heat in a single 42U rack filled with equipment is tremendous, while if you spread the content all over the room it will be more bearable for the hardware). Get a few very long network cables and take something out in other rooms, also (even if only the server room is ups-protected, it won't make a big difference when power goes down for a day).

btw fp :-)

Re:Keep ordering icebergs off Ebay :-)

[(H)elix1]

Dang, fp and spot on... checking to see if my cube froze over...

About 10 years back we had a similar problem. The main AC unit failed and we were watching the mercury rise after we did most of the above. We were able to bring portable air conditioners which pushed the hot air though 1' plastic tubing to the cool outside air. It worked, no holes in the wall. The generator adds a little difficulty, but nothing long extension cords wont fix.

Don't forget to pull the ceiling panels if you can - assuming the air is cooler. Was for me.

As for dry ice - don't forget that sublimes to CO2. I suspect every door/window is open right now, so less of a ventilation issue. Blowing a fan over what ever you use will help.

Re:Keep ordering icebergs off Ebay :-)

[drlock]

2) Ventilation. As long as you're not in Saudi Arabia, air from outside is cooler than what the server room would be without air conditioning.

I grew up in the tropics, most of my life without A/C. You'd be surprised what a box fan (that is what we called them anyway, 2 ft square fans that sit on the floor. They don't use a lot of power.) in the doorway can do. If you want to get really fancy, try and force air circulation: Find two openings to the outside world (ie, window and door or 2 doors). Open one and put the box fan in the other. If you put the box fan in a door put cardboard above the fan (ie seal the rest of the door), so the air doesn't simple circle around the fan.

6) (Illegal in many countries) Cooling with running water. Extremely effective, but a huge waste of water

Or if you don't want to run water, put a large pool of water (think kiddy play pool) in front of the fan. Water evaporating of the surface will act as coolant (same idea as running water, but requires more surface area and doesn't give a constant supply of cold water.)

Re:Keep ordering icebergs off Ebay :-)

[jandrese]

Er, most computer rooms don't have direct access to the outside (no windows, no outside doors). Usually they're buried in the middle of the building past a couple layers of security.

Re:Keep ordering icebergs off Ebay :-)

[spencerogden]

Right, because water vapor is going to condense on hot computer components...

Re:Keep ordering icebergs off Ebay :-)

[GigsVT]

Last I checked, cold things condense water out of air, not hot things.

The only danger is that once they get power back, when the A/C kicks in cooling off that air that is near 100% humidity, it's likely to get kinda foggy in the server room, and water condensing out of the air could settle on things.

The solution would be to make sure the humid air was circulated out before turning the A/C back on, and bringing the temperature down slowly to prevent dew.

Re:Keep ordering icebergs off Ebay :-)

[Alan Shutko]

The only danger is that once they get power back, when the A/C kicks in cooling off that air that is near 100% humidity, it's likely to get kinda foggy in the server room, and water condensing out of the air could settle on things.

That's not too likely. A/C units dehumidify very well. And when they blow cold air into the room, that itself will circulate out hot, humid air. And since A/C units don't cool a room instantly, the temp should come down slowly enough.

However, if for some reason this would be a problem in a specific place, the answer is to give the water to condense first and toss a dehumidifier in there. Once the power comes on, turn on the dehumidifier, and any water will condense first on its refridgerated coils than on anything else.

Re:Keep ordering icebergs off Ebay :-)

[Anonymous Coward]

Water and computers do NOT mix. A humid room WILL shorten the life of electronic components, and lets face it an open container of water in a server room (its not hard to have a half million worth of equipment in there, I do and I have almost no budget to speak of) is beyond stupid and inviting disaster.

Seriously this guy needs to find a new job. I work for one of the cheapest companies around (we're owned by an American holding company after all) but all the BASICs are covered. I might not have room to even walk in my computer room, but at least we have plenty of power, some decent UPSes, a proper air conditioning system, and environmental monitoring. This kind of shit costs fuck-all to a business.

OTOH we are a manufacturing company so we use fuckloads of power, so we have a lot of it running into our building.

Re:Keep ordering icebergs off Ebay :-)

[Anonymous Coward]

You'd be surprised what a box fan (that is what we called them anyway, 2 ft square fans that sit on the floor. They don't use a lot of power.) in the doorway can do. If you want to get really fancy, try and force air circulation: Find two openings to the outside world (ie, window and door or 2 doors). Open one and put the box fan in the other. If you put the box fan in a door put cardboard above the fan (ie seal the rest of the door), so the air doesn't simple circle around the fan.

Absolutely. For best results, get two (they only cost $15 each) and put one blowing air in from one direction and the other blowing air out in the other. If you only have one fan, use it to blow the hot air out - that will draw new air in, whereas trying to blow air in without the fan to blow air out is much less successful. And this way you also don't have to seal up the doors with cardboard ;-)

Using this method (with an extra fan to turn a corner), we managed to keep a tiny basement office with a generator, 6 computers, no A/C, and cracks for windows comfortable during a 95F+, humid summer by blowing cold air in from an air conditioned room waaaaay down the hallway.

Re:Keep ordering icebergs off Ebay :-)

[Anonymous Coward]

If it shares airspace with the people-rooms, flooding the floor with CO2 is probably unhealthy and illegal.

Re:Keep ordering icebergs off Ebay :-)

[GigsVT]

That's not too likely. A/C units dehumidify very well.

Usually at least. Inane anecdotes follow. :)

My high school computer labs had a really screwed up HVAC system, they had to run standalone dehumidifiers all the time, and empty them every 2 hours or so. There were several times that there was actually a pretty thick hazy fog in the building.

Maybe I've just had bad experiences with HVAC. Where I work now, even the new upgraded HVAC system put in the server room has failed at least twice a year since we got it. The old HVAC that covers the rest of the floor has shorted out and taken most of the building with it a couple times too (a bad breaker, and bad full-short fuse/breaker timing caused the upstream 1500amp leg fuse to blow rather than the two downstream ones)

Also, a couple times, my office windows were literally dripping with water in the winter, there was some steam leak in the heating system which was pushing the humidity up.

Anyway, maybe it's just the HVAC contractors in my town don't know what they are doing.

Re:Keep ordering icebergs off Ebay :-)

[GigsVT]

I agree with you, it is kinda a silly idea when you can rent those washing machine sized A/C units on wheels that other people have mentioned, for not very much money. We've used them before when our server room A/C has failed.

I'm a tech worker for a manufacturing company too, I've always thought we have a pretty unique experience compared to people working for tech companies, it would be cool to make some kind of chat room for people like us.

Bigger Generators

[Vokbain]

Is buying larger generators and hooking them up directly to the building's power grid an option?

Re:Bigger Generators

[N3Bruce]

About 12 years ago, I worked for a company that could provide trailer mounted diesel generators in sizes from 7.5 KW up to over 1,000 KW (that's 1 MW folks. Along with the generators, we could supply just about anything else everything else you would need to do the job, such as cabling, distribution panels, and so on. Many of these rentals were for construction sites and special events, but I remember one time we set up a huge 800 KW set for the switching station at a major long distance company, which had lost one of their 3 backup generators. The generators were typically tied in to the main panel, or whatever circuits needed to be tied into, and could be wired for just about any combination of 208/240/277/440 in either single phase or three phase current you could need.

Swamp coolers and portable air conditioning units could also be rented from us. It ain't cheap, rental of a large (300KW) generator, plus the chillers and related cabling could run up to several thousand dollars per month, plus the cost of fuel (figure roughly 1 gallon/hr per 10KW), but companies that work in this business can quickly and effectively solve your power problems until things get better again.

Once the immediate crisis has passed, you might want to at least look into a permanent emergency generator large enough to keep your servers powered and cooled. I currently work in a payment processing center for a large financial institution, and it has a fairly large (200-300 KW) emergency generator, along with a UPS which is mainly intended to hold the power until the generator kicks in, but I personally saw it run the building which is full of power hungry servers and mail sorting and extraction equipment for nearly 2 hours when the generator overheated and shut down.

If you go the generator route, either contract out or have someone on staff which will periodically do maintenance and exercise the generator, or else it is liable not to work when you need it the most. A generator can sometimes also help pay for itself by "peak shaving". Large users of power are often billed by their peak use, or otherwise based on how much power they use during times of peak demand. Electricity from a diesel generator will normally be more expensive per KWH than commercial power, but during times of peak demand, it can be cheaper to run the generator than pay extortionate rates for power on a hot summer afternoon.

Rent big truck generator & run whole building

[StormForge]

It's pretty affordable to rent a huge generator system mounted on a tractor-trailer. Probably have plenty of power to keep everything running. Maybe make the power company reimburse you even?

AC onna truck

[PapaZit]

First of all, you should have thought about this BEFORE now. If you're a decent webhosting company, disaster planning and recovery is as essential to your business as spare hard drives.

That being said, there are often companies who can provide air conditioning and/or generators on a truck. They'll block off a doorway or the loading dock and pump the air in through there. If you have a little more time (and appropriate permits, etc.), they're often willing to run temporary connections into your forced air system.

Whenever they do HVAC work on our building, they have the trucks set up and waiting. We have a few too many computers to even survive with "just the essentials" if the AC goes out.

Re:AC onna truck

[Nos.]

Agreed, and though I checked on most things, backup A/C was just something I did not consider yet. The problem is that while the building has backup generators running on natural gas, the A/C is not in this building, there is a campus wide HVAC system (we're located on the U of Regina campus). The power company is currently cavasing North America looking for transformers they can bring in to substitue in for the substation.

Just found out as well, the power company can only guarantee the campus 3.5 megawatts, we're currently drawing over 7.

Re:AC onna truck

[R2.0]

"the A/C is not in this building, there is a campus wide HVAC system (we're located on the U of Regina campus)."

Can you be more specific? You need to know exactly WHAT your central plant is supplying your building, because it's not cold air.

If they are supplying condenser water (between 60F and about 110F), then your building has local units cooling your server room - probably Lieberts (trade name). If so, you need to
1. power the leiberts from some temporary power source (your own generators with some temporary cables?) and
2. (if they have actually stopped supplying condenser water) cross connect domestic water to the condenser water supply (use proper backflow preventors) The domestic water part will be Mucho $$$. Or renting a portable cooling tower for your own condenser water - don't forget power for this too.

If your central plant supplies chilled water (about 25F-40F, depending on the chemical mix) you would need to rent a portable chiller plant and power for that, along with whatever power your building equipment needs.

Best option is probably to rent a truck mounten air conditioner and generators and run temporary ducting to blast cold air into your space.

Forget the swamp cooler ideas - you will have humidity problems, and it won't provide the amount of cooling you need.

Re:AC onna truck

[Nos.]

Not really, with the bit of talking I've done with the maintenance guys, I was under the assumption it was forced air they supplied (be cooling or heating), but I couldn't say for sure.

Unfortunately, we can't really power anything from our own generators. They are at capacity already, though I believe they are preparing to upgrade them.

If they're that critical.....

[Creepy Crawler]

Try to seal the room as much as you can (plastic sheets on all but intake) and put canisters of "Liquid Air" inside and turn the valves up to medium output.

If you use standard "Liquid Air", hopefully it's the 75% N, 20% O, 5% others so it would be breathable if you need to enter. You _could_ go with N2 canisters, but that'd be dangerous as it'd displace O2.

It would cost a lot for these cansiters, and you'd have to refill them every day. But that's why it's an emergency.

The only thing I can think of is pay "Energy Truks" : the ones that provide expensive portable energy to power your grid. Still, very expensive.

The best is having an off-site data center that can replicate all your data. That would immue to a shutdown of your plant there. That's not a solution now, but might be when you're back and running.

All the best.

dryice

[JDizzy]

If you do decide to go with dry ice, make suree to have a good supply of fresh air to wash over the ice instead of letting the dryice evaporate in the data-center. What happens is dryice melts and turns into carbon gas, which makes humans passout from a lack of oxygen. It is even possible to die as this would be comparable to running the exhaust of the car with the garage door down. It would fill the room full of noxious gas. So the trick is to use the dryice in conjunction with a heat/cold exchanger to cool the air in the room. Then again, punching holes for the generators doesn't sound like such a bad idea any more huh?

Re:dryice

[Tom7]

The noxious gas in the exhaust of a car is carbon monoxide. Dry ice is solid carbon dioxide. But indeed it can cause asphixiation.

Re:dryice

[gazbo]

...as this would be comparable to running the exhaust of the car with the garage door down.

Not quite. Yes, if you displace enough Oxygen with CO2 then you can asphyxiate. However, what makes car exhausts so lethal is the carbon monoxide. CO actually has a much stronger affinity to haemoglobin than CO2, so rather than dying because there is no Oxygen, CO will actively displace Oxygen from your bloodstream, even if there is plenty present in the atmosphere.

Of course, this is not such a problem in modern cars with catalytic converters, but still.

Re:dryice

[JDizzy]

indeed your correct, and I'm wrong... well sorta. I base my somewhat uninformed comment on the fact that there is lore about people who put dry-ice in a standard cooler to keep icecream from melting on a cross-country drive. As the lore goes: the couple with the cooler in the back seat suffer form a plauge of problems the entire trip. The man has a constant headache, and the woman is constantly sleepy. It turns out the dryice was the issue in the confined space. The dryice evaporates, and the humans breath it in, and eventually they got into an auto accident. So I fear that using dryice in a confined space like a data center will indeed cool it down, there needs to be a way to oxygenate the same air too, possibly scrub the CO out or the humans will suffer.

Re:dryice

[Glonoinha]

-indeed your correct, and I'm wrong...

It takes a big man to admit publicly that he is wrong.
It takes an even bigger man to laugh at that man. Hehe.

Just kidding. I took issue immediately with the dry ice thing ... jumped in here to say pretty much the same thing you said (so I won't bother repeating it).

We are talking Canada for crime's sake - if 30 degrees C is the peak you are worrying about ... that's about 86 degrees F. Hell I live in Austin, TX and during our 105+ degree hot spell (6 months of summer) (41 degrees C, for the record) we are doing good to keep my office at 80 degrees F ambient during the entire work day.

Your computers will work just fine at 30 degrees C, if you can simply circulate the air in the dungeon (server room) with fresh air from outside. See a window, open it and put in a big fan. No Windows? See a penguin / oh wait, wrong thread. No windows, run a flexible duct, a big one like kids use to make forts out of (use big cardboard boxes or get creative, sheets and hula hoops if you have to) and blow air in using big air moving fans.

You don't need a massive drop in temperature to be back to normal operations, you just need to have your server room be at the same temperature at it is outside. Sure 30 C feels warm to you, but in my world it is just a little higher than work office occasionally runs.

Re:dryice

[JDizzy]

I wasn't that wrong, actually i wasn't at all, except forthe aspect of carbom-dioxide compared to carbon-monoxide and the different ways they kill humans. At anyrate, at my former job I ran some SGI Onyx machines (the size of refigerators) which had a auto-cut-off feature to self-die if the temperature rised above 112F, which was easy for those machines unless you had a swam cooler pointed directly at it. The same was true of the big honking HP t-600. It seems that only i396 machines have the abillity to over-heat accidentally and unknowingly yet keep processing.

Allas, I have thought of a solution tothe issue. Simply watter cool all computers! Either submerse the mainboards in mineral oil (non-electrolyte/non-conductive) which pool-pumps to circulate the fluid. Or we can have a closed water cooling system which requires pumps, even a centrally located pump if the hardware vendors came to standard on the pump interfaces. In the later system we could simply move in treadmills durring a power outage, and have the IT system admin excerrsize on the treadmills wich would be attached to the standardized central pump system to circulate the water. Human rodents in a spin-wheel! har har

Re:dryice

[andcal]

A room filled with melted dry ice (carbon dioxide) can smother a person inside the room. Yes, this could be compared with someone running a car with the garage door down, but it is not the same, because people who die from running the car with the garage door down are killed by carbon monoxide poisoning, where the CO molecules actually grab ahold of their red blood cells with a tighter grip than oxygen does. CO2 doesn't poison, it just displaces the O2 in the air, smothering the person like a blanket.

Having said that, dead is dead. But someone who makes it out of a room of CO will be OK more often than a person making alive out of a room full of CO, taking molecules of CO with them, tightly bonded to their hemaglobin. This second person will need medical treatment ASAP.

Re:dryice

[Anonymous Coward]

Humans exhale when the CO2 concentration in their lungs gets high enough. You should run into trouble long before you smother.

Turn off the computers.

[Picass0]

Sue the power station for lost revenue/downtime.

If you're looking at dry ice and knocking holes in the walls, you're almost out of options. Save the hardware.

Nix the Dry ice

[jandrese]

I'd nix that Dry ice idea. Most server rooms don't have paricularly good ventilation despite the large amounts of A/C in use (it's mostly recirculated air). Releasing large amounts of CO2 into the room might just turn your server room into a silent deathtrap.

Re:Nix the Dry ice

[zulux]

Releasing large amounts of CO2 into the room might just turn your server room into a silent deathtrap.

But if you'r....say 6 foot three inches and your PHB is say... a dwarf.... well......

Invite him in, and when he falls over with his eyes bludging out.... run!

Dry Ice?

[GigsVT]

How much dry ice are you talking about? I'm thinking that could get dangerous pretty quickly, with massive amounts of dry ice in a windowless room.

lots of air flow

[TheSHAD0W]

It may be pretty warm outside, but the machines will take it so long as you can dissipate localized heat build-up. Get a few circulating fnns and set 'em up around the room, with one blowing air directly in through a window. (You DO have a window, I hope.)

Re:lots of air flow

[Zathrus]

I hope he doesn't... a window in a server room is highly insecure, and it also leads to fluctuating temperatures in the area of the window.

Of course, with a fan and some cheap ducting you can have a similar effect. You'd need a much more powerful fan to do it though.

A serious suggestion? Generators and portable AC units. I've seen them used by a former company when the AC was inadequate in the server room. They were about 1.3m tall and had large white hoses coming out of the top to make them about 2m tall overall. You had to feed them water on a pretty regular basis since they were not closed loop AC units, and you'd also need generators to give them power. It worked in a pinch though. No need to cut holes in the ceiling either -- they just vented into the room (suck air in from the bottom, output at the top).

Re:lots of air flow

[Loosewire]

I think your supposed to put the hose outside or all the Unit does ultimatly is heat the room up.

Re:lots of air flow

[Zathrus]

Er... right you are. That or you blatantly violate the 2nd law of thermodynamics (insert ob Simpsons quote). Maybe it was hooked into the HVAC system then... there certainly weren't any hoses running outside of the computer room.

Re:lots of air flow

[Loosewire]

ah hang on you said feed them water, then it was a swamp cooler not A/C (ie a humidifyer basically)

Re:lots of air flow

[DavidTC]

Is humidity a real good idea in a computer room?

And won't that just end up with all the heat in the air anyway? I mean, you can't make heat magically disappear, so it seems like all that would do is put a thin layer of water on everything, which would then evaporate, which is great for cooling people down, and might even work on the equipment if you don't short it out, but it's not going to actually cool the room down, so you're just delaying the inevitable.

Re:lots of air flow

[Loosewire]

well exactly (i didnt reccomend a humidifyer i was origionally commenting on the fact a real A/C needs external venting)

Re:lots of air flow

[Chmarr]

And won't that just end up with all the heat in the air anyway?

Actually no. Turning water liquid into water gas is an endothermic reaction... it actually /absorbs/ heat to do so. This is why swamp coolers, and in fact your body's sweat system, actually work.

Re:Rent big truck generator & run whole buildi

[bofkentucky]

CAT Entertainment services can have a truck anywhere in the US in 24-26 hours with a complete setup. GE Energy Services also rents truck generators 6KW-22MW and portable AC and chiller systems. Looking at their locations, they should be able to reach anywhere in the US in 8-12 hours.

most important of all --

[fraxas]

Ask Slashdot.

But seriously -- be sure to consider the relative risks of keeping your server room up versus taking it down and waiting. If you take it down now, you're guaranteeing yourself downtime, but you can come back online as soon as the substation comes back.

If you try to stay running, you're not guaranteed to have downtime, but if you do it'll be intense, because you'll have damaged hardware to deal with.

rent a generator.. that's the usual way at farms.

[gl4ss]

rent a small generator that runs on gasoline(aggregator? i don't know about the english term for such), run the ac from that if you have to, that's what they do at farms when they storms cut off the power and they absolutely have to keep those strawberries in cold.

and then theres the wet towels & etc..

besides... 30c isn't that much if you can have massive ventilation(keep it that 30c), it's 33c in here my flat now and the couple of computers that are here run fine..

Emergency where I work

[haplo21112]

We have these things called "movin' cools" basicly A/C on wheels...it does need to be plugged in, but you could run it off a generator and a long cord....

They have a Dryer hose like outflow pipe for the hot air which we can connect at various points to the sent system...you could probably micky mouse it to your vents with some cardboard and Duct tape....

Dude.

[Naikrovek]

You should have thought of this LONG before now.

Why do you host at a place without dual power companies providing power?

Why do you host at a place without dual redundant A/C on EACH power provider?

Why do you host at a place without dual redundant power generators?

Just how "essential" is the stuff you're hosting?

How is it important that you're "on leave from [your] Gov't job"?? That's no excuse (if it was meant to be) for not jumping on top of that HUGE MASSIVE INSANELY rediculous situation your servers are in right now, and taking the steps to fix it the very first day you started at that job.

in fairness maybe you started today, but i think somehow that you've had time to fix this before now.

Re:Dude.

[Anonymous Coward]

... Dude. It's a small webhosting firm. It is utterly unreasonable for clients to expect a small webhosting firm to have the capital or clout to seamlessly handle a disaster, and it is even more unreasonable for someone who seems experienced in this stuff to expect them to.

Why do you host at a place without dual power companies providing power?

Why do you host at a place without dual redundant A/C on EACH power provider?

Why do you host at a place without dual redundant power generators?

Because some of us live in the really-real-world where you spend real-time and pay real-money for real-services. We're not talking about a $50 million Exodus datacenter here. We're talking about a small webhosting company. It's possible that he's running less than 100 websites total. Put things in perspective. With that small of a company, it's hard to afford to have someone on duty 24 hours to take care of the occasional problems, and most of the time you're at the mercy of your upstream provider anyway so there's not much that can be done. And you want dual redundant everything? He should probably be multihomed on OC-3s too?

I run a small webhosting company too. I do have a disaster plan, but it's a real world disaster plan, and quite frankly if the disaster is serious enough, one of my options is to just close the doors and cut my losses. If my city gets obliterated by a nuclear weapon, then no thanks, I have better things to be doing than keeping my servers running.

There are way too many obsessive sysadmins out there who are appalled at anyone who doesn't have 100% uptime. Get a life.

Just how "essential" is the stuff you're hosting?

Not essential enough to go through nor afford all the stuff that you're implying he's an idiot for not doing.

Re:Dude.

[Nos.]

You should have thought of this LONG before now.
There's a lot of things that were out of whack when I started here about 6 months ago, I'm working on it. Redundant air was just not something I considered

Why do you host at a place without dual power companies providing power?
Because we don't have two power companies here. I live in Saskatchewan (Canada) and we only have one power company

Why do you host at a place without dual redundant A/C on EACH power provider?
The office is located on the University or Regina campus. The campus provides heating and cooling to every building here. Redundant A/C is out of the question on that kind of scale for this small a community (~225,000)

Why do you host at a place without dual redundant power generators?
We have natural gas generators, however, they only power certain things within this building, not the A/C for the campus.

Just how "essential" is the stuff you're hosting?
Depends who you talk to. Our clients think they're sites are critical to their business. If we can't maintain uptime, we lose clients. If we lose enough clients we go out of business, and I lose this job, and get stuck going back to the Government, something I DON'T want to do

The Gov't Job was for anyone reading my profile and wondering why I say I work for the Gov't and now say I work for a web hosting company. Its not an excuse. Secondly, I have been working on improving things here, but I was brought in for a development project which recently finished. Only lately have I started to work on the network, servers, facilities, etc.

Re:Dude.

[Anonymous Coward]

Why not colocate somewhere that does meet the requirements of your buisness? Or, you could at very least have servers set up at a colocation facility for emergencies like this.

Re:Dude.

[realdpk]

"Dude. It's a small webhosting firm. It is utterly unreasonable for clients to expect a small webhosting firm to have the capital or clout to seamlessly handle a disaster, and it is even more unreasonable for someone who seems experienced in this stuff to expect them to."

Bullshit.

How did they get to the point that this became a problem? I mean, what tipped them over the edge? I seriously doubt this happened instantly overnight - it's not like some hosting customer comes to a small webhosting firm and says "I have 3000 servers, where can I plug them in?"

Re:Dude.

[Glonoinha]

If the wall of the server room is next to outside, put in a window. Get a contractor in there today, cost you less than $1,000 Canadian and you can hand tailor it to your needs. Make it big enough to put in two box fans, one pointing in and one pointing out ... or put in two smaller windows.

Your systems will be just fine if you can keep them at 30 degrees C, and you can secure (brick up) the windows later. Hell, just remove (neatly) a few bricks for the duration and put a fan or fan w ductwork in there, brick it back up when this blows over.

I bet the electricity company can even recommend / supply you a good list of brick workers / contractors. Servers are expensive, clients are expensive, brick work is cheap.

Re:Dude.

[joshuac]

---snip
Why do you host at a place without dual redundant power generators?
We have natural gas generators, however, they only power certain things within this building, not the A/C for the campus.
---snip

Get the campus electrician to move a circuit from one of the generator backed circuits (say, heating, which is not terribly critical to you during the summer, even in Canada) to a circuit where it could do some good to alleviating this little crisis you have (your cooling system).

Depends, of course, on the size of your cooling system. If it is a very small server room we are talking about, then your AC unit may only need a 15a circuit, an extension cord from a powered outlet would work; if it is something bigger (which is probably is), your electrician will have to do a little (only a little) creative thinking.

This also depends on how bureaucratic your university is / how competent your electrician actually is.

Re:Dude.

[sjames]

How did they get to the point that this became a problem?

The unexpected total failure of the substation suplying their power grid probably had something to do with it.

Let's face it, there exists multiple levels of service in this world, and there is always a risk analysis to be done. Sure, all colo customers want a zillion nines uptime, but most won't actually pay for it.

A great many colo spaces would be ill prepared to deal with a protracted failure of external power, and would be scrambling for an answer (desperatly trying to get diesel trucked in before the tanks go dry for example).

The real question here will be can they keep things going without having to invest in triply redundant nuclear plants (and passing the expense on to their very price sensitive cuctomers). The problem is made worse by the many companies that sunk millions into huge colo centers during the dotcom boom who are now practically giving away colo space at a loss just to slow the bleeding down while they figure out how to stay afloat (fact is, many have closed down, many more will).

In the mean while, airflow is what will count. To some extent, it is possible to compensate for a warm room with a LOT of moving air.

Depending on the hardware, temporary underclocking might be a good idea. Spinning down hard drives could also help. If there is scheduled maintenance downtime coming up, consider doing the next few months worth NOW during the afternoon when the temp is highest and power demand is at it's peak.

Thermal mass might also be useful depending on how cool things get at night. It would at least soften the peak temperature during the afternoon.

Re:Dude.

[Anonymous Coward]

Don't take it so hard. Yeah, you don't have a GREAT hosting company but it sounds like you are trying to make do with what you have.

My suggestion:
Determine who your most important customers are, and get their shit backed up offsite... and on a webserver offsite.

Get some server space in your competitors if you have to. If your customers won't tolerate downtime then you are MORE likely to lose them over REAL downtime, than if you have to switch over to the competition.

You're buying bandwidth from SOMEONE anyways... try upstream first if you're that worried about your image.

If your site goes down, you can point DNS at the failover location (hopefully they DID give you a laptop :-)

Construction-site quality fans push LOTS of air (and use some power... less than AC tho). You're in Canada just open the window.

I'm not sure how well it would work, but ....

[jefeweiss]

Buy two big high output fans, the biggest and most powerful that will fit in the door to the server room. Stack them one on top of the other in the door. Face the one on the bottom in to suck relatively cooler air from the floor into the server room, and the one on the top out of the room, to suck the relatively hotter air at the top of the room out. This will work better if you can get the one on top all the way to the top, and seal the rest of the space in between. Also, this isn't going to be as effective as the server room opens onto a long hall, as if it opens onto a big room.

If the server room is on the bottom floor of your building you could also prop open the fire doors on the stairs so all the cool air in the building flows downhill to the floor your computers are on. This is going to be a fire code violation if your building is tall enough.

Low tech, often overlooked

[Unknown Poltroon]

its called "Open the window"

Seriously, get a few big fans, set one set in the door blowin in, one set in another door blowing out. DOnt have two doors? Make a hole. Drywall cuts easily, and also patches easily.

How much does water and sewage cost?

[stienman]

As water usually comes to the building through undreground pipes, it attains about 50-60 degrees farenheit.

Bring a hose into the room, and use a car radiator or cheap, large coil of copper tubing, and run the water through that and into a drain. Blow air through and you've got a fairly inexpensive way to cool.

Water is not expensive, but you could go through hundreds of gallons a day. Limit your water usage by watching the temperature of the incoming and outgoing water, and placing a valve in the outlet. If the temperature difference is great (60 in, 80 out) then let the water through a bit faster.

You could even set up several of these in series so you can cool different portions of the room. Think about how the air circulates - if you can get the air to go clockwise around the room the fans will use less energy, and the whole room should reach the same temperature.

Of course, it goes without saying that you need to be careful not only of leaks but condensation. Place buckets under the coils, and connections/transitions - make sure you have no leaks, and dump the buckets occasionally.

If the cooling isn't great enough, put a set of coils just after the inlet into a trashcan of water and dry ice.

Note that it may take several minutes of water running before you actually get the cooler water, depending on how much of the building it has to travel through to get to you. If it goes through a lot of the building, you may not have very cool water at all, as it'll attain the temperature of the building. Give it a good half hour or hour at full blast and measure it to see what is possible.

-Adam

Re:How much does water and sewage cost?

[eakerin]

Wow, that would be a funny site to see, car radiators, buckets, and hoses.... almost sounds like something I would cook up in an emergency :)

Our old server room (as well as the Phone Switch Room) were water cooled, although we had actual Cooling units.

The Server room was cooled by a large Air conditioner style unit, and the Phone room was cooled by a portable one (although this 2nd room was only about 5x5, but contained a Meridian Phone switch that took up half of it, it worked pretty well. See If you can rent a portable water-based cooling unit from someplace nearby, and hook up some hoses!

Re:How much does water and sewage cost?

[op00to]

This has been brought up before, but this is an egregious waste of resources. Why would you consider using FRESH, potable, treated water to cool your room? Many parts of the world are under water shortages, so it seems to me that you would want to conserve water and find other alternatives. Also, if you start pulling a few hundred gallons a day more than you normally do, you'll have the Water Company on you pretty quickly -- they don't like huge sinks like this. Needless to say, cooling in this manner is also illegal in most states in the US.

Re:How much does water and sewage cost?

[stienman]

"This has been brought up before"

Yes.

"but this is an egregious waste of resources"

Already noted.

"Why would you consider using FRESH, potable, treated water to cool your room?"

As I've already explained, it's cheap, available, and can be set up quickly - it meets the poster's needs.

"Many parts of the world are under water shortages, so it seems to me that you would want to conserve water and find other alternatives."

Ah. So you are suggesting that by saving water here, shortages will not occur elsewhere? I'm afraid you are generalizing a little too much. Most municipal water systems are self contained, and get water locally (wells, rivers, lakes, runoff, etc) I'd be interested in seeing research that shows how preserving my water here helps someone who lives on another continent. I live in Michigan - I've got plenty of water. Need more? I can dig a well and get more. Still need more? Truck it in from any one of dozens of companies who provide just such a service. Still more? I can buy property with a very large lake on it. It saddens me to see others suffering without water - but you can't really be suggesting that I should feel guilty because I live in an area where it is so abundant.

"Also, if you start pulling a few hundred gallons a day more than you normally do, you'll have the Water Company on you pretty quickly -- they don't like huge sinks like this."

Really? I'm sure an arrangement could be worked out - since the poster stated this was an emergency. Futhermore, the water company doesn't usually have the capability to find who is using an extra 100 gallons of water a day. They can name the trunk, perhaps, but in the end it usually means more money for them, and unless the system is severely over burdened it would be a waste of money and resources to spend time finding the abuser. I never suggested the poster should skirt rules and regulations - they should check the contract they have with the water utility.

"Needless to say, cooling in this manner is also illegal in most states in the US."

I don't find it needless at all, since I've never heard of that law/regulation from a reputable source. I'd like to see more than you assertion on this statement. I wouldn't be surprised if you were right, but it seems to be repeated many times in this story without any sources.

-Adam

move

[hswerdfe]

move to Northern Canada! :D

Re:move

[Anonymous Coward]

Depending on your definition of "North", it might not help..

I live in a Canadian city that's almost as far north as Moscow, and the temperature is supposed to go over 30C today.

Re:move

[Anonymous Coward]

Moscow ain't that chilly this time of year either.

Illegal

[Anonymous Coward]

AFAIK, this is illegal in many countries, and a huge waste of water.

Re:Illegal

[jafuser]

A closed loop with a swamp cooler outdoors would solve the water wasteage, assuming the relative humidity isn't too high.

Re:Illegal

[Anonymous Coward]

I don't consider swap coolers wasting water, since the water they use costs less than the electricity they save; however, you should be aware that swap coolers do use lots of water. Not just what they evaporate, but some has to run off or the filter mesh gets clogged with mineral scale.

In addition, I think you can get away with using a huge amount of water even illegally if it is for a few days only while an electricity outage is repaired.

Get rid of excess heat in the first place

[forged]

Well it is a bit late for that, but if you were a large datacenter hosting a lot of managed servers, wouldn't you consider low-power solutions such as....

"So, when Transmeta Corp. came along in early 2000 and announced a processor that was 85 percent to 90 percent of the mobile Pentium's performance with a fifth of the power consumption, it was a no-brainer," Hipp said.
The result was the RLX System 324, a blade configuration that packs more punch into a smaller space than any other server on the market?up to 336 blades in a single, 42-unit, industry-standard rack (..)

It goes without saying that a box that necessitates 80-90% less power than an equivallent Intel or AMD, produces less heat. (from an older but insightful eWeek article.)

Water Ice

[the eric conspiracy]

An insulated tank with water ice (don't use dry ice because of the CO2) and pumping the cold water through a couple of car radiators with fans blowing air through the radiators is probably the best you are going to do. Ice is very useful because it has the heat of fusion and a melting point of 32. The heat of fusion will keep the tank at 0 C and give you a good temperature difference between the radiator and room for a good heat flow; the heat of fusion gives you a good heat capacity. You can probably estimate how much ice you will need per day based on the capacity of your air conditioning equipment and it's duty cycle. 1 ton/day of ice is roughly equivalent to 12,000 BTU/hr.

Re:Water Ice

[MrResistor]

You might think about adding some salt to the ice, too. I've added some rock salt to the beer cooler at a party on a hot day, and despite the cooler lid being open most of the time it was still quite painful to fish for beers hours later.

The answer is in front of you Neo

[Anonymous Coward]

Your servers are surrounded by heat. Heat is energy. Use the energy to dissipate the heat. You will run out of energy only once there is no further heat to dissipate. - Morpheus

Re:The answer is in front of you Neo

[Anonymous Coward]

You have already decided how to dissipate the heat. Now you must understand why.

-Oracle

obviously

[syrinx]

buy several refridgerators/freezers, and leave the doors open!

Re:obviously

[Anonymous Coward]

That wouldn't work, at the back of the refrigerator it radiates its excess heat through a radiator, so unless you put one end of the refrigerator outside of the room, your idea would probably make the room even hotter than it already was. Dimwit

Re:obviously

[confused one]

This will actually heat the room up. Thermodynamics is a bitch like that...

Re:obviously

[Gorm the DBA]

Uhh....if he had the electricity to run the refridgerators...wouldn't he have the electricity to run the air conditioners?

Yeah, I know...-1 didn't get the joke...your point?

Reduce/remove the feedback

[Random Walk]

Part of the cooling problem is that the fans inside the computers blow out hot air, and later on the same hot air gets sucked in again. You should try to remove that feedback loop. Use duct tape and some foil to make sure that the air blown out of the machines gets to the exhaust, rather than into some other machine.

Swamp cooler

[wowbagger]

IF you are in a low-humidity environment, you might be able to use a swamp cooler (evaporative cooler) - this uses a small quantity of water trickled over absorbant pads plus forced air to cool the air via evaporation. In a low humidity environment it can cool air by 15-20 degrees C, and takes quite a bit less power than a compressor based air con unit.

Re:Swamp cooler

[MountainLogic]

Or just water down the roof. A small garden sprinkler can cool a building rather well. It may make raise the humidity and that will be uncomfortable for the evaporativly cooled humans.

Re:Swamp cooler

[angeles13]

I would be careful of swamp coolers. Here in Arizona they are used while the humidity is low all summer (or where someone doesn't have air conditioning).

The air becomes cooler with the water into the air. It also releases charged particles from the the water vapor and dust particles in the air reacting together. This could possibly short your systems.

Good luck! At least you aren't 106 degrees F!

Re:Swamp cooler

[wowbagger]

It also releases charged particles from the the water vapor and dust particles in the air reacting together.

Would you have a link for this assertion? Air with a high moisture content is actually LESS likely to hold a charge, not more.

Granted, wet air + dust can leave mud on your systems, which can be CONDUCTIVE (not charged), but if your server room air is that dirty you are going to have other problems.

Since a swamp cooler is drawing air through a filter media, it is more likely to be cleaner than dirtier.

Now, if you are confusing a swamp cooler with a mist cooler (in which water is sprayed from nozzles into a plenum in which the droplets evaporate) you might be correct in that the air will have a higher particulate count from the impurities in the water.

And of course, if you don't clean your swamp cooler, it will dirty up and start to stink (hence the name). If you don't clean your AC, it will stink. So what?

window?

[jtheory]

Have you ever seen a server room with a window? The general idea is, you want an environment that's as controlled as possible, which means no windows, and if possible, no shared wall with the outside of the building (isolation that usually works out nicely, until you have no reliable power to control the environment in the room...).

Besides, if they had a window, I don't think they'd need to punch holes through a wall to put in any kind of localized A/C.

They might be able to spread some of the heat into the rest of the *building* with fans (I don't think anyone's working there anyway, since they have no lights!)... but probably not enough to keep up.

Personally, I'd like to find out why the original problem can't be solved. Is the power company just planning on leaving things in the current useless state all summer? You might just want to relocate your servers to another hosting center in the meantime (...how small a host are you?).

Re:window?

[Jeff Fisher]

I work at the company also. We're not that tiny of a web host -- we host around 6000 domains currently. The power company is working on finding a replacement transformer if the current one can't be fixed. Last night they were draining the oil from it to check the inside of it for damage. It could be solved today or in a week (depending on how long it takes to get a backup transformer). I honestly believe they are trying to solve the problem quickly as a lot of people depend on the buildings we are located in.

Re:window?

[Anonymous Coward]

The general idea is, you want an environment that's as controlled as possible, which means no windows

All we have in our server room is Slackware. Now it only goes offline when we tell it to.

It doesn't work that way

[Baloo Ursidae]

Assuming you live in a subarctic climate like Portland, OR, consider this: You come home from work in the middle of winter, you turned the heat off before you left. You live in a cheap rental apartment downtown in the basement. You get home, and the apartment is cold. You've got snow up over your windows. You go into the kitchen. Electric stove (turned off), fridge (on). You open the kitchen door. It's warm. Why? That big coil under or behind the fridge is the exhaust end of the cooling system. The fridge heated the room.

Re:It doesn't work that way

[Anonymous Coward]

The second to last sentence in your post would've been sufficient. Concise writing conveys ideas more efficiently. Please try to practice it in the future.

Re:It doesn't work that way

[blibbleblobble]

"You live in a cheap rental apartment downtown in the basement. You get home, and the apartment is cold."

So what, run the servers at home and get free heating, is that the idea?

Re:It doesn't work that way

[Baloo Ursidae]

Heh, I wish. A downtown apartment would mean I'm near a server room to begin with, I just have a shitty, overpriced suburban apartment.

Exhaust heat from the room

[Kent_Franken]

You could use this. But seriously, if you have rack cabinets, hook up the exhaust fans to suck the hot air from the back of your computers out of the computer room. And, keep the air circulating in the computer room to alleviate hot spots.

I'm thinking what you need to do here is..

[Ophidian P. Jones]

Submerge the entire room in mineral oil.

Re:I'm thinking what you need to do here is..

[Unknown Poltroon]

Kinky!!

Why is this modded as +3, Informative?

[Ophidian P. Jones]

No, seriously, I don't have any exceptional cooling method to suggest, ...

The guy admits right up front that he doesn't have an answer, and he still gets modded up as Informative? That's just great.

btw fp :-)

More evidence that he's just karma-whoring.

Re:Why is this modded as +3, Informative?

[HughsOnFirst]

... so I'd focus on reducing heat production instead of dissipating.

Sounds like an answer to me.

you might look for a portable swamp cooler if you have less than 50% humidity

found this on google

http://www.air-n-water.com/swamp-coolers-evapora ti ve.htm

When I was a kid , in the california central valley, one of the tractor repair
guys had a big one bolted to the back of his truck powered by a lawn
mower engine.

Pretty neat to have outdoor air-conditioning in the middle of a field in
105 degree weather. This was when most cars weren't air-conditioned.

Re:Why is this modded as +3, Informative?

[Gruturo]

The guy admits right up front that he doesn't have an answer, and he still gets modded up as Informative? That's just great.

I don't have a cooling solution but I do have a bit of advice which might make his heat problem a bit easier to cope with. I was not offtopic, trolling or making useless jokes so frankly I don't see any problem with that moderation (but I'm a little biased, be warned :-) )

btw fp :-)

More evidence that he's just karma-whoring.

Sorry to disappoint you, no need to karma whore. Been at 50 for as long as I can remember, then this "excellent" thing came. And I like to have first post every now and then :-) , esp. when I have something more than First Post! to say about the subject.

heh, fire department comes to the rescue

[harryk]

literally :)!

I am a volunteer with the local fire department, and we use a 5 gallon cooler with ice water and a fan on top of it. It can truly cool the room immediatly. Its a special design what basically pulls water into the fan, and acts as a mister. It can immediately cool the room, you'd be surprised at how fast you'll cool the room. I would maybe setup two of these in the center of the room, with extra fans to keep the air circulating, low power, and work great. Can probably find them at Sams.

Umm... Condensation?

[DaRat]

Ummm... About the ideas using swamp coolers, regular ice or some other scheme that will end up increasing the humidity in the room.

Won't condensation be a problem if the humidity of the room gets too high? I would imagine that having the water vapor condense on the rack mounted equipment won't be a good thing!

Re:Umm... Condensation?

[Anonymous Coward]

As long as the racks are hotter than the surrounding air (a reasonable assumption since they are a heat source rather than sink), the water vapour won't recondense. That being said, I wouldn't put an evaporative cooler too close to the equipment; I'd suggest cooling the whole room, and direct the airflow with fans to avoid creating cold spots which could encourage condensation. Same goes for ice.

These types of coolers stop working if the humidity gets too high anyway, so there has to be an exhaust for the saturated air.

Come to think of it, there is possibly a greater danger from refrigerative A/C because static charges can reach greater potentials in dry air.

The worst conditions I've ever seen were 42 degrees C and 99.6% humidity (a record temperature in Lismore, Australia). Damn near killed me, but the electronics worked fine, if warm to the touch (but then everything was warm to the touch that day)...

Re:Umm... Condensation?

[N3Bruce]

A way to watch this is to buy a cheap $30 humidity meter at Radio Shack. It won't be terribly accurate, but probably good enough. I wouldn't get terribly concerned until the humidity reaches 80 to 85 percent or so. If the humidity gets too high, shut down the swamp cooler and increase the ventilation. Since computer cases are usually warmer than their surrounding environment, the heat from the CPUs, drives, etc will help keep condensation from many of the more critical parts. If humidity reaches dangerous levels, I would keep the hardware powered up and take active steps to reduce the humidity. Actually, some humdity is beneficial, as it keeps the static electricity at bay, but when moisture starts condensing on tables this is a bad thing. I actually sometime have the problem of trying to get enough moisture into the room during the winter.

During the day, as the sun heats things up, the air can absorb moisture better, and this is the time when you need the help the most. At night, when the temperature drops or on rainy days, moisture already in the air will interfere with operation of the swamp coolers, or their improvised equivalents due to the increase in Relative Humidity. At night, it is probably best to substitute increased ventilation for use of any kind of improvised swamp coolers, but it will be cooler anyway. Experiment, and see what gives the best results. After all, life is just one big self-perpetuating experiment.

Heat Rises, and airflow....

[ComputerSlicer23]

Okay, we had the AC to a our server room blow out. Two things, first, get plent of airflow into the room... Second, get more airflow into the room. If you have to use dry ice, or just plain ice in buckets. We put holes in the buckets, and let it drain out the floor (you've got a drain right?).

Heat rises. So you can keep the tempature of the room close to the ambient tempature of the rest of the building by getting enough airflow, and creating a way for the heat to escape up out of the room. Not sure how your server room is built, but create as many large holes as you can moving the air up outside the room. I saw a room drop 6 deg F, in about 20 minutes by removing all the ceiling tiles.

Oh, and get incredible airflow over the equipment. Ensure the air is circulating over the actual equipment, so the really hot compenents have a chance to pass the energy on to the cooler airflow, thus cooling the compenents. You don't want hot pockets of air right around the computers.

Now, the ambient tempature isn't exactly great from your remarks. Most computer equipment is rated to run up to 80-90F. So you might be able to get by for a couple of days. If not, read up on how A/C works, and how old time A/C works. Build something that works on the same princepal. Got me what it is, but I'm fairly sure using cold water as a heat exchange to literally carry the excess kinetic energy out of the room is your best bet short of finding an alternative source of power.

Kirby

Re:Heat Rises, and airflow....

[DavidTC]

Dry ice won't drain out holes in a bucket, silly. Dry ice sublimates from a solid to a gas without becoming a liquid.

And, frankly, it's an okay way to cool a room as long as you don't have people hanging out in there. Just dump the whole damn container on the floor, or possibly into a pan of water. (Note you don't want too much water, as that will make everything humid, but something like a cookie sheet might be okay.) Then you grab a box fan and aim it over the dry ice.

The danger is that most people can't breathe CO2, so you either can't go in the server room, or you can't keep doing this. (Of course, if you can't go in the server room, you can't keep doing it anyway, but you get my point.) But remember it's CO2, not CO, and it's not 'toxic' or 'poisonious' and it will be very obvious if there's not enough oxygen in the room, unlike with CO, which is impossible to notice.

Re:Heat Rises, and airflow....

[ComputerSlicer23]

No, but the standard stock frozen H2O type does in fact melt into water (I mentioned regular ice in the previous sentence, I only mentioned dry ice because the poster said something about it)... I wouldn't use dry ice, but that's just me. I can get straight regular honest to god frozen water, at the local gas station about 200 feet away from the building. We told them to fire up the ice maker, and we'd keep coming back for more. We had concerns about humitidy, but figured that if we kept enough airflow in the room, eventaually the himidity would follow the airflow out.

I'm not sure what property dry ice has that makes it more compelling then water. In this case, I want the damn stuff to melt to absorb the energy. Having it turn into water is something I can deal with. Having to deal with not breathing in a server room, just seems silly to me.

Kirby

Re:Heat Rises, and airflow....

[temojen]

Most computer equipment is rated to run up to 80-90F.

Ummm... Most computer equipment is rated to 70-80C == 158-176F. If it were only rated to 80F it'd already be melting down at 27C, or just when someone touches a vital chip (body temp == 98F).

Re:Heat Rises, and airflow....

[ComputerSlicer23]

I'm not so sure about that. A lot of stuff can be that hot, or stored in a room that gets that hot, but can't be run in ambient tempature that hot for extended periods of time. They can't dissappate heat that fast. There is a difference between approved storage tempature, and approved to run in tempature. I wouldn't run computer equipment that I cared about in a room over 80-90F. While CPU's can physically be a lot hotter, running them in a room that hot means they can't vent heat appropriately in my experience. The difference in temperature between the part, and the air around it, dictates how fast it can vent heat. So while a CPU can run while it's at 176F in the core, it generates so much heat internally, that it'd overheat if you ran it in a room that warm in a matter of minutes. There's a reason you need those big fans and large surface area heatsinks to run a CPU even in an A/C environment. The greater the difference in temparature, the faster a part can dissapate heat, which is the whole point of heatsinks and fans.

Also, it's not luck touching a lot of computer parts really makes them instantely turn to 98F, if that was true, ice would melt instanteously in your mouth. You "slowly" heat up computer parts while touching it, depending on the heat transfer constant of the part you are touching. Heat transfer is exponential, it's like half life. There is a certain period of time in which you can half the heat difference between two parts (discounting phase change). So if for a CPU, that is 1 minute. That means if the difference between the CPU and the air is 100 degrees, then you lose 50 degrees in 1 minute. Assuming nothing is heating it, you'll lose 25 degrees the next minute, and so on. However, when the CPU is running, it's getting heat added to it at the same time. The tempature stablizes when the CPU is adding the same amount of heat that it dissapates per unit time. So having the room tempature significantely lower then the rated temperature the part can run at is very important.

Kirby

Re:Heat Rises, and airflow....

[temojen]

Electronics parts are rated for the temperature of the part, not the ambient temperature.

Re:Heat Rises, and airflow....

[ComputerSlicer23]

That's not true. I've seen any number of parts that are rated for the temperature of the room. Look at the specs on your monitor, or a DVD player. They have specific temperatures that involve appropriate temperatures for running the equipment. I've rarely seen one over 120-130F. Computers bought in a single unit, normally talk about the temperature of a room. Then engineer did all the work to figure out what the temperature of the room will mean in terms of overheating individual parts.

CPU components are rated for the appropriate temperature of the core of the CPU, specifically because they have a temperature gauge you can measure with built in.

One doesn't walk around measuring the temperature of a computer parts, one measures the ambient temperature of the room. Call Dell, tell them your running computer equipment in a room that's 120F, I'll bet they tell you the warrenty is voided. Tell them the room temperature is 90F, and I'll bet they will tell you to move it to a cooler area before they will help you trouble shoot the problem. The ambient temperature of the room dictates that temperature of the part after you've had it running for any length of time. It's simple physics.

Kirby

How the fuck is this redundant?

[Unknown Poltroon]

just wondering.

Humidifier

[Jucius Maximus]

Getting a 'cool mist humidifier' (not to be mistaken for a warm mist humidifier) can do the trick assuming it's not very humid in your server room to begin with. Last winter I was living in a place that had very dry air so I got myself a CDN$50 honeywell cool mist humidifier from future shop and it did make the room more humid. It also made my freeze my butt off. My room felt air conditioned when the rest of the house felt like an oven because the landlord had the furnace running all the time.

A danger of this include making the server room too humid so it damanges your equipment. Most humidifiers will allow you to set a target humidity level so keep this a few notches below the tolerances for your gear.

Another danger is that you have to fill them with water. Having devices full of water that can be spilled is not so desireable in your server room.

Lastly, humidifiers are a bit noisy so if you need a quiet server room, this is not for you.

I would invest...

[acherrington]

in many marshmellows and sticks. SMORES!!!!

Wow this is what i my job!

[Vroom_Vroom]

There is no free lunch. Dry ice would be dangerous and too labour intensive.....Mixed mode ventilation (essentially openable windows with fans, and other variations) is a possibility. Whats the room like?

Cooling is difficult as you have to dump the heat somewhere....Do you have any extract grilles in the office? Any windows?

Without knowing anything about the situation. I advise: Get a ASHRAE certified (you are in america, i assume) engineer to look at it. Say you're looking for cooling that runs off a backup diesel generator.

Good luck.

ObKarmaHit

[TitaniumFox]

Like most, you'll point out what is wrong with a plan, but you don't have a proposed solution to the problem you've pointed out. I'm sure your supervisors must be happy all the time.

The proposed idea isn't a bad beginning.

If a person is already going to go the route of car radiators and fans, why not spring for a cheapo (so-many-gal/min) electric water pump at TehH0m3Dep07 or the like. A car's cooling system theory is already engineered for you. All you have to do is apply different temperature gradients. This idea has already been alluded to here.

(We are talking about a business or several businesses who are all concerned with a solution to their problem, and is probably willing to put about 500 bucks towards it. I assume they have people they can task, too.)

Open Source, Enterprise-Ready, Multi-platform Cooling System

BOM:
1 50-gal Rubbermaid trashcan (the yellow kind)
2 cheapo car radiators
1 electric water pump (however consumer-ized you need)
1 box-type fan

Method:

Wire-tie one radiator to the exaust of the box fan. Connect the two radiators together with an appropriate amount of hosing, the pump between them on one side. Fill the radiator/hose/pump system with water. Put the loose radiator in the trashcan and fill it with water and ice. Start the pump. Start the fan. The loop is closed, and water consumption is based on ice consumption.

Scoop water from the trashcan as the ice melts. Add ice as necessary.

This idea is GPL'ed.

This is simply a source of cool air. Not huge amounts, and not -20C, either. It'll have to be applied within a framework of air entry and air exit within the server room. A workable idea might be to put the radiator/box fan blowing into a room while other fans were concentrated on moving hot pockets of air around and out of a different door. Modify as needed. Replicate as often as you want. There's no tech support.

(Finally: Yes. I've actually built one of these using 2 Hummvee radiators, a box fan, and an electric pump from a tracked vehicle. It kept the temperatures inside an operations tent in Bosnia much cooler than the 110F+ temps we had.)

Re:ObKarmaHit

[Glonoinha]

One tweak : since the outside radiator you are dunking in water is ... outside, there is no reason not to supercool it using dry ice. Given all the dry ice floating around this thread I figured I would point out a good place for it.

Water shortages

[Anonymous Coward]

In most places water shortages are due only to the municipalities incapibility to filter enough water for its residance. If you are complaining of the water shortages seen in african or middle eastern countries, saving water here will do nothing for them or anyone else. Last time I checked the earth was 70% water. Aside from that life has always historicaly thrived in water rich conditions and existed sparcly in barron places. What this has to do with cooling a server room I have no idea.

Elevator shafts

[morcheeba]

Buildings usually have a whole lot of small rooms; once you get air moving with a fan, you're usually blocked by walls. Could you use the unused elevator shafts as a cold air source/hot air sink? Open the doors 2 feet wide, stick a desk in front of it (one without a vanity cover, like a folding table) for safety, and stick 2 box fans there (the lower pulling cold air from the shaft, the upper near the ceiling pumping warm air back in).

but, the real answer, like others have said, is that if it's so critical, you should have planned for it.

I need Emergency Power Captain!

[Anonymous Coward]

The engines no canna take it Captain! They're gonna blow! I've got to switch to impulse!

Sorry, couldn't resist :)

Evaporative cooling... outside

[temojen]

I'm no HVAC expert, but it seems to me that the most efficient way to get rid of the heat and achieve a temperature lower than the outside ambient would be an open-loop phase change.

But having either CO2 from dry-ice or excessive water-vapour in your server room is a bad thing, so you need to get the heat outside.

Also the description of your situation makes it sound like you may be in a place without so much commercial services, but posssibly with a lot of local inginuity. Otherwise you'd have either rented a big generator or an A/C on a truck already. I'm assuming that you're in a third world country, or a country that has recently experienced armed conflict.

So I'd consider this...

1. Get a plastic kiddie pool, or reasonable facimily. You want it water proof, and with a big surface area.
2. Put it outside in a level place in the shade, preferably at the top of a hill with no obstructions on the uphill and downhill sides.... You want the water vapour to be able to flow down the hill and decrease the local Partial pressure of H2O. Fill it with water.
3. Make an open loop of two garden hoses, an auto radiator with electric fan, and a big electric water pump. The two open ends of the hoses go in the pool, and the pump should stay outside the server room (it's annother source of heat)
4. Put the radiator in a high place in your server room. Connect a 12 volt power supply to the radiator fan. (warning: high current, don't draw power from a computer)
5. Turn it all on (you may need to prime the hoses)

This is essentially the same setup used in a lot of industrial cooling (like chemical plants and nuclear reactors).

I hope you find the solution before meltdown ensues.

Re:Evaporative cooling... outside

[temojen]

Sorry, I see from annother of your posts that you're from Saskatchewan, which is hardly a 3rd world country.

It does, though, have low humidity, which is ideal for an evaporative cooling system... just remember, if the evaporation tank is inside, the PPh2O will quickly rise and cooling effectiveness will plummet.

Fans FANS FANS!!!

[stinkydog]

There is a quick and dirty solution. Fans! Go to Lowes with the company card and stock up on fans, cords, sheet plastic, and duct tape. Get a couple of the monster "industrial" fans and plenty of box fans.

Lay out an air flow in the room (with an "in" and an "out"). If you only have one door, put the "in" on the bottom and the "out" on the top. Place additional fans to force the air to circulate. Set a box fan blowing air into the bottom of each rack. You want airflow you can feel exhasting from the top of each rack (with the heat). It will feel (and sound) like a wind tunnel, but it should keep you gear cool.

Best of all, the whole rig should run on 15-20 amps (The same power as one window air conditioner or 4 beefy servers).

SD

All you got to do is rig up enough of these:

[corkhead0]

http://www.octools.com/index.cgi?caller=articles/s ubmersion/submersion.html

Have fun :D

ummmm

[djupedal]

Why not just try reducing the number of heat sources..that is where the biggest 'gain' will be, considering the overall goal. Shut down systems and external drives, chargers, wireless phone cradles, and all those blasted heat pumping CRTs. Keep this incident in mind next budget, and replace those CRTs with LCDs.

Re:ummmm

[MarcQuadra]

I was thinking that you consolidate all services to one server and gawk as you realize that you spent WAY too much loot on all those blinkenlights.

Energy truck be damned I have 2 words for you

[Anonymous Coward]

Sterling engines

I cant wait for someone to make a decent HSF out of these amazing little units.

shields are failing?

[Unordained]

am i the only one who read the description of these power failures the way you'd expect to hear an actor on a star trek episode rattling off all the systems that have magically blown to pretty little sparks? (over a camp fire made of rocks heated by a phased electromagnectic beam, of course!)

in that line ... i'd say "find the escape pod" ... or perhaps more cynically, "brace for impact!" 'cause i'm rather sure your clients, investors, etc. aren't going to be happy. good luck!

Suggestion that has worked for me

[macrostiff]

If your water supply is cool grab yourself some heat exchangers (auto radiators will do the trick) and some fans. mount the radiators in the bottom of your racks, hose them up, and the rack fans will pull the air through the radiators and cool the racks. You may need some additional exhaust fans on top of the racks to compensate for the restriction intruduced by the radiators. No racks? Corrugated board and duct tape! Run your 'waste' water back to the drain. Some valves and thermometers you'll be able to adjust the flow to minimize water usage. If anyone else needs 'field expedient' solutions, write.

-----Blatant Plug follows-----

I do this for a living :-) RF, networking, communications, site planning and contract negotiation, specializing in tropical and tropical-marine environments. Culturally acclimated to Pacific, Asia, India. Skilled in electrical, electronics, electro-mechanical, mechanical, geodetics, RF propagation analysis and planning, antennas marine and land based. Field project expedition! -WG

Best combination of solutions

[confused one]

1.) Bring in fans to move air into and out of room (should draw less than A/C). Open all windows. Shut off AC, unecessary lighting, monitors, redundant servers, etc., to save power for critical servers and reduce heating. Don't forget to remove tiles from suspended ceilings -- sometimes you can get some convection cooling via this space...

2.) Call Caterpillar and see how fast they can get you a large portable generator -- large enough to power the center, servers, lights, A/C and all. 3.) Keep tabs on the power company, make sure you know how long they expect the substation to be down...

Reroute the water supply

[Kefaa]

This is only July, so the hot times, and outages are going to be coming. While I would go for the mobile generators, because someone may just shut all the power off, here is another alternative.

Find a home heating/air conditioning guy to run a heat pump to the roof or outside the building. Then put the exchanger/fan/etc. inside the server room. One room only.

The building owners have to conserve at the "building" level. In your case, you can claim this unit is critical to your business operations. Its draw will be nothing compared to the building and they may let you keep it running. Total cost? About $2000-$4000 depending on your setup.

Now you can keep it running later, or consider it a complete write-off. Once the real A/C comes back you will not want to pay for both.

As always: IANAL, YMMV and you can get into heaps of trouble doing this without speaking to the owners.

Re:Reroute the water supply

[Glonoinha]

The army guy Bosnia two radiators (one inside with a fan blowing air through it into the room, one outside in a big trash can of ice water, hoses and a water pump, sealed system) is a poor man's heat exchanger that would work pretty well, similar to what you are suggesting but on the cheap side.

Assuming he can keep the ice coming.

Water down the roof / brick walls outside

[Glonoinha]

Actually if there is a good breeze, and you don't mind a continual running of the water, this isn't such a bad idea. Remember the outside temperature is peaking at 86 degrees F, so he is pretty close to begin with.

Liquid Nitrogen

[Oculus Habent]

Alright, this doesn't require major ventilation, nor any electricity. Cost, quantity, and availability may be issues, though.

An open dewar of liquid nitrogen will evaporate off cooling the servers. You could get some flexible tubing, seal it onto the dewar, and pipe the cold air over your largest heat sources. All of this requires no electricity. You might be able to set up some kind of pump to increase cold air flow - I'm thinking gently blowing room air into the dewar to speed evaporation, not pumping liquid nitrogen around.

Assuming the room wasn't tightly closed off the extra nitrogen shouldn't be a problem. If you do keep it locked, you might want to air it out before entering...

I don't work or play with the stuff, so I'm making this up as I go, but it's no/low electricity and guaranteed to be cold.

Rent-A-Server

[irontiki]

Rent a server (or twelve) at your favorite hosting company and get to work duplicating what services you can there. You can probably have a good portion of your stuff shifted over in 24 hours. Even if you can't replace 100% of your setup, any hardware that you can replace with offsite equipment can be powered down saving the limited cooling for the remaining critical hw.

This is a useful exercise even if the juice comes back on tomorrow. Ideally we'd all have an emergency kit that duplicates as much as possible in this fashion in the event of a total site failure.

Others have said it but it bears repeating, your power company may help pick up the tab for whatever measures you have to take. Then again they may pee all over you guys.

Distributed generators might be the answer...

[kernelistic]

At work, we use a couple of 150 kW units from these guys. They monitor and dispatch the units as needed and even do peak shaving for our building when there's a high demand for power on the grid. Our units are gas powered generators and end up costing us less than using power off the grid during the summer and other times of the year.

DTE Energy Technologies has put together a realiable power solution together for our company!

Solar

[cultyellow]

Put solar panes on the roof gets you out of trouble now and a good environmental investment.

just ventilate

[Anonymous Coward]

your servers etc should work to ~40C ambient without any trouble, just keep the air moving and let the heat rise.

honest. the vendors do test to this minimum limit.

Standards like NEBS (www.telcordia.com) specify periods like 24 hours for equipment to work with the building AC failed, to ambient ~45-50C.

Don't do anything with water or dry ice, you're asking for trouble. Just buy a box fan or two and move the air around, and out into the hall or other area with larger thermal mass.