Slashdot Mirror
UPS Setup For a Small/Mid-Size Company?
An anonymous reader writes "We're a small company employing ~30 people and we are becoming increasingly reliant on virtual servers. Unfortunately, the hosts they are on don't have redundant power supplies because we simply don't have the capacity. We currently have one UPS per rack, which gives us about two minutes. This may have been enough time when they were put in — they've been there for some time — but it isn't really enough time to shut everything down in the event of a failure. Domain Controllers alone may take up to 15 minutes. So I'm looking at upgrading the UPSs to ones that would preferably give us around 15 minutes of breathing space and send an email or text alert when a failure is detected. Something that could trigger shutdowns automatically would also be nice. Of course cost is a key factor too. so given all of the above, what does Slashdot recommend?"
Get a generator that can power things from natural gas (or other available resource).
So when the power goes out, it will be seconds before the generator kicks on and the UPS are just there to keep power available until the generator is ready.
Mod me down, my New Earth Global Warmingist friends!
This is a much better solution. Plus, it can melt your face!
When you're afraid to download music illegally in your own home, then the terrorists have won!
Not knowing the load required on the UPS makes it very hard to tell what kind of UPS you need. You need to know how many watts are used in the rack to be able to plan some proper UPS capacity.
apcupsd can be networked between machines and can trigger auto shutdowns of all of them, including VM guests.
Some virtual machine system can also suspend all VMs on shutdown which could be a better alternative then shutting them down. Again, without knowing which VM system you use it's hard to get into details.
We have had good experiences with the HP R5500 XR. You may require a smaller and cheaper model like the R3000 or R1500 depending on your servers.
These UPS are fully supported by NUT.
The months are just too short. I can count the number of days on one hand.
I have 2 3000 watt APC SmartUPSes per rack. They have both Serial and USB notification. Since each rack has about 25 servers, I get around 25 to 40 minutes of runtime for each server. So I have a small PC for each rack that monitors those 2 devices. It connects by serial to the upses, and runs CentOS. Then I have APCUPSD installed and configured in multi-ups mode. On each server, I simply install APCUPSD (There is a windows version), and tell it which UPS it is on. I also configure the appropriate shutdown parameters (20 minutes of battery left for non-critical servers, 15 for DC, and 5 for other critical servers. I also hooked each UPS monitor into Nagios and Munin, so I can track each one's power output and time remaining. So far, it's worked great over 2 "brownouts", and 1 total power failure (a test where I simply tripped the appropriate breakers).
The rational behind having dedicated UPS monitors, is that I don't really care if the loose power while running, so I have them set to never shut down from UPS activity. Then, I simply implemented a script that on power restore issues a netboot command to each server under its control (configured with puppet for Linux, AD for Windows). That way, the whole system (all servers) automatically shut down, and turn themselves back on even if they never really lost power... So far, it's worked flawlessly (and with nagios, I get a text message on my cellphone within a minute or two of a UPS switching to battery (we have 2 dedicated internet connections that are on different power sources and different UPSs.
I hope this helps!
If a man isn't willing to take some risk for his opinions, either his opinions are no good or he's no good
Its time to break out the calculators and do some math. There are two main factors at work here, UPS load capacity and battery run time. I run a series of research clusters at a university, so only the core systems (landing pads, schedulers, auth, disk arrays) are on UPS and all the compute nodes just die at a power hit.
Retrofitting a datacenter for whole center UPS is a very daunting and expensive task, so odds are good you'll be replacing the current rack mounts with beefier units, either pedestal sized units next to their racks or rack mounted units.
When buying UPS gear for work, I aim to hit either 67% capacity with the planned load, or the smallest VA rating that takes 208V single phase, as long as its at least 1/3 under utilized for future expansion. That covers the VA rating. As for battery run time, most of the larger units accept external battery packs to increase the run time. I've never used them, since a 5KVA unit with my load gives me 20 minutes of run time, and if the power isn't back on by then, odds are good its not coming back any time soon.
Another option for extending UPS run time is to prioritize services/VMs. With the appropriate monitoring software on each host, you can configure each host to shutdown when the UPS estimates X minutes of battery time remaining or there have been Y minutes on battery, or both. Less load, more run time for the really important stuff. Almost every UPS I've used (APC, Tripp-lite, Powerware) comes with off the shelf software or there are opensource solutions (apcupsd, nut) for monitoring the UPS over serial, USB, or SNMP (Options vary with mfg and model). My shutdown schedule is: after 5 minutes on battery, power down the compute cluster landing pads. With 10 minutes remaining, power down the file servers with the archival data on them. With 6 minutes remaining, power down the primary file servers. With 2 minutes remaining, power down the auth box/network monitor/iLom control host (This is the only one that can't get powered on/monitored remotely).
I use a Su-Kam inverter at home. It powers a whole room, has a clean sine-wave output (unlike traditional UPSes), and its switchover delay is small enough that the SMPS in computers handle the switchover to battery power properly.
It uses two large lead-acid multi-cell batteries (~car batteries) for storing charge. The last time there was a major power cut, it powered my computer systems for 10 hours (yes you read that right... 10 hours.)
I was laughing at the old APC UPS which did 10 minutes before I had to power down.
This is India btw.. power cuts are common.
Banu
I had a $3000 UPS to keep a big Sun alive. After a couple of years they're $99 on ebay with dead batteries. But they have some of the cleanest pure sine wave power you ever saw. Best. Inverters. Ever. Capish?
The batteries last 5 years then you replace them, period. They're "Sealed lead acid" or "SLA", in plastic cases with two tabs. They come in various sizes. Get the same ones. Be careful where you get them. $8 batteries off ebay tend to be $40 in the wrong store, for example, one is the same as a chair lift, and the medical devices store that have them in stock want $40 ea. Of course shipping LEAD acid batteries ain't cheap.
The batteries for the UPC2200, just as an example, are $150 new for the pair plus shipping. $99 for the chassis (plus shipping and they're ungodly heavy without the batteries) and you have a $3000 UPS.
That'll keep a small server running for a while if you give them one each. But you'd have to be a bit of a dick to have a dozen of these running a dozen servers, what you want is a one ton 12V battery, the kind your phone CO might use, a huge ass inverter and some panic circuit to cut power over to battery when the line goes down. That's the proper way to do it. Once a year they come out and recharge your battery for a small fee. These batteries cost a grand or two but last a long time. Refurbs are fine.
The other nice thing about big batteries is if you get wind or solar stuff added on the to the
building you can just wire that power in to the battery with no charge controller. Cause, uh, there's no fear your solar panels are gonna overcharge a ONE TON battery.
Need Mercedes parts ?
It's no act. I am happy to put up photographs of my setup if you want. It's been working well for me the last year, so I don't have issues recommending it. Apart from being a customer, I have no connections to any inverter/battery company. You OTOH are an anonymous coward. Here is my website. Go find more about the shill there.
Banu
http://www.apcupsd.com/
Apcupsd can be used for power mangement and controlling most of APC's UPS models on Unix and Windows machines. Apcupsd works with most of APC's Smart-UPS models as well as most simple signalling models such a Back-UPS, and BackUPS-Office. During a power failure, apcupsd will inform the users about the power failure and that a shutdown may occur. If power is not restored, a system shutdown will follow when the battery is exhausted, a timeout (seconds) expires, or runtime expires based on internal APC calculations determined by power consumption rates. Apcupsd is licensed under the GPL version 2.
The developers system
http://matou.sibbald.com/cgi-bin/multimon.cgi
I'd go on a Vegan diet but the delivery time from Vega is too long. --brownkitty
Here they are: inverter1.jpg, inverter2.jpg
I'm sorry it's pretty dusty, but this has been exposed to the elements for the past year. It has to be kept outside because of the lead-acid batteries, which need to be ventilated. The stand is an old TV stand reused to host this. The inverter is on top. The batteries are at the bottom.
The little yellow alien looking caps that you see filters that the acidic fumes from the batteries. Each cap tops a cell. The little stick on top indicates the liquid level inside the cell. After about a month, the levels go down in some of the caps and I call the local shop to come and top-up distilled water. Basically the water evaporates whereas the acid is still there, so they fill in distilled water. I could do this myself, but the local shop does it for about $1 so I just let the experts handle it :).
Banu