Slashdot Mirror
UPS Setup For a Small/Mid-Size Company?
An anonymous reader writes "We're a small company employing ~30 people and we are becoming increasingly reliant on virtual servers. Unfortunately, the hosts they are on don't have redundant power supplies because we simply don't have the capacity. We currently have one UPS per rack, which gives us about two minutes. This may have been enough time when they were put in — they've been there for some time — but it isn't really enough time to shut everything down in the event of a failure. Domain Controllers alone may take up to 15 minutes. So I'm looking at upgrading the UPSs to ones that would preferably give us around 15 minutes of breathing space and send an email or text alert when a failure is detected. Something that could trigger shutdowns automatically would also be nice. Of course cost is a key factor too. so given all of the above, what does Slashdot recommend?"
should serve 2 purpose - give you temp power and keeps your IT guys fit
This is sort of off topic, but when was the last time you tested the UPS units that were installed "some time ago". The batteries can eventually go flat. You better check what you have ASAP. You may need to replace them sooner than you think.
I can't remember the brand, but some of the higher end UPS units I have used came with monitoring software. They software polled the UPS unit, and started the shutdown as soon as a power failure caused the switch over to battery.
HTH.
putting the 'B' in LGBTQ+
Get a generator that can power things from natural gas (or other available resource).
So when the power goes out, it will be seconds before the generator kicks on and the UPS are just there to keep power available until the generator is ready.
Mod me down, my New Earth Global Warmingist friends!
Not sure how it compares to some of the purpose built UPSes around, but you could go find a big power inverter like those used in renewable energy systems and get a bunch of deep cycle batteries, maybe a 48 volt bank. A lot of those big inverters have built in chargers and transfer switches, allowing you to basically use them like a UPS. You'd have to roll your own auto shutdown solution, though. And I guess it wouldn't be a true UPS, as there would be some transfer time (20 ms maybe?) as the relay switched over. Though I doubt this would have much of an impact.
Saying "I'll probably get modded down for this" in a post is the best way to get it modded up.
With redundant connection.
Deleted
Not knowing the load required on the UPS makes it very hard to tell what kind of UPS you need. You need to know how many watts are used in the rack to be able to plan some proper UPS capacity.
apcupsd can be networked between machines and can trigger auto shutdowns of all of them, including VM guests.
Some virtual machine system can also suspend all VMs on shutdown which could be a better alternative then shutting them down. Again, without knowing which VM system you use it's hard to get into details.
It's not about the amount of people, servers, or a fixed time limit to preserve power. First and foremost, you need to identify what the critical systems are that need to be protected. These may include the VM farms, NAS storage, obviously the underlying network infrastructure, and at the very least, some management terminals that can be used in the event of a failure. Once you identify these systems you need to reference the electrical in/output specifications. If possible, you would want to measure the real requirements in production with inline monitors or passive taps. After you have built your requirement set (mind you, you may decide it's better to have a few small UPS vs one very very large one) you need to explore what needs to be up, and for how long, and build yourself a model. There are dozens of UPS manufacturers, and tens of thousands of combinations for any sized company. Once you have an outline of the systems and their individual power requirements, coupled with your own requirements for their availability/protected power, it will be relatively easy to build yourself a good level of protection on a small budget. Mind you these devices (UPS) can often be found on the second hand market due to company refresh, datacenter closures, etc. Many can be easily re-certified by the manufacturer directly or a variety of 3rd party vendors who specialize in this type of infrastructure.
No matter how much battery capacity you have, it will eventually run out. If your site truly needs availability, you have to get a diesel generator.
from the UPS
You haven't provided enough information. To answer your questions we'd need to know how many racks, how many watts or Vamps per rack, or even the type of servers you're running. On top of that you mention that cost is an issue, but you don't mention a budget.
Without having that info imagine the following scenarios:
1. You have 1-2 racks with 4-5 piece's of equipment each
Get a Large APC (or comparible unit for each rack)
2. You have 1-2 racks halfway populated
Get an expandable hardwired rackmounted APC
3. You have 1-2 (or more) racks fully populated
Get a large hardwired dedicated UPS
Of course none of this considers anything beyond just bringing the systems down gracefully. If you want something more than that you might want to consider an outsourced datacenter or a generator.
Would it be more efficient to put a new, top of the line server on with multiple UPS?
The open source world has NUT to offer (Network UPS Tools).
We've been using it at work for all our critical servers. It works with pretty much all UPSes, and on pretty much any production OS, so you can use your existing servers and just buy whatever hardware the budget affords.
The linux/unix servers and clients are excellent, and there is a reasonable Windows port for the client (which we've modified a little to suit our needs).
The cost is just your sysadmin's time, as with all F/OSS solutions.
We have had good experiences with the HP R5500 XR. You may require a smaller and cheaper model like the R3000 or R1500 depending on your servers.
These UPS are fully supported by NUT.
The months are just too short. I can count the number of days on one hand.
And one UPS per rack. Is that like 2 servers each?
Deleted
I have 2 3000 watt APC SmartUPSes per rack. They have both Serial and USB notification. Since each rack has about 25 servers, I get around 25 to 40 minutes of runtime for each server. So I have a small PC for each rack that monitors those 2 devices. It connects by serial to the upses, and runs CentOS. Then I have APCUPSD installed and configured in multi-ups mode. On each server, I simply install APCUPSD (There is a windows version), and tell it which UPS it is on. I also configure the appropriate shutdown parameters (20 minutes of battery left for non-critical servers, 15 for DC, and 5 for other critical servers. I also hooked each UPS monitor into Nagios and Munin, so I can track each one's power output and time remaining. So far, it's worked great over 2 "brownouts", and 1 total power failure (a test where I simply tripped the appropriate breakers).
The rational behind having dedicated UPS monitors, is that I don't really care if the loose power while running, so I have them set to never shut down from UPS activity. Then, I simply implemented a script that on power restore issues a netboot command to each server under its control (configured with puppet for Linux, AD for Windows). That way, the whole system (all servers) automatically shut down, and turn themselves back on even if they never really lost power... So far, it's worked flawlessly (and with nagios, I get a text message on my cellphone within a minute or two of a UPS switching to battery (we have 2 dedicated internet connections that are on different power sources and different UPSs.
I hope this helps!
If a man isn't willing to take some risk for his opinions, either his opinions are no good or he's no good
Key points that are missing:
How many servers and network devices. Total amps each unit is pulling.
Total peak watts power usage of each server power supply.
If you need 10000vA and 20000 watts, you're in a totally different ballgame than if you need 500vA and 1000 watts.
Total peak vA of apparent power usage of each server power supply.
For redundant power supply configurations, peak should be double the average.
Pick a UPS that is rated for an amount at least 20% above the peak load, and has the rated run-time for each set of server PSUs it will be powering.
For redundant power configs, two UPS should be used.
You need to get sizing right before being concerned about fancy features like auto-shutdown.
Which most enterprise-grade UPS should provide in some form (may require extra add-in module).
Its time to break out the calculators and do some math. There are two main factors at work here, UPS load capacity and battery run time. I run a series of research clusters at a university, so only the core systems (landing pads, schedulers, auth, disk arrays) are on UPS and all the compute nodes just die at a power hit.
Retrofitting a datacenter for whole center UPS is a very daunting and expensive task, so odds are good you'll be replacing the current rack mounts with beefier units, either pedestal sized units next to their racks or rack mounted units.
When buying UPS gear for work, I aim to hit either 67% capacity with the planned load, or the smallest VA rating that takes 208V single phase, as long as its at least 1/3 under utilized for future expansion. That covers the VA rating. As for battery run time, most of the larger units accept external battery packs to increase the run time. I've never used them, since a 5KVA unit with my load gives me 20 minutes of run time, and if the power isn't back on by then, odds are good its not coming back any time soon.
Another option for extending UPS run time is to prioritize services/VMs. With the appropriate monitoring software on each host, you can configure each host to shutdown when the UPS estimates X minutes of battery time remaining or there have been Y minutes on battery, or both. Less load, more run time for the really important stuff. Almost every UPS I've used (APC, Tripp-lite, Powerware) comes with off the shelf software or there are opensource solutions (apcupsd, nut) for monitoring the UPS over serial, USB, or SNMP (Options vary with mfg and model). My shutdown schedule is: after 5 minutes on battery, power down the compute cluster landing pads. With 10 minutes remaining, power down the file servers with the archival data on them. With 6 minutes remaining, power down the primary file servers. With 2 minutes remaining, power down the auth box/network monitor/iLom control host (This is the only one that can't get powered on/monitored remotely).
It would help to know about how many physical hosts and their wattages though. You say you have one UPS per rack but how many racks are there and whats the average wattage per rack? Exactly how big are the current UPS'es?
There are numbers missing here and if we had those numbers (they dont have to be exact, just close), it would help immensely in finding the best solution for the money.
I use a Su-Kam inverter at home. It powers a whole room, has a clean sine-wave output (unlike traditional UPSes), and its switchover delay is small enough that the SMPS in computers handle the switchover to battery power properly.
It uses two large lead-acid multi-cell batteries (~car batteries) for storing charge. The last time there was a major power cut, it powered my computer systems for 10 hours (yes you read that right... 10 hours.)
I was laughing at the old APC UPS which did 10 minutes before I had to power down.
This is India btw.. power cuts are common.
Banu
It sounds like you may have outgrown the traditional "UPS". They're fine and dandy as long as you're only powering so much equipment. There are some huge options (large in physical size, and more so in price).
A decent alternative may be a DC power room, with generator backup.
Basically, you have banks of batteries, with true sine wave power inverters on them. The power coming in goes to charge controllers. Depending on how you set up, these can get pricey too. There are some nice (and expensive) units that handle both the charge controlling and inverting, and will automatically switch between the incoming power and batteries. Look at the higher end Xantrex units, made for on/off grid purposes.
The less expensive way would be to break up your battery banks by power circuit. Say a 15A power circuit per set. Put a dependable inverter on the rack side of the batteries, and a good charge controller on the line side. Separate inverters for each circuit may not seem like the best idea, and the overall efficiency will hurt because of it, but an inverter failure will only mean one circuit goes down, not the whole place. It's affordable to keep a few spare $300 inverters on hand, where it's harder to ask for a few spare $3,000 inverters.
You'll also want an automatic crossover, if your line power should fail, you can bring up a generator. The batteries shouldn't be intended to last for hours. They should only last as long as it takes to bring up the generator (say 1 minute). Expect that there may be generator problems though. In a prolonged outage, you may need to shut down the generator to refuel, so the batteries may need to last for hours. At very least, if your generator fails, and line power doesn't come back up, you have that hour to gracefully shut down your equipment.
Such a setup can be made to make your company more "green" too. Are you in a situation where you could put a large array of solar panels on the roof, and have enough battery power to last you through the night and then some? You could bring your power bill down to almost nil, or possibly feed back to the power grid (with the appropriate permission and power meter), and make a little money in the process. The long term savings may warrant a raise for you. :)
There are plenty of consultants that can evaluate your needs, and provide the appropriate solutions. As you talk to various consultants, several will say the others are giving you bad advice. Look at all of them, and research them for yourself before making a decision. Remember too, it's in *their* best interest to sell you the most expensive units possible, while you probably want the most reliable and cost effective.
Serious? Seriousness is well above my pay grade.
What's the cost of a good set of UPSes vs simply migrating to a Colo & fatter pipes? Datacenters (most of them anyways) promise at least a few hours of generator uptime, and it sounds like you're already using a colo somewhere (dns relocation, etc).
moox. for a new generation.
Liebert UPS's have an application called MultiLink, which will automatically shutdown the servers it communicates with based on certain conditions, like battery life remaining, among others. It communicates between the UPS and the servers on it via SNMP, and then triggers the shutdown based on parameters you set. Nice little automated way to accomplish what you are after. Of course, it requires a Liebert UPS, but I am sure there are similar solutions out there for almost all large UPS manufacturers.
Co-locate your equipment at a carrier-grade data center in the nearest major city to your location and get a leased line to your premises. A decent data center will have proper battery backup and generators and know how to handle it. They'll also have the time and manpower to do proper tests, etc.
Learning HOW to think is more important than learning WHAT to think.
As you mention virtual servers, I'm going to guess that part of the problem is that you have large-ish servers.
My suggestion would be that if you have different uptime requirements for different services, to segregate them to different machines with a dedicated UPS.
Our office has between two to four 3000VA MGE Pulsars per rack, depending on how much power they draw and how long we need to keep things up. (Although APC now owns them, the MGEs are more power-dense than the APC Smart-UPS line, as they're only 2U each)
Build it, and they will come^Hplain.
You're absolutely right. One place I worked had about 20 employees, 150 servers, but had an income of millions per year. The income averaged out to about $5,700/hr. 12 hours of outages per year could cost almost $70,000 in lost revenue. Is it worth $10k in extra equipment to mitigate that? Obviously.
Smaller companies have to evaluate their acceptable losses. Sometimes it's not worth $100 to make sure you stay up through power outages.
"5 9's" of reliability still leaves 1.14 hours per year of outages. Of course, that doesn't assume that it's all power related outages. Redundancy across physically diverse locations can and will help there.
Serious? Seriousness is well above my pay grade.
You also need to keep in mind local regulations - in our area, once the UPS gets above a certain size you need a hydrogen gas extraction vent for the batteries.
It's no act. I am happy to put up photographs of my setup if you want. It's been working well for me the last year, so I don't have issues recommending it. Apart from being a customer, I have no connections to any inverter/battery company. You OTOH are an anonymous coward. Here is my website. Go find more about the shill there.
Banu
As subject.
http://slashdot.org/~GuyFawkes/journal
The problem is without enough UPS to cleanly shutdown your brief 30 second power outage can turn into a whole day or multiday affair of repairing boxes and verifying and fixing data integrity issues.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
Unless your servers themselves are redundant, you'd be better off with one large UPS with an external battery cage/frame. You can often find those for sale used, but you will almost always need to replace any batteries that come with them. Buy units that use commodity batteries - you can shop around for price among battery specialty shops - often there are substantial savings available. By having an external battery box, you can add battery strings in the future to increase backup time as needed. Having more than one string (if it's wired correctly) also means you can take one string off-line for testing but still have some backup. There are some bigger units that are rack-mountable if this is in a CoLoc, but I think they offer less bang/buck than other units I've seen.
The MTTF of the bigger UPS systems is typically longer than the individual smaller UPS units (not to mention the the effective MTTF of the smaller units is reduced if any one of them failing brings down your whole application), and the bigger ones usually have much better ways to do maintenance on them without taking your AC down (such as bypass switches, which could be available for smaller systems but would have to be wired for each one, which doesn't ever happen).
Next, if you buy a system that is fairly common, you should plan (by leaving space, etc.) to buy another one just like it some day. Then you can do a "Red/Blue" power setup, where all of your key gear has dual power supplies, with one plugged into each of the different UPS systems.
Eventually, batteries run out. And eventually, you will go down long enough that some VIP will recommend buying a generator. Generators are also best purchased used. I used to like the idea of natural gas powered generators - in a true emergency, it can be hard to get fuel deliveries for diesel generators. However, I found that in many areas the natural gas pressure depends on a functioning electrical power grid to keep the pump stations running - so I'd go with diesel power and make sure I had a contract in place that guaranteed fuel delivery even in case every generator owner in town was screaming for diesel.
I wouldn't mess with small gas powered generators - they aren't reliable enough to be worth it.
And finally, whatever you get, make sure you have a process in place that tests the functionality, and a way to make sure that if the tests fail that in is obvious and can get fixed. I can't tell you how many times I have seen backup systems fail, and as it turned out it had been failing weekly tests for a month previously, but the e-mail account that notification went to (or cell-phone number, or whatever) was no longer monitored. Embarrassing.
We have to have a similar deal on the farm, loss of electricity would be devastating, and there's only a few minutes window there. So we have multiple large diesel generators on auto start. They each have four starter batteries,(that is for redundancy as well) and get tested and run periodically. Loss of grid power (or an out of bounds temperature reading, or loss of water pressure, or low propane pressure for the heaters, or low feed levels in the bins, whatever, it is all computerized, etc) causes loud external sirens to go on, and it sends a message to both an external monitoring service, then to the individual farmers, and the entire incident gets logged. That's really the only way, have good backups and fast response. When loss of power can cause an emergency situation that effects your bottom line, you can't have too much "insurance" there. The machinery is designed to respond to emergencies automatically, plus gets the relevant hoomannzz ass in gear to go check things out anyway.
If you can handle downtime and just want a clean shutdown, well, just get a better UPS and swap out to new batteries periodically. Always size one bigger than what you think you need, give yourself a cushion there.
One challenge is knowing how many VA your servers draw, which varies depending on how much RAM they have, how many disk drives, and even how busy the servers are. There is no boilerplate information that can help you with this. To spec your UPS's properly, you need to connect a power meter to each group of servers and monitor the power consumption under typical load.
Once you have an accurate idea of the load, you can look at UPS manufacturer's data to determine how much runtime to expect. For example, if a 2KVA UPS is rated for 10 minutes with a particular battery, you should get 20 minutes if your load is 1KVA.
Might be lot faster.
WeirdStuff is selling UPS cases without batteries IIRC $60 for a 3000VA unit. Add new batteries and you'll save a ton of money. With any used UPS you will want to put new batteries in anyway. We have 6 UPS, all from craigslist. We replaced the batteries and have had no problems.
That all depends though.
I have a site that gets decent traffic. It makes a few hundred dollars per year. It's not worth it for me to spend even $100 on even a good UPS.
You have to consider the need. If it makes a few hundred dollars per year, 2 days of outage is a trivial cost. Say at $1,200/yr ($100/mo, a high estimate), an outage of a full day has a lost income of $3.28. I fix whatever breaks in my spare time, so my manhour expense is $0. If my server were to go down at 10am on a weekday, I'd be there around 7pm to fix it, and it would (hopefully) be up before midnight.
So, you have to weigh the difference between the cost of the hardware, and the loss of income for the period.
Serious? Seriousness is well above my pay grade.
5 9's would be just over 5 minutes and 15 seconds.
I moved away from monster APC & Leibert units a bit over a year ago, and I'm so glad. I encourage you to look at the Powerware BladeUPS units. Each provides 12kW capacity, with internal batteries and the ability to string two additional external battery modules (EBMs) for increased time. In addition, the unit is stackable up to 6 high in a cabinet yielding 60kW (in an N+1) configuration. You can grow it as you need it. Nice Web/SNMP card that can be added for all the info you could want. With the N+1 config, you can shut down any single unit for removal, repair, or battery swaps. These things are so much less hassle than my old equipment that I won't be looking elsewhere for a while.
Not enough information to make a decision here. 30 end users with some virtual servers. . . What's the impact (co$t) of downtime? What sort of traffic do these systems support? Some virtual servers How many physical servers and what are the network, power and cooling requirements. You probably don't want your UPS to run these systems when cooling has been out for hours.
So, work out what financial impact downtime will have. Then you can start looking at options. At one end of the scale, move everything into a hosted data center and bring up a point to point VPN. Let someone else deal with power, cooling and redundant networking. Your midpoint will be rack sized (half rack maybe) UPS that can feed your equipment. Include cooling and or temperature monitors as well. At the low end, buy a couple of rack sized UPS for 15 minutes or so and hope things work out. Don't forget to scale this option to allow for that new storage expansion or the server uplift which will add more CPUs and have larger power budget.
Work out the numbers cost versus business requirements. I worked with one company when we had a prolonged power outage, (greater than 15 minutes) we shut down our development and staging servers, left customer facing equipment and networking running and located ginormous fans on UPS power to purge hot air and bring in outside air. It's always cooler outside than behind the racks.
Bottom line is the cost of downtime against the cost of preparation. Run the numbers, make a recommendation and remember I told you so although satisfying may or may not be good long term career option.
15 minutes to shutdown domain controllers?? Get rid of 'em!
Might as well have a cluster of UPS's like a cheap APC where one main ups has 3 backup ups's.... I know its a quick and dirty hack, but it should work I think... :-)
What about FedEx or DHL? Are you just a big fan of the color brown?
Be Excellent To Each Other
We have around 20 servers, and the usual routing/firewall/switch/remote access stuff.
Our 16kVA Symmetra LX is 70% loaded and gives us about 30 minutes of runtime.
Total cost for the UPS and outboard step-down transformers was around $10k
-ted
Not properly maintained propane storage/transfer sites have be known to explode. For instance: the 2008 Toronto explosions, the 2006 explosion, and the Feyzin disaster. Propane is highly vulnerable to a BLEVE - Boiling Liquid Expanding Vapour Explosion. Many more smaller propane exposions have occured, they just did not make the news.
I've seen what happens when Propane explodes. I would think twice about using it for an emergency backup fuel source.
You run Windows Vista on your servers?
First, do the math. Calculate the run-time power consumption of your servers. The easiest way is to use real numbers from the existing UPS units, or by using a kill-a-watt.
Second, buy APC UPS units to meet your need. UPSes are rated for the number of actual watt-hours they support. If your servers consume 1500 watts, and you need them to operate for 30 minutes on battery, you'll need at least 750 watt-hours. Considering adding 50% for battery deterioration, and future expansion.
No, I don't work for APC, but they have worked exceedingly well for me and they are supported on practically any operating system you run.
In my network, we have a Linux machine monitor the UPS via USB serial cable using apcupsd, which you can find in your distro's repository. Then all the other machines are linked to that machine also using apcupsd but with an ethernet target instead of USB. When the UPS fails, the others find out within 20 seconds (or whatever your poll time is), and take action. Any data the USB host has, the others have from the network. It is easy to fetch the data via SNMP, graph it in Cacti, etc.
With most UPS systems on the market, the battery is sized for an estimated 5~10 minutes of run time for the power capacity of the UPS. There are actually two things to consider: The storage capacity of the batteries and the power capacity of the UPS. So, if you have a 1400VA UPS, you'll probably only have a couple 12V 17AH batteries. To get a longer "runtime" commercially, you would have to get a higher capacity UPS. This would cause you to buy a heavier duty UPS than you really need and spend a lot more for it, when all you really need is higher storage capacity in the batteries.
If your company is really small and cheap, and has a hands-on mentality, which in this economy seems to be making a come-back, you could take a properly sized UPS and extend its run-time capacity merely by buying a couple batteries. For instance, I have an APC 1400 UPS for my office servers, I got two 12 V 35VA batteries in addition to the standard 17A ones that came with it. I tripled the run-time storage while leaving the load capacity the same.
UPS companies don't like this, because they like to sell bigger products, but its your money.
Have you considered trying a bunch of these?
"5 9's" of reliability still leaves 1.14 hours per year of outages.
5 nines - 99.999% - reliability is about 5 minutes of downtime per year, not more than an hour.
~Idarubicin
Sorry, my math failed me this morning. Don't expect too much from the math department on weekends. :)
Serious? Seriousness is well above my pay grade.
Actually Windows tends to do the best with unplanned shutdowns, I've had way worse times with Solaris and Linux boxes then I ever had with Windows. I hear AIX and zOS are even worse, but they are generally used in professional datacenters and so unplanned power outages are much less common.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
If you use Hyper-V VM-s then don't go into APC SmartUPS if you're not willing to spend additional money for their "dedicated" network shutdown software...
"I can smell the curry from here."
Shmuck detected.
Large lead-acid batteries (or Optima batteries if servicing is a concern) are cheap compared to most UPS batteries, inverters are old, proven technology, and systems using them are easy to set up.
"This post is an artistic work of fiction and falsehood. Only a fool would take anything posted here as fact."
Liebert makes some really nice middle class UPS. Plus they have external chassis that can exptend uptime. Add the web card and you got email alerts whenever it does anything.
You say you don't have redundant power supplies because you don't have the
capacity.
Redundant power supplies don't double your current draw... they allow you
to continue to operate if one power supply (or one power feed) fails.
If you need uptime and you DON'T have redundant power supplies, you're...
misguided.
Ideally you'll have separate circuits that can independently handle
the full load, too, but don't be fooled into thinking a UPS will solve
everything.
That said, APC SMART-UPS are quite good, but if you have a big enough
operation you might want to look into a bigger datacenter UPS solution.
You might consider a larger UPS from Eaton like the PowerWare units. These units can be sized to power the entire data center and your desktop computers. They can handle printer and photocopiers and easily become worth it when you start looking at purchasing a pile of individual UPS units.
We have two PowerWare units because our servers have redundant power supplies and this gives us full redundancy in the data center. We can also run the couple of laser printers we have in the data center. Then we wired half of the building to one UPS and the rest to the other UPS. The orange outlets at the desktop are for the computer equipment and are wired to the UPS units. We get a lot of thunderstorms and power fluctuations and now our operation continues faithfully. Note that all of our networking gear and telecomm are also on the Eaton units. We saved thousands of dollars in our situation. We have 48 servers in 14 racks and about 350 desktops.
Lease companies take care of the messy bits like maintenance, and you don't need to worry about the ability of the technicians - it's the companies problem.
Of course as part of this you will probably need to review the state of your existing UPS's - if they are no longer covered by the manufacturers warranty then the batteries probably need replacing by now.
With the current availability of fairly inexpensive bandwidth, why are you running servers at your location? There simply isn't much justification for any business not in the fortune 500 to go the route of "build your own" Catacenter. If it must be up, look at the option of renting rack space from a Telecom provider that takes care of generator power for you. Most of these will do a rack for a couple hundred a month that includes the generator backup. You may need to get a small UPS that handles the "blip" until the generator kicks in (they usually tell you that you need a few seconds of UPS), but it sounds like you already have units to put at the bottom of the rack that will handle that. You then have servers that will survive as long as the provider has fuel. Anything else is going to cost you far more. Most likely you can find one that will provide decent bandwidth from your location to theirs and provide you with an Internet connection at the Colo that is less expensive because it doesn't have the local loop to your facility. This probably would offset much of the cost for bandwidth that you will need from your office to your servers at the Colo.
An UPS is not an alternative to redundant PSUs, it just seems that way until your PSU fails.
I've got a server room with about 150 servers (physical and virtual) every physical server got redundant PSU and the whole room runs on a PowerWare 9305 30 kVA UPS.
...and yes I should have redundant UPSes too, I just dont have the room for another one.
I know with only 30 end users getting the funds to do this can be a challenge....It was for use (local council) until literally every time our HVAC systems would draw extra because of heat or cold, the whole phase would trip out...Thus we would have a mad panic to the DC to shutdown servers manually! Finally got a electrician (big hint here, get one familiar with Datacentres!) who looked at all three phases of power coming into our building. The first thing that we did (after a facepalm of the original setup) was to load balance ALL the power coming in, and moved the Datacentre power to a different phase than the HVAC - a no brainer, but no one before me really had thought of this! The second phase of what we did, was to run our own 3 phases right from the main switchboard into the Datacentre with the main breakers in the Datacentre, then a couple of sub-panels.We now have our Datacentre with a North/South power distribution feeding two Liebert UPS's that in turn, feed PDU's. So if we loose one phase, things will "scream" and email off alerts - but things will stay up. We still (because of the age of our first UPS) need a physical server connected to the UPS via serial, but the newer one has a web card in it. I have a script that if both UPS end up on battery mode, a graceful suspend of the VM's starts, with our exchange and DC's being the last to suspend. We happen to be a Windows shop, with a HP C7000 blade enclosure that is our VMware farm, connected to HP EVA 4100 which the two SAN switches have up to 96 hours of write cache. The other thing that I am looking at on its own small UPS is a MikroTik 411u with a prepaid 3G to give us independent source of SMS's. There is little point of having a SMS server that is going to be shutdown because it is on UPS. The key thing to remember about UPS (as far as I am concerned) - is that they are not a replacement for mains power. The suggestions for a generator are interesting given the fact that you only have 30 end users, the expense, maintenance of them would make them cost prohibitive tive in your environment. But 30 minutes to bring up a DC?
- Check your batteries by opening up your UPS unit maintenance doors. At a guess you are probably using APC 700/1000 class units that have bulged batteries or blown batteries. An APC 700 should have a 5 minute burn time on a 1Kwatt load if it is working. An APC3000 class unit has about a 15-20 minute burn time on a 1Kwatt load.
- Server Rack wiring: If your servers have redundant power supplies, and you have the money for a duplicate UPS unit, put each power supply 'side' into a different UPS.
- Run the ups monitoring software.
I personally like the following configuration:
1 APC3K class UPS for the entire rack.
1 ACP700 class UPS for every server.
- "Split" redundant power connections (see above)
- All servers monitor their APC700 class UPS.
- Script shut down sequence in proper order.
Your shutdown/reboot times are a little scary. Delete any "dead" user profiles in your C:\Documents and Settings\ directory. Defrag your sys and data drives, disable all unneeded services. On your exchange server, go into the network configuration and change the DNS server names to the IP#'s of the DNS server(s). Unbind NetBios from TCP/IP on the servers that don't supply printer/file sharing services. Go into your exchange server managment utility, look at your mailboxes and delete any dead email boxes. Clean up your mailbox stores by defragging them.
It's not perfect, but most of that should help.
At home, I use 1 APC UPS per PC, except for the laptop. These cheap units give about 30 minutes for the PC and its immediate peripherals (displays, external disks). The PCs run Ubuntu, and automatically recognized the UPS units when they were connected via USB ports. Another UPS powers the fiber switch, headless server, and router. Yet another keeps the network printer and an ethernet switch powered. Total of 4 APC UPS units and less than 350euro investment.
We need this, because we're in the countryside, where power glitches are depressingly common - but often last just a second or two. If the outage lasts until the UPS goes below 50%, then the systems shut down gracefully. By then, of course we would have saved any work in progress. Keeping the whole LAN in operation during this time is essential.
Those who can make you believe absurdities can make you commit atrocities. - Voltaire
Since Windows has so many 'unplanned shutdowns', they probably had to make it handle them well.
I don't need any fancy new UPSes, but I sure could use a whole lot of cheap ones - maybe 150KWh total capacity. Where's somewhere to buy the lowest $:KWh that can actually be gotten from some old ones, even if they're a little worn out, so long as they'll last another 5 years at that superior $:KWh? Even if they fill a whole room.
--
make install -not war
APC sells a Symmetra 3-phase UPS from 20kA to 80kA. The small ones only take up a whole cabinet. Run two power distribution units to each rack and plug the redundant server power supplies into different PDUs.
Batteries are hot-swap able.
power can be switched to bypass if the unit itself needs to be serviced.
Modules are available that connect to the network to provide monitoring and alerting capabilities.
software is provided that monitors the UPS and will shutdown servers if battery runtime gets too low.
APC can be contracted to perform annual or bi-annual maintenance.
The unit is very expandable should your needs grow, and is usually as simple as adding an additional power unit or just more batteries.
I am starting a (very) small business, with one computer to be left on 24/7. The power requirements are not large, but in my estimation, the biggest worry I have is not a power outage lasting tens of minutes (or hours or days), but rather fully transient. Primarily, if I can keep the server from going hiccup for 10-20 seconds --either a lightning strike or a brown out-- then I will feel comfortable. I had a UPS on a small machine that would do that, (good for maybe 3-5 minutes) but my biggest concern was things like transients, spikes, ring waves, etc. No lightning strike or brown out ever lasted more than 30 seconds, and the UPS and surge protectors might get hammered, but not the server. If your revenue and needs require 10-20 minutes, then go get more (even a backup generator that will keep things going for days).
I run the infrastructure for a couple of small companies. I use PowerWare 9120 fully online UPS's that can power the rack of 5 servers and misc routers, firewalls, whatnot for about an hour. For around $6K I had a Generac emergency auto-failover generator installed which has worked flawlessly. Power goes about for about 45 seconds and I'm back online with full power.
When running on a smallish generator there are some things you need to know. You'll need fully online UPS's. The cheap consumer grade ones you buy at Best Buy probably won't cut it. Here's some more info: http://yml.com/fv-b-1-71/Generac-Guardian-Emergency-Standby-Generator.html
har har har har har I'z so phuny!
If your domain controllers take 15 minutes to shut down, then:
- your domain controllers are running on very dilapidated old hardware
- your domain controllers are sluggish because they are infested with spyware & rootkits
- you have the biggest active directory installation anyone has ever seen
Domain controllers should take less than a minute to shut down, and typically much less.
You sound like you're big enough to have a "computer room" (even if it's just a closet).. Once you get to that point, you need to start involving professionals in the design. Have you looked at Emergency Power Off switching for the DC bus (so the firefighters don't get electrocuted)? What about fusing of that battery bank? (DC rated fuses and switches?) What about venting requirements?
I have apcupsd setup so when the battery is a 50% the system hibernates to disk. That way when the power comes back on they pick up where they left off and an extended power outage turns into a break instead of trying to remember what was gone on. I justify hibernating at 50% battery left to avoid deep discharging the battery and there's a few minutes of reserve capacity if I really need something before the power comes back on. The wake on lan feature is one thing I haven't implemented yet, and is needed for one of my systems that doesn't have a BIOS mode always turn on when it gets power.
What kind of hell-hole has 12 hours (!) of power-outs per year? I think I haven't even experienced that much power-outs in my life (though, admittedly, they can occur at night while I'm asleep).
Condition your incoming power with one of these babies:
http://www.gamatronic.com/PowerPlusUps.aspx?prod=52
Nicely expandable, redundant and just perfect for small to medium sized server rooms.
P.
Beer Coat: The invisible but warm coat worn when walking home after a booze cruise at 3 in the morning.
I'd love to see photos of your setup.
The Signal/Noise ratio can be improved in two ways. Remaining silent is the OTHER way.
- Determine your power usage per UPS
- Look at the runtime specs (at that load) for a new UPS
- Purchase the new UPS.
It's really almost that simple. Really.
The only other considerations are
(a) Realizing getting an "el-cheapo" UPS is often a great way of ensuring failure
(b) Properly determining any other needs the UPS should handle (active line filtering, etc)
That's pretty much it. Yes, it may cost you more money than you want to spend, but sometimes spending the money is the only option.
Now, there are other alternatives that may save some money.
- Find a used UPS that has inexpensive batteries. Before purchase, make sure all the electronics/charging circuit(s) are functional. Then purchase it and replace the batteries. I've scored 3 decent APC UPS that way and then replaced the batteries in them, leaving me with three "virtually new" UPS units for far cheaper than buying the exact same units new.
- Find a new or used UPS that does NOT meet your needs but allows adding battery expansion modules. But first, research it for "aftermarket" expansion methods. As an example, the APC UPS units I have allow such. I can "simply" find the correct AGM batteries, wire them correctly and hook them up directly to the battery harness for my unit... apparently, some crafty people out there have used the same model UPS in such manner and gotten DAYS worth of runtime. Of course, it also means DAYS worth of charging time.
- Look for an alternative method of power backup, such as an "instant" on generator that is "electronics AND computer rated" (and then still run the UPSs on the inside end of it and ensure they do line filtering, sine wave correction and power out-of-range (brownouts, over-voltage) protection.
- Create your own UPS units using high quality true-sine inverters and decent deep-cycle batteries. You can find the inverters for such at any decent "off-the-grid"/solar supplier. Get one that will recharge the batteries when main power is restored. Also ensure it can charge as many batteries as you need for the required runtime. The advantage of this method is, with the correct inverter, you can add a ton (possibly literally) of batteries to have massive amounts of runtime - and you can buy higher Ah batteries cheaper. Ensure the batteries are properly cabled, stored and (sealed or otherwise) vented.
StarTrekPhase2 - The Five Year Mission Continues!
Hello-
Most larger APC units have an external battery connector on the back. It uses an Anderson connector to connect an external battery pack. Also, you can chain battery packs, to have more than one. The external battery packs are expensive but they can be worth it for this very problem.
There's also the DIY method for VERY long uptime!
At my shop, in the locked cabinet, we have an ancient APC 1400 unit, with the batteries REMOVED, and two wal-mart deep-cycle marine batteries connected to the external connector. The batteries are 24-DC 12Volt, 75 amp hour batteries in series, for 24 volts. (grey connector) We disabled the beeper in the unit (with pliers... YANK) This keeps a rack with 7 computers up for several HOURS.
This worked so well we decided to go one better! In the main closet, with extension cords running to the machines in the rest of the shop, we have a 3000VA APC unit. It has a 48 volt input (blue connector), not 24. On this sucker we put FOUR 27-DC 115 amp hour deep-cycle batteries, again from Wal-mart (best price, sorry) in series, connected to the back of the unit with the external connector. For these I got lucky and found a really nice set of cables.
This sucker powers all the machines outside the rack, as well as some flat panels, and a desk lamp. (So we don't break our necks!)
I was worried that the current would be too much for the charger, but I've run them down and back up again and they're fine. I guess the UPS units are made to handle two or so external battery packs, so they handle the lead-acid jumbos just fine.
We've had these for a few years, and even had the same power outage here in SF that took out 365 Main a few years ago and had no problems. I need to do another plug-pull test, but our loads are not that high and we can get up to eight hours! Again, pull out the beepers or you'll pull out your hair.
Some tips- You need nice, big cables to do this. Also, there are some code issues for large lead-acid batteries, so if you want to be completely legal buy the APC external battery units. If you buy at Wal-mart find some old dead lead batteries before you go or they'll charge you $9 core per unit. They really don't care if it's the same kind of battery, I traded in the old APC Sealed units for the marine batteries ten times as big! Unlike the sealed ones, the deep-cycles are spillable so be careful. I have never had them spill but if you tip them over they will probably spill some acid.
Have fun!
=Rich
P.S. If you want more info on this or pictures, you can email me (public account) at rich underscore humphrey at yahoo
An AD DC for 30 odd people takes 15 mins to shut down? I'd fix that first if I was you.
Next, decide what can stand being killed off immediately as soon as you switch to battery - eg one out of two DCs, backup servers, !database servers. It's your decision here.
Next look at your requirements ie kVA to run systems, decide how long to keep them alive in the event, do the maths and then look at the price list. Oh and buy a small diesel generator and get people educated in what to do with it. If your nearest petrol (gas) station is 20 mins away then you'll need at least 45 mins on battery unless you can store fuel on site.
Screw diesil. A natural gas generator will serve you far better and reliably.
If gas is available at your location that is. If the gas goes out there are likely far larger problems to contend with.
However I would probably fix your domain controller first. 30 people... 15 minute shutdown? What's up with that?
I am very small, utmostly microscopic.
I've started using laptops for internal servers. You should too. They're small, energy efficient, and have huge upss
Quit talking about generators and shit.
Here's my anti-recommendation: IBM. Forget IBM when it comes to cheap consumer-level UPSes. Didn't work worth a shit. APC worked much better, though can't say they're good enough that I'd recommend.
The sad truth, though, as a few others have mentioned, you really have to replace 'em every few years. Even if you hardly ever have outages, they only last 3-4 years. No fire and forget; your servers will long outlast your UPS. Budget accordingly.
Redundant power supplies are to provide high availability in the case one fails. You can look at a couple things though.
Most UPS vendors have a network-based communication between server and UPS. It may be a premium feature, but it should shutdown a server when the power goes out.
As for power in general, if you need longer times, I would look into some form of generation. APC even makes fuel cell in-datacenter generators. You need a Hydrogen supply and a water drain.
Our site runs 24/7.
We have two Eaton Powerware UPS'es running our server room in tandem. They are linked so they stay in phase with each other and incoming mains power. both UPS'es run at about 40% capacity, so if we lose one, or need to shut one down for maintenance, the other can handle the whole server room. We've had problems in the past with UPS'es suddenly failing; so our critical gear runs on the assumption we'll have a UPS fail again at some point, or we can at least keep running whie we change out a UPS and/or it's batteries.
Our setup is designed to keep everything running until our diesel genset kicks in, usually within 2 minutes or so - I think the UPS'es have about 15 minutes capacity at full load. If that fails to start then the UPS monitoring software gracefully shuts everything down. We test everything every three months by simulating a power loss (ie, we put the relevant apps in maintenance mode, throw the mains power switch and watch what happens.)
The genset also powers the airconditioning in the server room. There's no point keeping your servers running for hours if they're not kept cool; they're just going to cook once your server room gets over 50 degrees anyway.
If you're expecting to run on backup power for more than an hour or so, make sure your backup power solution can handle your cooling requirements too! Also make sure your cooling comes back on when power is restored!
By the way, some people are saying "secondary site instead" - in my opinion you need both; ie a disaster recovery site and backup power for both sites.
Let me translate this for you: "I have no clue, don't have the slightest idea about how to gauge power requirements, despite the fact that I have access to all of the hardware, and, if I HAD a clue, could do this myself. So, please, do my job for me?"
I am of the opinion that "Ask Slashdot" should be renamed to "Do my job for me" or "think for me" or "Google for me" - all in favor - respond by saying AYE.
Here they are: inverter1.jpg, inverter2.jpg
I'm sorry it's pretty dusty, but this has been exposed to the elements for the past year. It has to be kept outside because of the lead-acid batteries, which need to be ventilated. The stand is an old TV stand reused to host this. The inverter is on top. The batteries are at the bottom.
The little yellow alien looking caps that you see filters that the acidic fumes from the batteries. Each cap tops a cell. The little stick on top indicates the liquid level inside the cell. After about a month, the levels go down in some of the caps and I call the local shop to come and top-up distilled water. Basically the water evaporates whereas the acid is still there, so they fill in distilled water. I could do this myself, but the local shop does it for about $1 so I just let the experts handle it :).
Banu
Going cheap? I've had to. Here are a couple options that are surprisingly cheap:
1) If your UPS can actually handle the whole amperage of the servers, but the batteries only give you a few minutes, consider going to Wal-Mart or the nearest auto parts store and buying a few deep-cycle marine batteries. They aren't cheap, but they aren't exactly bank-busters, either. Find out how many volts your UPS is (typical voltages range from 12 to 48 volts and are basically always in 12-volt increments, EG: 12/24/48. If you are frightened by wiring work, finding an electrician buddy isn't too hard typically, but it's barely more complicated than jumper cables.
Years ago, I got ahold of 4 deep-cycle marine batteries that were about a year old for a song from an upgrade. I took apart my craptastic APC UPS and wired in the deep-cycle marine batteries. It worked fabulously for about 3-4 years, and gave me some 8 hours of battery life for my two small servers. With the bigger batteries, the UPS wouldn't even start a low-voltage warning beep for nearly an hour when the power went out!
2) The other option is probably one you hadn't considered, because you thought it would be "expensive" - and that's a dedicated colo. Surprisingly, you can get a half-rack and a burstable 100 Mbit Internet connection at a colo starting at about $200 or so per month. And as the amount of bandwidth being consumed climbs, the deal gets even sweeter.
I have 10 servers in a half rack serving hundreds of high-paying clients, our company's primary revenue is in a complex, hosted application, and our hosting bill is less than either our office power or our phone bill. Despite this, we make sure to brag everywhere we can about the high quality of our hosting! If your needs exceed a certain minimum, dedicated hosting is the way to go, and we have redundant power, redundant network feeds, all the bandwidth we could ever use, and 24x7 monitoring to boot! It's a really, really good deal if your needs are compatible.
I have no problem with your religion until you decide it's reason to deprive others of the truth.
Change the inverter circuitry to a continuous running configuration - of heat sinking the power transistors etc., and fan cooling them and other components. Then add in a bigger battery bank. Test with a dummy load of 120% for 30 minutes and then put it into service.
.
Voting up, Voting down - If I really gave a fuck about your approval or not, I'd come and ask you.
First off, no redundant PSU because they "don't have the capacity"? You not only need a UPS, you need an electrician.
Second, no sizing data? What is the peak load at startup per rack? What is the average running load per rack?
What would be your ideal runtime?
How many racks?
Peak Load will tell you how large a UPS you need (in KVA), Average load and run time will tell you how many expansion batteries you need to buy.
Buy the management card for your UPS and configure it to send you email alerts. While you're at it load the client software on your VMs so they know when to gracefully shutdown.
Ask your electrician if you need 3phase or 2phase (my money's on 2 for you)
Then get the electrician to install enough dedicate power circuits to drive your new UPS(s). While you're at it, ground the racks.
Then call your AC guy, because once you've done enough math to buy a UPS you'll have enough data to correctly size the AC in your server room.
When you find out management doesn't care about extended recovery time and data loss, and they shoot down the CAPEX for your UPS project, you can focus on things that really matter: good backups and your resume.
Actually, "5 9's" is 5.256 MINUTES per year of outage...still not zero...and amazingly difficult to achieve.
Not for everyone but google as a really clever solution http://news.cnet.com/8301-1001_3-10209580-92.html
I don't like those big UPSs, they are expensive, a lot of unneeded HW and they barely work when you need them most (expensive, small discharge time, batteries losing charge capacity .. )
So for our own datacenter we keep the currently bought UPSs and added some laptops for really critical services, we virtualice a lot too (OpenVZ) and both technologies enables us to degrade services gracefully and still keep high uptimes, laptops are great, cheap, and batteries can last up to 2 hours. It's not a solution for everyone, but works if your platform can be adapted to the architecture.
You want to say 99.999% of availability, not reliability.
This is BTW just 5.26 minutes of unavailability per year on earth, but maybe you live elsewhere? ;-)
For Solaris you need to ensure "logging" is enabled in the vfstab for each ufs filesystem. Once this is set the box won't sit there in single usermode after an unscheduled powerdown, it should fsck without any issues.
I've been using UPSs for close to 3 decades and I'm not sure they have saved more more grief than they caused.
My latest attempt to get things working correctly involves a telco grade N+1 type system using 4 1KW (maybe they are 750w?) inverter modules form Eltek Valere that run off a -48V battery bank made up of 8 truck sized deep cycle floating cell batteries in two banks. Its a pain to check the fluid levels but it can cope with far worse situations than the sealed lead acid batteries. The battery bank is charged by a a pair of rectifiers also from Eltek Valere. The next goal for that is to hook up a KW or two of solar panels to the battery bank. The idea is to power some things directly off the -48V system as well.
The second system I run was intended to use a Selectronics PS1 and a charger/inverter but it had a habit of trying to feed the grid for a few milliseconds before it decided it couldn't and then shut down both sides of the load. Its now uses as a batter bank charger and a UPS for things that need protection but won't be a problem if they fail due to power outages. The PS1 is the cleanest battery charger I have ever seen but its a bit expensive for that role. I wouldn't have a problem using one of their units in cases where the power was very bad and I had a proper generator.
I also run small UPSs in the racks for things that should have ATS but don't. I stand by the statement that they have caused more harm than what they have helped. My old sun power supplies can take worse spikes than the 2RU UPS. I had an idiot electritian who kept tying phase 1 to phase 2 when my gear was on phase 3. Gee... every time I do this, it goes pop! I wonder why it doesn't work and try again just about the time the machines had rebooted. 240V 3 phase pumps 415V or more into odd phase out when you connect the other two. outch!
A word about Automatic transfer swtiches and dual power supplies. When they have smarts built into them they can get into the game where "opps, phase A is down, switch to phase B" followed by "Opps phase B is dropping fast but Phase A looks ok now". The power flapping ends up doing nasty things to the grid power factor and modern power meters will take that into consideration and may charge far more for the power. Around here normal power can be had from $.08/kwh but bad power factor power can cost $.50 a kwh.
I monitor my power with a Sentron PAC3200 which is a 3 phase, 4 inch digital volt meter with an ethernet jack. I get data every 30 seconds on volts, power, power factor, THD and other things it knows about and I plug that into rrdtool so I can look at the nice pictures after things break. If your only doing 10A, you can connect the thing directly and it will give you enough precision to tell which server power off even if they are nearly identical but if you take more than 10A you need current transformers which introduce more slop on the data. I put my PAC on the back end of my 3 UPS systems but I would like to get one for the front of the charges, the building as a whole and the computer room A/C systems.
Did I mention I hate UPSs?
...Don't Believe Them. Most UPS' are woefully under-powered, with the intent to offer only a few minutes of uptime. I'm looking for a solution that uses external automobile batteries, instead of those "convenient" internal, sealed batteries. I recently acquired a 3,000 VA Rackmount UPS and with the load of a single 5W clock (to measure runtime), it was dead in 8 hours! About 2/3 of power in the batteries is used just to power the DC-to-AC converters, so count on a lot less power usable than specified.
check out deeyaenergy,com, they build long kWh batteries which are cheaper to run than lead acid.
In order to be environmentally friendly so we don't have global warming and end up in another ice age there is something you can do. We all must do our part or you will get blamed next time we see a bald polar bear shed of its fur drinking an umbrella drink and munching deep fried penguin.
Put up a wind charger system and backup array of solar panels on the roof. Set aside a spare office and fill it w/ 12v car batteries and the appropriate inverter boxes for your countries power needs. That way you can save a poor little penguin from the deep fryer and Al Gore can finally shave his male poser beard.
O.K., that was a LOT tongue in cheek and I apologize to any penguins who may have been offended by my mention of Gore. I'm still burning from reading another article on /. about Minitrues subdivision of Miniclim.
Seriously though I'm not even a greenie and I firmly believe wind and solar power not only make your properties look techie cool, but will also offset that horrible electric bill.
There is a DIY site that can show you how to accomplish all this. http://www.otherpower.com/otherpower_wind.shtml I've spent many fun hours planning my own system to get off the grid and rid myself of a paycheck sucking bill.
You may think I'm joking, but consider this, do you want to entrust your servers to the wiles of your local stupid greedy unionized electric company and some stupid UPS that needs replaced every couple years or take matters into your own hands?
Food for thought,pass the penguin.
*Repent!Quit Your Job!Slack Off!The World Ends Tomorrow and You May Die!
Most have programs designed to help business with power issues; they may have a solution that fits your needs and budget. It's free to ask; and you may wind up with an affordable maintenance free solution.
I'm a consultant - I convert gibberish into cash-flow.
Windows can fsck itself up without any extra config work. It works right out of the box! Come to think of it.....you don't even need the power outage....
If the size of your company is best described by ~30 employees, the fact is that you will never be able to make it financially feasible to run your own infrastructure with a high enough service level. In this case it is best to move the infrastructure out to a service provider. On the other hand it is possible that you are not accurately describing your operation (multiple domain controllers for 30 people)?
UPS should only last long enough for your generator to fire up and your automatic transfer switch to cut over to generator power. Trying to power an ever increasing datacenter on batteries is an exercise in futility. You think it's a problem now wait until you have a few HUNDRED servers. Half of your square footage will be batteries, not very cost effective when you consider the cost per square foot to power and cool your datacenter.
But if your entire environment is virtualized, why not let someone else host it? Have you really considered whether or not you want to be in the business of running a datacenter? It can be far cheaper when you consider the total cost even including the transport you'd need to provision to access these systems. You even have the option of keeping a few systems on-site which you could power with your current UPS (DHCP, local domain controller for faster authentication, etc).
Your 2 minute holdup time is more than sufficient to start a Diesel generator and for an automatic transfer switch to fire.
Put a 20kW (or whatever size you need) diesel generator outside and wire it into an ATS. It usually takes just a few seconds for the generator to start and switch the mains.
Hi,
How did you calculate your 5 9's allowed downtime of 1.14 hours?
I may be mistaken but I figure it to be :
365 x 24 x (1 - 0.99999) = 0.0876 hr = 5 min 15.36 sec / year
99.999% availability is really quite good.
I may be mistaken between availabilty & reliability or something?
I live in the UK and most people I know have power like you describe with power cuts being things that happen at intervals measured in years. Most people I know in the UK experiance the same.
However in more rural parts of the UK there is a lot of power carried by groups of three bare 11KV wires on poles and these are rather vulnerable to tree branches, unusually high winds etc. Power problems are common in these areas (mostly it's very short glitches as something briefly touches a power line but such areas get more long outages than urban areas as well)..
Also remember that the downtime caused by a power outage can be considerablly longer than the power outage itself. Without a UPS you have at best a reboot and a disk check and at worst a major corruption issue to deal with after tha power is back on.
note: i'm known as plugwash most places but i screwd up registering that here somehow in the past and now can't register
inverter1.jpg, inverter2.jpg
Banu
Google uses on Mobo 12v batteries.
Of course that would mean switching out all of the current servers.