Slashdot Mirror
Power Distribution in a Datacenter?
d00dman asks: "I work for a colocation reseller. We don't own our datacenter, we lease cage space and with it we get an array of 30amp and 15amp power circuits. My problem is, inside of the cage we need to do our own power distribution. We have around 50 hosts in the facility, and are growing quickly. We need to add another 20 or so hosts in coming month. We are looking for a simple, scalable, and even power distribution method. As always, money IS an object. Right now we have a group of UPS's with power strips plugged into each. While this was sufficient for as many as 30 hosts, its starting to become a bit of a mess. Does anyone have any advice to offer...any creative tricks?"
...But have you considered moving to a datacenter which provides conditioned power? After all, stacking UPS' in your rack is not very efficient. Especially when you're running powerstrips off them.
Many datacenters now provide fully conditioned, bus-sized UPS-backed power. Some even let you control individual power outlets from an SSL/password-protected web interface (Which is extremely useful if/when a server hard locks). Even if the datacenter doesn't provide the web-based outlet management, you can pick up a 1u, 8-outlet APC power switch for $300. 5 of them would provide power for the whole rack (Assuming single power supply), and only cost $500.
- James
Call an electrician, preferably the one your colocation provider uses so s/he knows the existing system already.
You could also call a vendor like APC and ask about power distribution to x00 rackmounted hosts, I'm sure they have a solution they will sell/lease you.
Get a 220v APC UPS with a PDU. Run a fewBaytech RPC2s out of the APC PDU.
Doing this myself actually.
Get paid to code OSS
Stop the water works, call the electric company (oh, the pain of the lame joke!)
Seriously, just call up UI or whoever does the electricity in your location. Tell them you need better, and more power. And get a single backup generator, rather than many UPS'. Sell all the UPS and just have the electricians install many power outlets where necessary. You'll have a few minutes of downtime will you unplug/replug, but that's about it.
The GeekNights podcast is going strong. Listen!
What's wrong with the data center's UPS's? By definition, if it's really a data center then it's gotta have UPS's, right?
You do *not* get better reliability by putting two UPS's in series. You do get better reliability by getting rid of unreliable UPS's.
(Simple Solution - APC Power StruXure)
First off, get rid of the stand-alone UPS systems. They are inefficient and not reliable from a system standpoint. You are much better off with a single 30kW unit rather than ten or twenty 1,000VA units. With single UPS, it is easy to either install a local panelboard (US), or a busbar distribution system (UK).
Second, anything that you do to enhance flexibility will compromise reliability. If that is an acceptable risk they go ahead and do something modular.
Given these constraints, a 30kW UPS is 100A or so three phase, 208V or 150A single-phase 208V. 120V would not be appropriate; 480V is a cheaper input feeder, more hassle for bypass.
The best solution might be to go the traditional panelboard route, and run branch circuits to all your equipment in liquidtight flexible non-metalic conduit (the blue stuff).
Alternately, you can install a 100A plug-in busbar the length of your cage and tap off 30A, 208V, 1PH for each rack. They are designed for light industrial applications, but work pretty well if you can sacrifice on reliability.
The APC solution is neat because you don't need an electrician, and it is all modular. Might not be as flexible as the component route, but it works. I think it includes managed outlets in the racks as well.
- Convert incoming power (whatever form) to 42VDC.
- Have a big bank of batteries charged by the 42 VDC.
- Run all your equipment from power cables attached to big copper busbars with 42V between them.
Advantages: you get rid of your UPS's, you have a very scalable power system (telcos never unplug anything!), lots of big manly metal things.Disadvantage: dropping a wrench across the 42V busbars is a bad thing
If you don't have access to a lot of surplus rackmount PC's with 42VDC-in power supplies and are really on a budget, do it with 12VDC instead and use these servers that run off 12V very nicely.
Doesn't seem like a real datacenter if you're having to do your own UPS on top of the power they provide. If they are conditioning the power and you're still having to do UPSs, something is wrong. If their power is conditioned, and you're putting a UPS on top of it, you're wasteful.
Can you let us know more details that you weren't able to ask in the original question?
If you're looking for something affordable as well as easily scalable, it might be easiest to just pickup some 4u rackmount UPS units from APC or BestPower and install one in each rack. There is usually a separate circuit to each rack anyway. For a few hundred bucks, you get enough conditioned power for a full rack of equipment IN EACH RACK, and you also gain the ability to remotely control and monitor power to each device that is plugged into it. If you're going to spend $1-2k per cabinet anyway, what's another $400 for power.
What you want is PDU. Here's how hard it is find one:q =%22power+distribution+unit%22&btnG=Google+Sea rch
http://www.google.com/search?hl=en&ie=ISO-8859-1&
I personally would not buy from APC. I've had more server failures due to APC UPSs shutting off the power output than I would have had with power failures. Our power company is more dependable than APC. Small UPSs are banned here.
Power conditioners and UPSs serve different functions.
A power conditioner insures that the electricity you are using is clean, fairly glitch free, smooth power. Protects you from power spikes or hot power (like sustained voltages over 120v on your 110v line). When there is no power (power outage) you get a nice smooth glitch free zero volts.
A UPS is just a quick switch over from wall power to a battery. When there is no power the UPS uses an inverter to give you 110V AC from batteries, generally lead cell sealed batterys, I am guessing two six volt batteries hooked up in series to give the UPS 12V DC to start with.
Glonoinha the MebiByte Slayer
In the context of a good datacenter conditioned power also implies building wide UPS and multiple backup generator systems.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
Take a look at .
Prevent email address forgery. Publish SPF records for y
A "Double-Conversion" UPS provides power conditioning and battery backup. Alternately, a UPS with internal choke can provide conditioning without full double-conversion. A breadbox-style UPS is usually just a single-conversion device, which works as you describe.
Usually people plug small systems in for the wrong reasons... 'what if this huge building system that is maintained regularly "breaks?" Good think I have this little box here that I've been using for the past five years without any problems..!' or 'I want to be able to move the server around the room without unplugging it' nonsense.
There are very real problems associated with plugging the little shits into a large system; the cheap inverters have trouble tracking and can create all kinds of problems.
Also, always remember that the sealed batteries only last 3-5 years if you are lucky!
between YOUR customer, and the datacenter owners (YOU are a customer of theirs). IF you need more circuits, they should be able to swing a tail from a PDU over (under) your cage to meet demand. If you need 10 tails, then tell them so. They charge you, you pass the charges on to YOUR customers. You should not be mucking aroung with their power. Also, the datacenter should be providing UPS and backup generator support to you. Don't plug your smallish UPS's into a circuit that is aldready UPS protected.
If you don't get any help from the datacenter, call an electrical company that specializes in datacenter power (for example, Computer Air and Power (CAPS) in the Bay Area).
Many of the folks here are used to thinking small. I am not. Have you considered a pair of Pillar data center UPSs, turbine generators, and local fuel storage?
It's really amazing what you can get for $500k these days.
The solution we are planning on using is some neat power racewys from Universal Electric Corp. Here's the link: http://www.uecorp.com/starline/starline.htm
These are pretty slick--the tracks are fixed in place, but the outlets are completely mobile. Think track lighting, but for power outlets. The modules come in all the various plug types ranging from standard 110v 15A plugs to 208v plugs to bigger modules with fuseboxes and circuit breakers (the list of plugs is pretty extensive).
The major benefit for this raceway is that once the track is installed, the plugs can be relocated without shutting down power, or requiring an electrician to come in and do it. You can also buy modules as needed, instead of having to guess at options for future expansion. In our case, we are going to get several 110v/20A modules and a few 208v outlets, place them directly over the racks as needed, and have power strips internal to the rack plug into those.