How Would You Build a Datacenter?

InOverOurHeads asks: "Some of my coworkers and I are building a new datacenter for our company. We're a growing startup, we have about 50 servers now and expect to have about twice that before too long so building to grow is key. Now that we're about $15,000 in to the project, it is looking and feeling more and more like we were way over our heads. We have 4 racks wired to a single 20amp circuit. Our UPS is at 90% load and we only have 10 machines on it. We have all of our cooling on one side of the server room where it is about 60 degrees, the other side of the room where the servers exhaust is about 30 degrees warmer, so it appears that we have some convection problems with only a handful of the machines on, right now. We're realizing that there is a lot more to building a datacenter than racking servers, what else have we missed?"

"On the positive note, we have a really nice overhead wire rack, that's looking good and all of our wiring is really tight looking; all the colors match, all the cables are labeled, they are all the right length, etc.

Are there any guides or how-tos on this? Since we're going to bite the bullet and tell the boss that we messed up we want to try to correctly measure the rest of the work involved in making it work. What happens when the UPS is at 100% load and how Dell servers react to being under powered?"

Comments

Quickie...

[BrookHarty]

Dont run everything off 1 UPS. You already stated you have 1 UPS at 90% of the load as it is. 1 Point of failure and the money you saved just caused a horrible outage and pissed off customers.

Sounds like you already know this, plan on secondary backup, AC, and make sure you have terminal servers. (Using unix right?)

Jumpin' Jesus on a pogo stick

[bitty]

On the off chance that this isn't a troll ($15 grand for 50 name brand servers, plus racks & UPS? It don't smell good here), did you guys do any research ahead of time, or did you just start slapping shit together? If I were your boss you would be FIRED, because you obviously have no clue as to what you are doing.

You obviously didn't talk to an HVAC engineer, because they would have set you up properly from the start, getting accurate heat output ratings for all the present and planned equipent (3.413 BTU per Watt, they tell me). Then, looking at the placement of the racks, would have had the cold air pumped in at the right places. This may be correctable by tossing a box fan or two in the room to move the air. Not fired yet.

You also didn't consult with an electrician or electrical engineer, because they would have given you some very sound advice on your load and UPS needs. You are so woefully underpowered and under-UPSed, your servers will lose power long before they have a chance to properly shut down. By the way, do you mean to say that you have 50 servers and only 10 are on UPS? FIRED!

"But look on the bright side, Mr. Boss, the cabling is all neat and pretty!" FIRED for spending more time on the cabling than you did planning this disaster!

Do the tech industry a favor, and go get a job shoveling shit at the zoo.

Get professional help!

[digitect]

You need help. (With an architect, I used to do this stuff for the "largest router company in the world".)

From an architectural perspective, don't underestimate the complexity of space planning. Equipment access, emergency egress, and growth of all engineering and supporting systems may put you at a very different place than you might imagine if you consider only your direct server capacity. I'm sure every geek around here would like to think they can solve most engineering type problems with a little extra effort, but building design has more than a few gotchas you don't want to miss.

On the building engineering side, the general trend is for higher and higher densities. Ten years ago, one might have projected that data centers would be getting exponentially larger, but the increasing density of electronic components keeps that growth more reasonable. However, density of equipment has a nasty side effect in that it pushes HVAC, power, UPS, and structural limits far beyond what your average spec office building is designed for. I know from experience that increasing structural floor load capacities from 80psf to 150psf is eyebrow-raising expensive with an operative data center!

Don't make dangerous mistakes. Beyond the expense, embarrasment, and possible job loss, you could create a serious life safety problem for yourselves or those working around you. Obviously four servers isn't exactly a major data center, but if that triples in the middle of a low load floor bay, (or if they're already some mondo racks) you might be closer to floor capacity than you realize. Sounds like you're beyond UPS, power, and HVAC load now--hire an architect with an engineer in tow for a few hours ($400-ish) to advise you. (Or mail me with your geographical location if you need recommendations. ;)