Planning a Small Server Room

An anonymous reader writes: "Our company is planning to build a small server room. Initial requirements are for two or three enclosed server cabinets in which various servers and network gear will be installed. The cabinets are planned to hold between 15 to 20 servers of various types and sizes, switches, routers, four dial-in modems for after hours use by staff who do not have ISPs and a KVM switch. We would expect for a small desk as a work area, a book case, storage for some spare parts as well as server documentation and records. We know that we need some power protection in the way of a UPS and a generator. We also expect that this room will get quite warm in the summer months so it will need more air conditioning than the rest of the office. What should we expect for power and cooling needs? Are there any 'rules of thumb' when it comes to building a server room. Good suggestions and help would be appreciated."

Comments

Link!

[NWT]

We've had this kind of question once, but it was for a home server room, this shouldn't be too different!
Link

Try to use common sense.

[duffbeer703]

If you are in an old building, make sure that the floors can safely support the weight of alot of computers.

When you initally layout the room, pack everything as tightly as possible. You'll be happy you made that decision 5 years from now.

Be careful with roof-mounted air conditioners. They have a habit of spewing ice cold water all over the place when they have a problem.

Offset the racks far enough away from the wall so that you have enough room to work. Make sure that some dope doesn't push them back on you.

Remember to install a phone

[geirt]

The one thing I am missing in our server room is a plain phone....

Get a subfloor

[infernalC]

You should set this up with a removeable tile subfloor so that you can run cables between racks in a pretty fashion. Power is gonna be a big concern. Get cable trays - separate ones for power and data - and mount them under the subfloor. Get yourself plenty of wire ties, too.

Most of the stuff you'll want can be gotten cheaply at Anixter.

Fire Supression

[thefatz]

Nobody has mentioned anything about Fire Supression except one guy who said sprinklers. Hehe NOPE. Consider getting a Halon dump system or some other chemical fire supression system that will not damage your equipment.

I have seen when a diskpack caught fire from a crashed disk. When they opened the door to the disk pack a sudden backdraft type explosion occured. The Halon released just seconds later putting out the fire, all while the other servers, printers, and mainframe continued to work. Sprinklers would of been a $20 million dollar mistake.

Fatz

Our one mistake...

[arrow]

We currently have 2 server rooms, one here in MIS, the other in another building. The second room can't be locked(!) due to pointy hair policy and sticking useless crap in there. At some point a user slipped a line printer in there to cut down on noise in their office and the room has been left unlocked ever since.

My recommendations would be:
1. NEVER EVER EVER EVER let lusers into your server room. Put decapitated heads on bamboo sticks all around your server room. I almost killed someone when I came in one morning, and realized someone had manualy ctrl+alt+del'ed our timeclock server because their PC couldn't access it and they assumed it was a server problem.

2. Replace the door handle on the door with a deadbolt. Nothing says go away more than no handle, and its fairly easy to just turn the key and push.

3. Use racks. If your room is already going to be temp controled, and its locked up tight, cabinets aren't needed. Plus if venting fans on one of your cabniets dies, it turns it into a big thick metal blanket for your servers.

Re:Cooling, Power, Cooling, Power

[sigwinch]

A good electrician will be able to hook up a meter to a few sample servers and get the exact amount of juice they pull. Use the GREATER of that number and the name plate rating on the computer.

Ignore nameplate ratings on big devices like computers and monitors. They're usually overrated. Ignore measurements. Speaking as an electrical engineer who designs computer peripherals, getting true worst-case measurements is very, very difficult. You have to exercise the hard drive heads, CPU cores, RAM busses, and I/O busses fully, and that's near impossible. Switching power supplies also draw more current as the voltage goes down. If you make the measurement when the line voltage is 130V, the equipment will draw 20% more current at 110V.

For little things like KVMs, modems, inkjet printers, etc. you can safely use the nameplate ratings.

For big things, determine how many machines you would ever conceivably want in the room. Choose the biggest, baddest equipment you could possibly want. 1U dual-proccesor machines, arrays of 15000 rpm hard drives, a desk full of 21 inch monitors, you name it. Then go to the manufacturers web sites and find the nameplate ratings for the various things, and add 'em all up. The total will be a number you won't easily outgrow.

Be sure to account for start-up loads. You don't want to trip a breaker by turning everything on at once. Hard drives draw a lot of power while they're spinning up, monitors while degaussing, laser printers while warming up the fusion rollers. This is just an educated guess, but use a factor of 2 for hard drives, and 5 for monitors. Read the specs for the laser printers very, very carefully and find the worst-case.

Also most wiring in commercial spaces is done in conduit and more than 3 wires in a conduit requires that the wires be derated and not all electricians pay attention to that (again per the NEC).

I'd go even farther. When many surge protectors divert a surge, they divert it into the ground wire. This causes a brief, high voltage spike on that circuit's ground relative to the other circuits in the room. The longer the ground wire is, the larger the spike. This spike can do nasty things to serial lines, KVM cables, and so forth that connect machines on different circuits.

So if the building breaker box is farther than, say, 50 feet from the server room, I'd have a small breaker box installed in the server room. Also this lets you recover from a tripper breaker without getting the main breaker box unlocked.

If you can afford it, have a couple of separate circuits run from the main breaker box. This gives you someplace to plug in coffee pots and vacuum cleaners without disturbing the electronics.

If the room gets its own air conditioner, make sure that has a dedicated circuit from the main breaker box.

If you can afford it, have a big industrial surge protector installed at this breaker box. Also the breaker box is a good grounding point for surge protectors on your external data lines.

The net effect is that you should plan on the electrician using #10 THHN in any conduits.

This is excellent advice. The electrical code is based on safe operation of motors and heaters. Bigger wires make your electronics more reliable by reducing voltage droop.

Also, computers often don't draw sine wave current. They draw less current at the beginning and end of the AC cycle, and more in the middle. This means the peak current is larger than the sinewave loads envisioned by the electrical codes.

Computers often need good grounding systems, so I would also require a separate ground wire to be run in the conduits even though it is usually not done since the conduit can act as a ground.

More excellent advice. Conduit is completely unacceptable for grounding computers. A grounding wire is cheaper than the cost of a single computer crash caused by a poor ground.

You will also want to make sure the racks are grounded and you may even wish to consider putting a wire mesh beneath the floor and grounding that.

Have an electrician tie all the racks and other metal stuff together with big ground wires. This will help protect the rest of your equipment if one of the devices has a ground fault. It'll also help reduce static electricity by giving you lots of big grounded metal things to touch. Wire is cheap compared to the cost of a single failure.

Finally, if possible, require that the communications cables be run in over sized conduit as well. It makes expansion much easier in the future and also provides a measure of RF shielding.

Conduit does make running wires much easier. If there is no other wiring or fluorescent lights within a few feet, I'd use nonmetallic conduit, as metallic conduit can actually act as an antenna for picking up radio waves and coupling them into your data cables. OTOH if there are AC lines parallel to the run, metallic conduit is probably better, and be sure to make the electrician ground the conduit properly.

three things are important in this case

[booyah]

Cable management cable management cable management!!!

well really more than just those, those are just my big pet peeves.

Good things to have include

a work bench

a tool cart

a phone

a seperate test subnet (firewalled from the real net)

a good lock

cooling

UPS

generator

all internal walls

static floor panals

and make sure there is room to work today and a few years down the line...

-Booyah

Audit trails and cameras

[Nonesuch]

If you are protecting valuable hardware and/or data, consider requiring keycard+PIN for physical access to the server room.

Make sure you control all access, including the potential for intrusion from above and below -- dropped ceilings and raised floors often make an easy path for a skinny crook to get from a public area to a controlled location.

For around $1K in equipment you can set up four cameras, a quad combiner, and a time-lapse VCR system to provide a video record of everybody entering and leaving the room.

We've examined many different options to handle the camera monitoring and recording with a digital system, but there is no PC solution that comes close to the good old $200 surveillance VCR. Plus, videotape is going to be more acceptable when you need to involve law enforcement.

One last note -- make sure the VCR itself is in a seperate controlled access location. Not much point in a videotape record when the thief can simply eject your tape and walk off with the evidence.