Slashdot Mirror
Planning a Small Server Room
An anonymous reader writes: "Our company is planning to build a small server room. Initial requirements are for two or three enclosed server cabinets in which various servers and network gear will be installed. The cabinets are planned to hold between 15 to 20 servers of various types and sizes, switches, routers, four dial-in modems for after hours use by staff who do not have ISPs and a KVM switch. We would expect for a small desk as a work area, a book case, storage for some spare parts as well as server documentation and records. We know that we need some power protection in the way of a UPS and a generator. We also expect that this room will get quite warm in the summer months so it will need more air conditioning than the rest of the office. What should we expect for power and cooling needs? Are there any 'rules of thumb' when it comes to building a server room. Good suggestions and help would be appreciated."
You've already listed some good rules of thumb, the Air Conditioning, shelving space, etc. I can't express how important it is to have good organization. Organize your unused cables too, otherwise one day you'll end up with a 200 pound rat's nest of cables you're trying to pick through to find a spare UPS Serial cable, and it'll take you half the day to un-knot it. Keep your servers and network equipment well labeled too, this way you don't have to describe to a new employee which server to power cycle on the phone.
Locks, you'll want to have good locks on this room. Maybe a camera in it too, Security is always important. Not only security, but preventing some uneducated employee from accidently wondering into the room and pressing buttons. It happens I've seen it. I've also seen employees wonder in and realize their monitor isn't as good as the one you have in the server room, and switch them.
Keep it clean - I can't stress this enough either. Server rooms are a breeding ground for dust. Keep it well filtered with air filters, de-humidifiers to keep the moister down, and try to limit what kind of cardboard products are in the room.
I'm not a good expert on Power and Cooling, but I think one rule of thumb is as much as you can get it. And Redundancy, cooling included. Multiple Air Conditioners, and Multiple power backups. I've been in many places where Air Conditioners go out in server rooms and those things jump to 100 degees in just a few hours.
That's about all the advice I can offer, good luck.
..There's a-dooin's a-transpirin'
A good electrician will be able to hook up a meter to a few sample servers and get the exact amount of juice they pull. Use the GREATER of that number and the name plate rating on the computer. I would plan on having UPS's that can take 125% of your calculated load. Also the UPSs should be considered continuous loads so the circuits that feed them need to be rated to 125% of the actual load (per the NEC). Also most wiring in commercial spaces is done in conduit and more than 3 wires in a conduit requires that the wires be derated and not all electricians pay attention to that (again per the NEC). The net effect is that you should plan on the electrician using #10 THHN in any conduits. Computers often need good grounding systems, so I would also require a separate ground wire to be run in the conduits even though it is usually not done since the conduit can act as a ground. You will also want to make sure the racks are grounded and you may even wish to consider putting a wire mesh beneath the floor and grounding that.
Finally, if possible, require that the communications cables be run in over sized conduit as well. It makes expansion much easier in the future and also provides a measure of RF shielding.
Brings back memories of a previous job I held. Our server room (about 40x65 feet) was a glass-enclosed room with a raised floor for cabling and ventilation. The AC unit was the size of four industrial refridgerators side by side. The UPS was a cabinet slightly larger than the AC unit and held dozens of batteries in series which could keep the equipment in the room running for 30 minutes -- more than enough time for the outside generator to kick in. Each battery was roughly what you would find in a large truck.
The room housed the servers for our local network and for the WAN which consisted of roughly two dozen buildings scattered around the county. We had a mix of HP/UX, Linux and NT servers -- and even one MPE/iX box (an HP3000 server). We also had our dial in, frame relay, outside Internet connection and terminal servers in the room. I believe there were 6 rackmount cabinets for most of the servers and the network equipment (the HP3000 and our voicemail systems were their own fridge-sized units outside the cabinets).
It was actually separated in to two parts, as well. The main room, which housed the actual servers, was about 40x50 feet. The second part was separated by a glass wall and was 40x15 feet. The smaller area had desks and a couple enclosed rooms where the support staff would usually work. Hardware work was done inside the main server room because of the air control.
The main things done right with the room were:
- AC Unit: this thing kept the room at a nice 54 degrees Fahrenheit no matter what was going on outside. The AC in the rest of the building would go out and everyone would start opening windows and turning on their desk fans, while I would retreat to the server room and put on my fleece.
- The raised floor: We never had a single cable on the floor to trip someone, and we could put a power outlet anywhere in the room we wanted. The floor was about a foot and a half off the real floor and covered the entire room. I loved that raised floor.
- Security: Sure, someone could break the glass walls (although the building's security system included glass break detectors in the server room), but the doors were very heavy and very thick. Access was controlled by individual keycodes which we had to change regularly. Out of the 50 plus people working in the same area of the building the server room was located, three of us had passcodes to the server room. So we always knew when someone was in there because one of us would have to escort them in and out of the room.
- Shelving: We had tons of shelving. We devoted one side of the room to just aisles of shelves, all clearly marked with their contents. The actual types of items were kept in alphabetized order. So, we had our boxes of cables near the first aisle, memory was near the middle and "Wyse" terminals near the end (a brand of basic vt102 dumb serial terminals).
- Deskspace: My desk actually was located in our server room, though I was usually on call in another building. But we also had an "island" in the middle of the room for general use. It was large enough to have four people simultaneously working on hardware with all their components spread out around them. We also had a couple workstations on the island that could be used to log in to the various servers and other equipment. These were convenient because they could remain logged in with privileged access to certain servers and we didn't have to worry about someone using them when we went to chat with mother nature since access to the room was stricly controlled.
The only complaint I ever had about the room was that when we would get shipments of 100 new workstations, they would cramp the room up a little until we got them all set up and shipped out to the various other buildings.
The suggestions I would make for things to consider when setting up a new server room:
- AC (obviously) and UPS (obviously)
- Raised floor (you can get by without one, but when you have one you never want to get rid of it)
- Entranceway security and if possible video monitoring
- Strict, enforceable access policy (there's no need for the the new graphics temp to be wandering around in the server room, but sometimes you'll want to be able to escort the VP through the room so he/she can see all the pretty blinking lights)
- 1.5 times the rackspace for your initial machine count at minimum, with twice the space initially needed reserved for cabinets
- Tons and tons of shelving, plastic ties, rubber bands, electrical tape and sticky labels. You never have enough of any of these things. Get plenty of bins of various sizes, too, to use on the shelves for things like screws, jumpers, adapters, etc.
- It's really helpful to have a common area for all the tools. We actually didn't do this at first and we'd lose a crimper or a screwdriver or something once or twice a week (more often than not we'd find them under the raised floor).
- If you find you're running a lot of cables in the to ceiling to distribute to the rest of the building, get some regular PVC plumbing pipes and a hacksaw to create basic conduits in to the ceiling and then above the ceiling to outside the walls of the server room. One of the easiest ways to feed cables through these is to get a string and tie it in to a loop where it will run one length inside the PVC pipe and another length outside. Create a few loopholes in the string and then whenever you want to feed a cable through it, hook the cable's connector in to a loop and then pull the string.
Get some drains mounted under it. When the sprinklers on the floor above go off (they will), or the roof leaks, it'll go under the floor, and drain away. Don't put it in a basement, and if you are in an area prone to earthquakes, hurricanes, or tornadoes, put the server room lower in the building. If the area is prone to flooding, move it up a floor or two. If in doubt, mount a moisture alarm in a low spot in the room. (They sell them wherever they sell sump pumps.)
Under the floor is really where racks are meant to have their cables run. Some flooring units have inserts that act as vents, and that works nicely. Your under-floor is ventilated, kept dry, and your smoke sensors have a higher chance of sniffing the smoke if the air moves through that closed area. There are actually some commercial smoke alarms that continually pump and sniff air, rather than the passive ones we have in our homes that rely upon convection and diffusion for the smoke to reach a sensor. Put some sort of dust-handling equipment on your air-conditioning. The folks that sell you the AC will be able to help. See if they can tie the ventilation into the smoke alarm so that if there is an alarm, fresh oxygen stops getting pumped to that room. (They do this on some modern highrises.) The folks that sell you the AC should also be able to help you with sizing the air conditioning to your requirements.
Call in your local pest control expert to mouse-proof the building, then make sure there's serious screening over all entries into the building. Mice get bored, and for fun, they pick their teeth with the fiber core of the cable running to your most mission-critical server. However, they have to chew through several cables before they find the right one.
Consider one of those big panic buttons that shut the room down in a hurry. Just make sure under a cover so that someone doesn't accidentally punch it. At the very least, place the circuit breaker panel in the room, and clearly exposed (meaning don't stack crap in front of it), so that someone can get to it and flip things on and off.
Also place several of the correct class fire extinguishers there. Place a wall-mounted first-aid kit (some cases have sharp burrs that cut fingers well) near the door. Doesn't have to be fancy. Could just be something on a shelf. Also have a paper-towel dispenser (or just a roll of Bounty) for when someone forgets, takes their drink in, and then knocks it over.
Finally, plan for expansion. Make the room a bit bigger than you think, but leave one wall that can be bumped out to claim the room next to it sometime in the future. (However, I think server sizes have stopped growing, so the need for more physical space is lessening.)
You will want TWO analog POTS phone lines, dedicated to the room. They should bypass your company PABX or VoIP system. They should be ordered as business grade lines, so you can get better service from the telco if they have problems.
:-)
These phone lines will save your career sometime when the power is flaky, or your PABX has gone down, or you have to call two different hell desk lines at the same time (finger pointing? Who? Not me!)
Since you will have some dial-in modems, ensure one of your telephones is a simple, plain, ordinary telephone, which doesn't require electricity to operate. For the other, follow the other suggestions in this sub-thread; i.e. cordless, handsfree speakerphone, etc.
And a selection of RJ-11 (not RJ-45) cords, long enough to reach from corner to corner of your machine room. And a couple of banjo breakout connectors.
And depending on the theft/wandering kit factor in your place, florescent spraypaint to mark your easily lost phones
the AC
I'm back!
Hemos is like...sci-fi fans;he thinks technology is cool, but he hasn't bothered to understand the science it's based on