Slashdot Mirror
Ideas For a Great Control Room?
lewko writes "Our company is about to build a central monitoring facility and I'm looking for ideas/suggestions about the best hardware and the best way to make it comfortable for those manning a screen. It will be manned 24x7 and operators will be monitoring a variety of systems including security, network, fire, video and more. These will be observed via local multi-monitor workstations and a common videowall. This is going to be a massively expensive exercise and we only get one chance to get it right. The facility is in a secure windowless bunker and staff will generally be in there for many hours at a time. So we have to implement design elements which make it a 'happy' place. At the same time, it has to be ergonomically sound. Lastly, we will be showing it to our clients, so without undoing the above objectives, it would be nice if it was 'cool' (yet functional). Whilst Television doesn't transfer to real life always, think 'CTU' from 24."
Here is a blog post with some thoughts about the control room design at the LHC. Here is a picture of the CMS detector control room for comparison. You might also take a look at pictures of the CERN control room for some ideas.
Lastly, we will be showing it to our clients
In other words its going to all be for show.
Big projected screens showing something management finds important to brag about but the employees will never glance at.
All individuality quashed, no pics of the family etc.
My advice, in all honesty, is to build two. One that actually works, and one that is a star trek mock up. Whenever they did marketing picture shows they hired college age models to "staff" our network management center anyway, so non-operational equipment is not exactly a problem for the models to pose with.
"Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
I worked for a couple of months in doing NOC work, you pick up some basic must-haves really quickly in regards to comfort, I'd like to add them. Temperature and air quality is among the very top priorities for sure, here are a few other points.
Get the best and most comfortable chairs you can find and then get the ones that are even better. People will be sitting in these 24/7, you need the best you can get or they'll get worn down in mere months, both chairs and people. Full range of adjustability is essential, as is full back and lumbar support. Look at the seats in taxis, cop cars, any sort of car where people do long shifts, look at long-haul trucks if you can. Taxis where I live are almost uniformly Mercedes-Benz or Volvo, partly because of their excellent seat ergonomics.
Get people out of the chairs. I know this sounds silly considering you've just bought amazing chairs, but give people options. Exercise balls, kneeling chairs etc., let people mix it up so they aren't stuck in an ordinary sitting position for hours on end. Some basic exercise equipment is good too, doesn't have to be anything more fancy than some wall-mounted bars etc., just to let people stretch a bit.
Quick access to toilets and kitchen(ette). This type of work demands as little downtime as possible. Make sure the kitchen has at least two microwave ovens, depending on the number of people who'll be working there, a good meal is essential to getting through a night shift and even the slight inconvenience of having to wait a bit longer can be amplified by being sleepy and in a job where you deal with very stressful situations on a daily basis. Night shifts have been shown to have a correlation with heart disease and possibly cancer. Don't make people wait too for their food as well :-) The place where I worked had lots of people doing 9-5 jobs as well and the cafeteria always kept well-stocked vending machines with sandwiches and the hot dish of the day for the night shifters. It was highly appreciated.
Speaking of the kitchen, get really good coffee. No scratch that, get great coffee and reliable high-capacity coffee makers. These people will suck down black coffee like you wouldn't believe. Keep the fridge(s) well-stocked with ice-cold Coke (or whatever caffeinated soft drink they like) for the non-coffee drinkers.
I cannot stress this enough: Have fresh fruit available at all times. This was an absolutely life saver for me. You haven't had a tasty apple until you've bitten into a fresh, ripe Golden Delicious at 4:30 in the morning after a long nights stressful work. Keeps people from gorging on chips and other unhealthy snack foods as well.
Lastly, let people have their distractions. We used fancy multi-monitor setups where we could put just about any system we monitored on whichever monitor we wanted, very slick. I can't remember what they called it, but it seemed to work pretty well. My point is that the universal setup seemed to be one monitor for desktop stuff, one for TV (most people had 24/7 news on) and the rest for monitoring etc.. The TV feed was brilliant, during the downtime you could catch up on the news and so on, it helped you get through the boring parts of a night shift.
I'm sure I've forgotten a lot of stuff, but these things stood out in my mind as the most important.
Eat the rich.
I had a short stint with a fire department in a fairly large metro area. They handle emergency response for a few million people and have a pretty efficient set up for their war room.
It is fairly roomy so that you can get up and walk around.
desks are set up in pods with groups a people sitting facing towards the center of the hexagons they are on the perimeter of.
They have phone communications with headsets that allow them to move around
there are multiple monitors on each desk and large monitors on every wall that can be switched to show any desktop
The lighting and noise were somewhat subdued, but in no way dimly lit or overly comfortable (no high back chairs)
I do not remember the snack food situation, but there is a lot of security and it would be a pain to have to walk in and out all the time)
fwiw, I would save your company a ton of money and just use IP kvms and a software kvm management solution to tie together a bunch of desktops in a relatively open area together where the operators have some room to walk around, but are not overly distracted or lulled to sleep
ya know, massive operations centers are just soooo glass house IS anyway, totally 80's thinkin
Wherever You Go, There You Are
What hardware you have doesn't matter much. Any run of the mill dual headed desktop will do what I need - a few browser windows, a dozen xterms, and some email up in the corner. Anything else is just for show.
What's really valuable when I'm on point is documentation. I need to know:
What does this server do?
What's the procedure to rebuild it?
Who built it last time, in case I hit trouble?
Who's the business owner?
How do I reach them? A cell number would help.
What's the escalation path when they're AWOL?
Where are the contact numbers for our transit providers?
What are their SLAs?
Where's a map of our network, in case I have to creatively reroute traffic in ways that OSPF won't?
Is it up to date?
What services are exposed in the DMZ?
Where's the ticket requesting this port be opened?
Are there supposed to be 100,000 different IPs connecting here, or just three?
Where is the password vault?
What's the procedure to update the password vault if I have to change one?
Being able to find these things quickly will make me a much happier sysadmin than any creature comforts, excluding caffeine.
If you want to get into creature comforts in a windowless bunker, make it lighting. I don't want it bright, but it should be well designed to cover the space well. Good warm triphosphor fluorescents with high frequency electronic ballasts are much much nicer than the old cold ones with 120Hz flicker from magnetic ballasts. Color rendering index matters. That makes a good base lighting for the workspace. Then get a few of those industrial HID grow lamps, and have them light a big picture of a forest scene covering an entire wall. Or actually grow plants under them, and don't pay too close attention to what else people plant when you're not looking. Careful not to make it too bright, but the sun-like spectrum will break up the monotony of the fluorescents. Add some bright halogen task lights for when you need to see something well, without having to flood the whole room.
Raise the ceiling as high as you can. Rip out the ceiling tiles. Suspend the lights on cables. Let the ductwork show. Paint it all black. I hate living in a box. Exposing all the HVAC and cabling breaks it up, and I actually like the minimalist industrial aesthetic. If you want a softer look, hang some tapestries up amongst the machinery.