Slashdot Mirror
Ideas For a Great Control Room?
lewko writes "Our company is about to build a central monitoring facility and I'm looking for ideas/suggestions about the best hardware and the best way to make it comfortable for those manning a screen. It will be manned 24x7 and operators will be monitoring a variety of systems including security, network, fire, video and more. These will be observed via local multi-monitor workstations and a common videowall. This is going to be a massively expensive exercise and we only get one chance to get it right. The facility is in a secure windowless bunker and staff will generally be in there for many hours at a time. So we have to implement design elements which make it a 'happy' place. At the same time, it has to be ergonomically sound. Lastly, we will be showing it to our clients, so without undoing the above objectives, it would be nice if it was 'cool' (yet functional). Whilst Television doesn't transfer to real life always, think 'CTU' from 24."
You need one of those old cabinets with tape reels ticking around. Adds to the atmosphere and will remind clients of Thunderbirds etc,
Some fake windows, even just glass blocks with lights behind them, will do wonders. Also, make it so that people have to get up from chairs once in a while.
Like the bridge of the USS Entreprise?
Whatever you do, don't make it dark with blue-ish lighting, like on TV. That strains your eyes. Provide good lighting, and make sure the persons can sit or stand comfortably while watching the screens.
http://ccs-pk.chace-school.net/files-2008/at-and-t-noc-pic.jpg
Plenty of ideas here:
http://www.villainsource.com/lairs.html
One word: Kegerator
These are print subscriptions, not website subscriptions.
A big red button with a sign above it that says 'DO NOT PRESS'.
'We are trying to prove ourselves wrong as quickly as possible, because only in that way can we find progress.' RPF
Well, it sort of goes against 'being in a bunker', but if I was going to work somewhere for many hours, I'd like some natural light.
Of course it's still possible to achieve that using reflective tubing or such like, though it might still undo whatever it is you seek to achieve by being underground.
If it's not possible, I'd suggest paying lots of attention to lighting. And add some real plants too - they'll generate oxygen as well as making the environment seem less bunker like.
And you came to Ask Slashdot? ABORT! ABORT! TOO LATE!!!! *boom*
Include a water cooler somewhere. If it's a secure facility, you don't want staff popping out every five minutes for water.
Dim lighting looks incredibly cool. It's also uncomfortable. Resist temptation.
Borrow some ideas from the utility control rooms I've been in. Everyone has and uses their own headsets, I might extend that to keyboards. Keeps people from passing the contagious thing of the week around a confined space when sharing monitoring stations.
That's our life, the big wheel of shit. - The Fat Man, Blue Tango Salvage
Here is a blog post with some thoughts about the control room design at the LHC. Here is a picture of the CMS detector control room for comparison. You might also take a look at pictures of the CERN control room for some ideas.
I sincerely hope your "client" is not the US Government. The number of contractors I have seen build "CSI" control rooms to try to impress their government counterparts is incredible. Typically these control rooms control very little, or at least, very little worthwhile. At any rate, I would give the advice: form follows function.
Don't forget the machine that goes 'Ping!'.
it's that it is essential to have the ductwork accessible by anybody who wants to crawl in them. You can't leave that out, it's essential.
Make sure to tell your HVAC guys that you want t
I've just spent a week in an Equinix facility which on first impression scored high on the wow factor. They used a lot of dark colors lit with red and blue; looks awesome but gave me horrific migraines which none of my usual drugs could help with. Very tiring after long periods and very disorientating, it was playing havoc with my internal clock.
If you need to ask slashdot you're in over your head
If an experiment works, something has gone wrong.
1. lots of game servers with 3d cards on all desktops
2. lots of reading material and dimmable multizone lighting with color temp controls in room and automatic color temp selection from warm to cold white following day/night cycles.
3. tv + blu ray players
4. large wallscreen with cool maps for impressing clients
5. 2 machines on each desk - one for entertainment, the other for continuous display of net stats and ssh
6. central server with helpdesk ticket display and network map with the ability to switch it to the wallscreen
7. multiple zone climate control with controls on the wall easily accessible
8. fridge with healthy snacks + microwave
9. water dispenser
10. cleaning staff for once/day cleaning.
Numerous commentors here point out the error of using cool colored dim lighting for a facility where actual work gets done. But you can have your cake (or Cake, if you are using cool indie rock background music) and eat it too - just have a button that switches the lighting to "cool mode" whenever a visitor comes in. Meaningless but cool looking graphic "screen savers" could also pop up on the screens.
Starships were meant to fly, Hands up and touch the sky - Nicky Minaj
Do ensure that it has nearby and adequate washroom facilities. Nothing worse than having to travel up a couple flights of stairs every time personnel has to take care of nature, know what I mean?
Adequate lighting, and ventilation / heating / air conditioning is also something to consider. Nothing worse then working in winter with cold fingers, let me tell you.
I've been NOC for just shy of 3 years now, and I can tell you the environment you work in plays a huge role in how comfortably you handle the workflow. Its nice to focus on the more technical bits such as equipment and infrastructure, monitors, etc, but do not forget that people have to comfortably be there for hours at a time. We do 12 hour shifts here, and the most important consideration would be the temperature and air quality, imho.
user@host$ diff
Comfort, comfort, comfort...
Proper chairs. Set up at the proper height, then adjust all the screens from there.
Good lights. Stay away from fluorescent, anything that flickers. Don't make it to harsh, whatever it is. Nice and soft.
Plants would be good. Things that make the experience not painful after the 6th hour.
I would contract with a design/build firm that specializes in these facilities. You don't want to do this by the seat of your pants relying on Slashdot advice.
My company invested millions of dollars into a central monitoring facility, with a large video wall driven by Crestron equipment. The idea was the video wall could display news/weather alongside alarms/outages in real time, with geographic mapping capabilities. Workstations were quad displays on adjustable motorized desks which sat atop a raised platform for simple network runs. A large executive "war room" style conference room was built with a glass wall overlooking the platform and video wall believed to be useful in the event of some catastrophic failure. All other staff sat in cubicles surrounding the platform with glass cube walls anywhere that would otherwise obstruct the view of the platform/video wall. A secure mantrap was put in place to restrict access to the facility. Dedicated bathrooms were installed with showers in the monitoring area in case critical staff were quarantined for extended periods of time.
It was impressive when it was built, but within a couple years, the video wall has been dismantled and parts sold off due to its impracticality. The right software was never found to perform the type of "geographic" monitoring conceived, partly due to bureaucracy. Network redundancy was overlooked, which made the monitoring facility itself non-functional during an outage. The facility lacked appropriate backup generators and UPS to keep the facility running during a thunderstorm. The platform desks required too much real estate and allowed no room for growth, so they have been replaced by cubicles. The secure mantrap was an inconvenience for upper management, so the inner door was disabled, defeating the mantrap. The quad displays ironically obstructed the view of the video wall when it was still in place, and did not fit in the cubicles when they were installed, so these were reduced to 2. All critical staff were sent home to telecommute because they took up too much real-estate required for day-to-day operations, and it made more sense to not have critical staff in a single central location anyway.
The point is, don't get too caught up in building 'CTU' from 24. The right monitoring software platform makes all the difference, as does intelligent network redundancy, telephony and backup power.
I'm not an expert, but I play one on slashdot.
Medal Of Honor to give your control room zombies a break from the tedium of
monitoring the proletariat.
Yours In Minsk,
K. Trout
See http://www.tbcconsoles.com for a few ideas.
Chairs - Haven't found anything to beat the Herman Miller Aeron yet.
If you're placing PC's in the control room, beware of heating and cooling requirements. Most common mistake I've seen.
Second most common mistake, way over-spec'ing the AV system for the videowall. Keep it modest, in keeping with your actual requirements and conop, with room to grow.
I like the single tier monitor approach. Get monitors above other monitors makes for an ergonomically uncomfortable experience.
Do not, under any circumstances, cheap out on the monitors. Buy decent. Voice of experience here, cheap has the potential to kill your project dead
Completely unbelievable troll. I don't own a dog.
"linux is just DOS with a UNIX like syntax" -- Galactic Dominator (944134)
Lastly, we will be showing it to our clients
In other words its going to all be for show.
Big projected screens showing something management finds important to brag about but the employees will never glance at.
All individuality quashed, no pics of the family etc.
My advice, in all honesty, is to build two. One that actually works, and one that is a star trek mock up. Whenever they did marketing picture shows they hired college age models to "staff" our network management center anyway, so non-operational equipment is not exactly a problem for the models to pose with.
"Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
the fuck?
windowless bunker or happy place. choose one.
Seriously the most obivous thing that stroke me in that question was that you need to get it right. Why ? Try to use items that can be relocated or used in different ways without spending a fortune. After 1 year of operation they will know what they want different.
And of course make it a nice place to be in.
I used to know someone who had worked for an alarm monitoring company. She said the chairs and other furniture were used 24 hours a day, yet the accountants were depreciating them like ordinary office furniture. As a result, the furniture was not replaced often enough and was falling apart and uncomfortable. Make sure to plan on fast depreciation for your furniture.
steveha
lf(1): it's like ls(1) but sorts filenames by extension, tersely
"secure windowless bunker"
"many hours at a time".
"'happy' place"
"ergonomically sound"
"would be nice if it was 'cool'"
Dont do drugs folks.
No matter what you do, the people working there are going to be spending at least a little time looking at things that aren't work related, especially on breaks. Make sure there's a "boss key" that flips things back to something that at least looks work related, so they can look busy when visitors come in. Yes, that includes their own managers, but so what? Presumably there's a way to monitor what they're doing, and if they need the boss key when they're not on break, they'll get busted sooner or later, but it's better if you don't have to explain to J. Random guest that it's OK for them to be checking out Slashdot because they're off-duty at the moment.
Good, inexpensive web hosting
So, a couple things to think about:
1. How much money do you really have? Lots of people think they have money, but run out when it comes to all the details.
2. Do you want flash, or functionality? The two are sometimes complementary, but often one trumps the other.
3. How many people will staff? What's the schedule? This helps you figure out workstation configurations.
4. Are you putting multiple tiers in the same room? This is "best practice" if you do it right.
5. Are you handling customer calls directly? Do you deal with customers?
Basically, you need to figure out a lot of goals. A true "global" NOC can cost $50M easily for a telecom or comparable organization.
I've been a big fan of Barco for large projectors, and their IP-based solution is quite powerful. Recently, I rolled out a "public safety" SOC (security operations center) with 8 SXGA+ rear-projection displays. The largest I've worked on was 40+ of that style of display. Your garden-variety projector isn't cut out to handle this kind of duty-cycle. They're not cheap, but they're designed to operate 24x7x365, and many models have multiple lamps, etc. so that you can service them while they're online. So here's a few more things to think about:
* What goes on the "big screen" has to be useful. It must be grokable in a very short period of time. If you can't look at it for 2 seconds and get a good idea of what's going on, it's too complicated.
* Multiple displays per operations person
* Operational "graphs" that show overall statistics that matter to the people working, not to management.
* Good task lighting. Good lighting period is everything. Pay a real designer to do this.
* Good seating. We have let operations people pick chairs that fit their needs. Expect to spend $800-1k/person on seating.
* Sound deadening/management. NOCs get loud, and managing the acoustics is important to make sure that people can "think" and they can interact with one another.
* Ticketing is everything. Look at systems that are available commercially and for free. Consider writing your own if needed. If the system is streamlined to your own business, it will always be an impediment to getting the job done, which means people won't use it. If they don't use it, lots of knowledge is lost and post-mortems are more difficult.
Also, a few things that seem superfluous, but ended up being critical in some places I've worked (not all these were at the same place):
* Virtualized desktops (think RDP, X11, etc.) so that people can move and maintain their setup :)
* Color-shifting lighting to compensate for normal rhythms of people on weird shifts. Turns out green is effective after lunch at helping people maintain focus. This isn't cheap, but it sure does have a big impact.
* Keep your customers OUT OF THE NOC. A glass wall into the NOC is fine, but actually letting them in is distracting, and depending, can come with legal issues around privacy, HIPAA, etc. Best to keep them at a distance.
* Before you let customers see the NOC, you warn people. We had a blinking lighting strip under the displays that was linked into the Crestron system so that you couldn't flip the LCD-glass for 10 seconds to give NOC operators a warning. You don't want customers seeing people picking their nose.
Finally, as nice as good facilities are, if you don't have the process and people, it's useless. People people people people. Good people create good processes. Promote from within, and develop a strategy to give people a career path. Otherwise, you'll burn people out, and get huge turnover. That sucks for everyone.
But my first somewhat obvious thought would be to build a very detailed list of what has to happen in the room, then use that to drive your design.
Positioning of people I would also imagine is quite important.. which groups of users need to communicate with each other.. who will be using the video wall.. who is going to be making the most noise (is someone going to be on the phone every 10 minutes.. if so a separate sound proof cage might be in order)
Things like white boards might also be a good idea. For all the high tech collaboration solutions out there, I've found nothing beats a whiteboard for figuring something out or just tracking status of a short term issue.
I'd also watch the cool factor stuff. A lot of the stuff that looks really neat on TV actually sucks in real life. Moody blue lighting for instance is depressing and hard on the eyes. Maybe you could have some kind of "holywood mode" switch or something for when people are being toured through.. though that is a little extreme.
Finally I'd say good quality monitors and the most comfortable chairs that the budget allows.
In such a situation the greatest hazard will probably be from your own company executives. Establish a great monitoring process and it is carved in granite that some executive will add tasks that take attention away from the primary task. Reports leap to mind.
The second hazard will be from employees that man the monitoring station. They can become disgruntled or even be paid to do foul deeds. Good encrypted backups kept off site may help as will a monitor that watches the people that man the station.
It may help if you disallow electronic gizmos of all types from being brought to work in that monitoring station. Also tools that could open a computer case and install a USB card make it clear that you need to have absolute control of all items brought into the room.
Beyond those factors have you considered Faraday shielding?
I'm serious. If people are going to be there for long periods of time, they should get a little exercise.
I'm not talking about running, just a leisurely ~1mph walk.
Standing and walking are probably the absolute best options for maintaining good ergonomics.
The office furniture company Steelcase makes one.
I'm not saying to ban chairs, keep them as backup and for people who physically can't stand for extended periods of time.
When information is power, privacy is freedom.
From my experience from working in the control room of a nuclear plant there are a couple of design elements that make life easier. First, any controls and monitors should be separated by an aisle where you have desks and computers. Additionally, if you have operators and supervisors in the same room, a tiered design is helpful. Think of it sort of like a semi-circle, with the controls and monitors at the lowest level, the operators desks and workstations at a higher level, and the supervisors in the back. This allows people to come into the control room to talk to the supervisors without bothering the operators, while the supervisors can still see what is going on. If a casualty happens then the operators will descend to the bottom aisle by the controls and the supervisors will descend to watch the operators and read the procedures. Management can then come in and observe from the level the supervisors previously were.
Second, it is useful to have mimic boards for complicated systems. This means you have diagrams connecting electrical busses and breakers or pumps and piping on the control panel itself.
Third, alarms need to be designed so that they can be quickly silenced, yet allow other alarms to come in. Preferably this should be over the controls where everyone can see.
Fourth, look for items where people can make mistakes. Color coded labels on your controls, monitors, etc., as well as logically positioning them can prevent mistakes. Don't just ask for suggestions. Do dry run evolutions and see where design changes can help. Build a mockup and run it with a crew.
Finally, don't screw up the HVAC design or soundproofing design. It will piss off your operators.
Exercise equipment. Personal workspaces for each 24/7 staffer. A schedule that is not completely idiotic. A decent break room. A big red button with a sign that says "NEVER PUSH THIS" but doesn't actually do anything. The red button that actually does something should be several layers deep in locked cabinets. A box within a cabinet within a safe should work. Seriously, the first few things I mentioned should be considered. I speak from experience.
FAQs are evil.
my list... my opinions...
1. Don't make it all dark(overstated and redundant here).
2. Give your people some sort of privacy if you don't want high turnover. This could be a simple privacy screen.
3. If people have to talk on a phone to multiple customers at once, make sure that you have a strategy to keep the sound from bouncing everywhere.
4. Nice chairs... really nice
5. Watch out for cloth chairs(they absorb spills)
6. Keyboards, mice, etc become gross. I honestly would assign people these items.
I know why companies like to put big impressive screens mounted to the wall or ceilings, its for show, makes the room look kinda like mission control at nasa or something. But heres the difference. The big screens up top are for information that is not really something you need to stare at constantly. You can glance up, get what you need and put your head down and continue work. Please do your employees a favor and put their respective important screens they will be spending most of their time looking at, down at sitting eye level. It greatly reduces neck strain and headaches. A high back chair is worth its weight in gold, these crappy rolling chairs with about a square foot of back support just do not cut it. After 8 hours of my back being halfway supported, I will hate that chair, and go find a better one. This usually leads to other unhappy co-workers asking "where did you get that from?" and eventually we will find your comfy chair, and during the graveyard shift, yours might go missing. Save yourself the trouble, just go ahead and get the chairs that fully support back and shoulders. Its worth the extra money. (get a few extra too, they tend to break) Carpet is great for making a room a bit quieter Matte screens reduce glare and eye strain a bit The white buzzing fluorescent lights overhead suck, but you can hide them a bit behind light diffusers, or make the light bounce off other surfaces, and it makes a big difference. YMMV, and good luck!
Natural lighting has been mentioned, and I second it, especially the fake windows... but I would also lay sod down (and maybe add a few grow lights to be switched on at night). Not because it has any benefit. I just think it would be neat. Also, some quiet fans on one side of the room that only blow occasionally... hidden behind a bunch of potted trees.
Bad news... I bought you a dog.
My first idea as an audio guy would be to take some cues from the music industry in their control room designs. Sure, maybe you don't need a client couch for hanging out, but you should be able get everywhere you want in the room by rolling chair (minus if you have to dive behind screens/machines to get things fixed, but even that should be accessible), favor more, smaller, warmer lights over florescent (on dimmers, if you really want them to be as comfortable as they want).
You might want to look at Proximex Surveillint as a command and control package (www.proximex.com) for the various physical security systems.
You can purchase some really high end equipment to manage multiple monitors on a videowall, but you shouldn't. Use standard PC level hardware (or lower end rackmount depending on space requirements) with no more than two display cards each. Drive all your monitors separately then tie them together with Synnergy. You can still administer them all from a single workstation, fairly seamlessly, but you don't have a single point of failure, and you've probably saved hundreds of dollars. The videowall systems can also run some light duty servers especially system monitoring. (I like Xymon over Nagios, but it depends on what you want to do with it.)
So far as the monitors themselves, purchase flat-panel HDTV's. They are likely to be cheaper than similarly sized monitors, and you won't want greater resolution than an HDTV can handle for a video wall anyway. This gives you the added benefit of being able to tie in training videos, or third shift entertainment on to one or more screens if needed. Also, if one of your videowall servers goes down right before clients come to view the installation, you can quickly switch those monitors over to CNN, CNBC or another relevant channel.
The workstation tables should be glass or some other surface that can support either dry erase or grease-pen writing. Being able do simple notes on your desk will reduce scratch paper usage and make maximum use of available areas. Glass cubicle walls will cut down on noise like a cubicle would, but does not give as much of the feeling of being in a box as standard cubicles. They allow unobstructed view of the video-wall and you can write on them with grease pens.
Have more workstations than you need, and do not tie people their workstations. If someone wants to claim one that is fine, but some people will really like being able to log off, walk across the room, and log back on. This will also allow you to bring in off-shift workers when shit hits the fan.
As a security measure, get a dot-matrix printer on your firewall. Feed tail -f /var/log/authlog directly to it. If anyone gets in that shouldn't they will NOT be able to erase their tracks.
Put in a breakroom or break area that still has a view of the common videowall. When your people are taking a break during downtime, they should still be able to see if it is suddenly no longer downtime.
For the love of God (and your staff) put in a drink fridge or soda fountain and a coffee pot.
Little Brother, watching the watchers
Why don't you do your job and quit trying to pass the buck onto /. where you will just waste our time with your lack of imagination and incompetence. All you have to do is put yourself into the position and think of every eventuality -- sort of what you are asking us to do for you. Gawd the lack of hardworking young people these days ... hmm I need to look good shamming out on this project I've been assigned: BINGO! I'll post it on /.!!! PROFIT!!! Then I won't have to do any research -- they will do it for me -- and I won't have to even think for myself!! And cool be damned!! WHO GIVES A FSCK IF IT LOOKS COOL TO SOME DORK IN ADVERTISING???
Well, the best example I can think of that combines "secure underground bunker" and "happy place" is the title facility from Dollhouse. Think underground zen spa. Honestly, that space could very easily be modified to be an amazing and impressive control center--it even has the mezzanine monitoring section, but might not suit the Star Trek screen.
If this doesn't end up at least vaguely resembling the bridge from either the TOS or TNG Enterprise, you're FIRED.
Don't forget the full ESPN package. In HD, games look awesome on a 20 x 35 - foot screen.
Just to be on the safe side, make a big screen shot of an all-nodes-green Net Manager layout or whatever you use, and keep an image viewer running with that image handy. (I had a previous employer that actually did put up a screen shot of all-OK Net View or whatever, for VIP visits.)
Give a man a fish and you have fed him for today. Teach a man to fish, and he'll say "WHERE'S MY FISH, YOU IDIOT?"
Check this URL for an (swedish) article for the swedish ISP's serverbunker Banhof. They do have some great and cool solutions, even to make it un-bunky feeling.
http://www.sweclockers.com/artikel/6109-sweclockers_besoker_bahnhof
24x7 ops in a confined space that you want to actually work inside means caring about the two things overlooked by both IT and Management.
1) Human waste
Unless able/willing to get time outside the tank, people _are_ going to: snack,drink coffee & water, etc at their stations.
The bathroom (mens and womens) is going where? Outside the double tier biometric locked doors past the guard?
In a crisis or crunch the time use for bathrooms actually goes up as more coffee and crap food (fatty, sugary, glutten, etc) are ingested & a trip to the sewage system is required.
Also bathroom facilities need to be built with those things necessary for those with disabilities; with obestity / diabetes and those with really bad digestion (IE get a really good odour neutralizaing vent system) Also some sound damping is necessary. After a shift change would you want to hear 14 flushes in a row at your station?
If you want a real good idea of how human smells take over a confined space take Greyhound to anywhere on a trip longer than 8 hours.
After a few hours you'll be begging for fresh air & a decent restroom as well. You can figure out the max time people can hold it as would need to if they pass outside the security areas of your ops center.
2) Housekeeping & Janitorial
How are and just who will clean the bathrooms as well as the control room area?
The guys & gals making so little as opposed to the IT guys, but do the really important grunt work during the night to porter the bathroom with their cleaning carts and supplies.
Really, although the janitors have a key to everywhere they typically will block doors open. The cleaning team is so "trusted" it isn't questioned about bypass of security doors.
If Solid Snake could hide in a janitors cart not a cardboard box - he'd go anywhere.
Additionally these are also the people that are going to empty the waste bins, recycle bins, spritz down empty cubes/stations with disinfectant / de oderizers.
My former company had a policy: if techs are on a station that station will not be cleaned.
You have to have some sort of desk rotation to move out your personnel (that monitor now covered by a different station) so the area can be cleaned adequately.
If this is not done you deserve the thick stank that will descend upon your control room.
If your running really critical Ops: A HR policy on proper hygiene and showers should also a clause in the personnel contract. Everyone laughs until you have to term a tech for stinking to much.
As a bonus I'll add that my experience is that the AC design will never be adequate nor will the heating. It will be visited often by AC techs in the first 5 years until everyone gives up hope.
The failure is that AC is typically the retail mall design of a large scale dumping of cold dry air into a large volume of space and somebody's desk (hopefully not yours) is just under where this happens. Hot and cold spots are intractable in a large open floor plan arrangement.
I worked in a Comm Center for 2 years, and now support it as the IT manager. Here's a few key points....
1. Space is important. The only thing that sucks more than working with some one your not too fond of is having to rub elbows with them for the whole shift. Every position should have enough space so that the person sitting there feels like they have their own workstation. Along this same theme, every position needs it's own phone and computer. Don't expect people to share.
2. Lighting is paramount. Avoid direct lighting if at all possible. Idealy you would like to have lights on the floor that bounce off the ceiling, and they should be dimmable. If you really have a decent budget, you can two two sets, and regular set and an off color like red, blue, or green. Often times at night, the best set up is dim ambiant light with workstation mounted dimmable, and movable worklights.
3. Depending on how much communications are going on, headsets are ideal.
4. Ventalation. After thinking about it, rubbing elbows with some one your not fond of isn't as bad as smelling them.
5. Shoot for standardized equipment, as opposed to complicated propitery ones. Those consoles may look real nifty until you go to replace the workstations in 5 years and realize you need blamo brand brackets that you need to order from Bob in Cleveland for 90 bucks a pop.
6. Less is more. When it's all said and done, you will be happier with a bare bones high quality desk/workstation combination than a complicated console set up you will need to tear into later.
We just refurbished the subway control room here in Brussels, to improve ergonomy. Things not to miss for you :
- split desks with height adjustable desktops : the half with the 2x4 screens and the half with keyboards/mice/phones/... individually adjustable with a motorized system equipped with some memory function. Each operator adjusts it differently to feel good, but if this adjustment has to be done again each day they will not adjust it after 1 week and work in unconfortable positions
- super-ergonomic seats, comparable to a car seat (with many adjustable things)
- let make the lighting of the room by a light&lighting specialist : and yes "false windows" with an "artificial sun" behind them is a pretty good idea
- this specialist must work with the designer which chosse the colors of the walls
- acoustics have also to be taken into account
- organize the workstations by taking into account who must often communicate with who, to avoid continuous usage of a phone to talk to the colleague which ist at 3 meters of you but just 1 meter out-of-sight. A "big staircase" design can be an answer
- put some "green", this is good for the brain
- don't overload the sight of the operators : more things you see more your brain queues all this mess and less you have CPU cycles to do productive things
Design it for humans, that means put the hardware as much as possible in another room or out of the way, use naturalistic colors and basically don't try and make it look "cool" since nobody really wants to work in a server room.
I'd pay close attention to indoor air quality, ergonomics and lighting.
If you can pipe in natural light, that'd be great, but if not, indirect lighting, non-fluorescent full-spectrum.
Also, make sure your software has meaningful alarms, there shouldn't be sound except if it is actually urgent. Nobody wants to have crap beeping all the time and stupid alerts mean that people ignore them or disable them (e.g. Gulf oil spill).
Indirect lighting and plants will add to the atmosphere, do not forget the little fridge for the beer and a smoke lounge.
Tomorrow is another day...
Look, you need to be practical here. There's a good reason control rooms like the one in 24 don't exist in the real world: they don't work. Forget the video wall and just make sure that your employees have access to what they need on their own workstations. You want this thing to be functional in a crisis situation, right? Then that means that the facility may need to run on backup power, and a bunch of huge monitors are going to use a lot of that power. Not to mention that that expensive video wall is going to quickly become a white elephant, and then something on it is going to fail. At that point, management is going to know the whole thing was pointless, and they aren't going to want to shell out more money to repair it.
If you want this to work, then focus more on getting your employees comfortable and functional furniture. Dual-monitor systems work well for displaying critical information, provided that you get desks big enough to accommodate them and other needed equipment. As for employee comfort, make sure they have an adequate break area that includes a large fridge, microwave, sink, water cooler, and table. If you don't want visitors to see it, just use partitions to hide it. And if you feel the need to make it look "cool", then get everything in black and/or stainless steel. And if you want to give them something fun to do, then throw in some old arcade games. If management is worried about these being a big time-waster, then make them run on tokens, and give each employee a certain number of them per day, week, etc.
The thing is, you want a facility that will actually be useful to you. Build that, and you will have made a good investment, and, believe me, clients will take more notice of that than of something that looks good but isn't functional.
Don't forget security guards stationed all around. They should all be female (and not built like J. Reno..), and dressed like Lara Croft. I mean, this is *important*.
Hey, it works for Muammar Khaddafi!
As in these
I am sure that NASA, JPL and ESA/ESOC have implemented some nice ideas in their control rooms.
Here are some pictures from the ESOC MRC http://www.google.de/images?q=ESOC%20MCR&um=1&ie=UTF-8&source=og&sa=N&hl=de&tab=wi , and NASA http://www.google.de/images?um=1&hl=de&tbs=isch:1&sa=1&q=NASA+control+room&aq=f&aqi=&aql=&oq=&gs_rfai= , and JPL http://www.google.de/images?um=1&hl=de&tbs=isch:1&sa=3&q=JPL+control+room&btnG=Bilder+suchen .
Space Agencies like the above are also used to host media upon launch events, so their launch main control rooms are on par with that need.
You do not want any visitors in a live production control room, period. Same goes for server rooms. All these areas are off-limits for visitors.
Set up one fake control room for show, with fake data on the screens, if you need to offer a visitor tour.
But do not lead anyone that is not part of your staff into a room where screens might show confidential client data (and if it's only the internal IP range and network layout of a client)!
Also, the more people are in that room, the higher the chance that someone pushes the wrong button at the wrong time, distracts staff members from their duties, etc. That's a clear DO NOT WANT.
And even if you have to bring some external techie in, say, because the A/C broke down, designate one staff member to monitor her/him at all time. One staff member per external techie. And no, that's not a task these staff members should do in addition to their regular shift. When they're monitoring a person, they're monitoring that person only, no one else and no systems, and no taking customer phone calls, either. Yes, monitoring them means literally following them to the bathroom stall door and back, or outside if they want to have a smoke (while they're outside, it's OK if you monitor them through the glass door, or lacking one, from a few feet away, so you can avoid the second-hand smoke*), and back. No matter how well you know them, no matter how often they've been there before.
*Even if you are a smoker yourself: No, you shouldn't smoke with them. Just like you shouldn't chat with them while they're working. It's fine to invite the techie to a cup of company-paid coffee after doing a good job (or if there's a longer waiting period while a device spins up or down), but keep the chatter to a minimum.
If you think I'm taking this too serious, then why did you get the idea you needed a bunker-like control room? It's not safer than the regular office if you allow visitors who might accientally spill their bottle of $SOFTDRINK over some expensive equipment (maybe the control panel for your redundant power supply) and take out half of your control room with that. NB: If it takes out the entire control room, it seems you had an avoidable single point of failure there. ;-)
( the best fluorescent light you'll find, is actually a mixture:
50% designer/high-CRI Daylight, and
50% designer/high-ColorRenditionIndex WarmWhite, mixed, so
you get both the yellow-red end of the spectrum AND the blue-green end, fully, and people look healthy in it! )
or you will keep losing people ( long-term ) to S.A.D:
We're *evolved* to have about *60*x the light we get in our "offices",
and it produces significant changes in our long-term function.
If you want functional people in there, long-term, you're going to have to engage the biological mechanisms of health in 'em.
That includes the requirement for lots of light, lots of fresh air,
stick a Consumer Reports recommended eliptical machine WITH heart-rate monitor built-in in the break-space ( so it is possible for them to get the aerobic vitalization they need, without detouring in their off-hours to some membership/gym ),
iow CONTRIBUTE to the vital health of the people there,
and they'll be healthily contributing to your work, long-term.
To do otherwise, and then pretend surprise when people aren't vital long-term,
aren't healthy long-term,
or aren't remaining long-term,
isn't worthy of the word "intelligence".
Another trick: if you NEED people to be alert to what they're seeing, you're going to have to work-with the 90-minute internal cycle...
Make it so that people are alternating, on/off, so the ON-time is only 90 minutes long, or you'll have eyeballs without alertness on the screens.
Also, group the screens so they are in 3-wide groups, and similar stuff in the same row:
most can't maintain awareness of 4 things at once ( limit extends from salamanders through us, so it's fundamental ), so making it setup so that one is looking-at only 3 monitors at a time ( the 3-wide groups ), and one can go row by row down one's block of monitors, looking for out-of-line stuff...
makes it more likely that someone's going to notice the anomalies that occur.
Go for it, get it right, enjoy the results!
check out this swedish bunker for ideas.
http://www.bahnhof.se/panorama/
At the hospital that I work at we have a room where the main phone line goes into. This is a great place to have security and all those things because you have to over staff phone lines to help with peak time that are few and far inbetween. They have a room with like a 15x15 square in the middle with open space. Lots of desks around the corners, and each person has one computer. Each person has a large screen computer for main tasks and one above them(just an eye movement, not a neck movement) above them for specific security cameras that warrant extra attention. There is a larger bank on the back wall that have more computers. There is always one computer open(just in case someone needs to be called in or one goes down) This room is joined in the secure area by the security office(down a back hallway that is only used by security) The only people who can use their badge to get in are the communication workers and the security staff(and certain managers that may need to be in there) All others must pick up a phone, smile for the camera above the phone, and ask to get in for whatever reason. In the end function over form is very important. If you have staff staying for longer than 8 hour shift you need to think strongly about their comfort. Either way, make sure to get nice chairs and some place that they can put drinks and food away from the computers. If you are worried about the staff themselves, lock computers under the desks in cabinets, and always have atleast 2 people in there. If you are worried about the access to the secure room being in a crowded area, it would be good to have a set of double doors, one unlocked and one secured with the phone/camera I was talking about.
I had a short stint with a fire department in a fairly large metro area. They handle emergency response for a few million people and have a pretty efficient set up for their war room.
It is fairly roomy so that you can get up and walk around.
desks are set up in pods with groups a people sitting facing towards the center of the hexagons they are on the perimeter of.
They have phone communications with headsets that allow them to move around
there are multiple monitors on each desk and large monitors on every wall that can be switched to show any desktop
The lighting and noise were somewhat subdued, but in no way dimly lit or overly comfortable (no high back chairs)
I do not remember the snack food situation, but there is a lot of security and it would be a pain to have to walk in and out all the time)
fwiw, I would save your company a ton of money and just use IP kvms and a software kvm management solution to tie together a bunch of desktops in a relatively open area together where the operators have some room to walk around, but are not overly distracted or lulled to sleep
ya know, massive operations centers are just soooo glass house IS anyway, totally 80's thinkin
Wherever You Go, There You Are
Strippers and a beer volcano!
If money is no object then make a couple of fake windows by installing windows frames with a TV monitor enclosed with a live feed from ourside the office. From a distance it gives the impression of a real window when one is not always possible.
.
You may wish to add animated graphical elements, flashing lights etc. These should only be used for show-and-tell with clients and investors. Normal day-to-day elements should be simple yet maximally informative (trend screens > text; basic line drawings > animations).
http://royal.pingdom.com/2008/11/14/the-worlds-most-super-designed-data-center-fit-for-a-james-bond-villain/
Coolest Datacentre ever. They have submarine engines for backup power!
Bathrooms close to the control room. So if a person needs to go there, is is just next door. Best if it is build to same space (with walls and door so noises etc does not come out) so the time to be there is limited.
Then have something relaxing in the room. Lighting should change and there could be more like a "Zen garden" environment as well. TV's and other entertaiment systems are just bad as you know what happens when there comes sport or other stupid (sport is not stupid but you get the point) entertaiment what takes time. Then people is watching TV and not the screens.
And if there are monitors for security cameras, have a light next to them what blinks when they notice movement. This especially for camera monitors what dont usually have motion.
The chemical processing industry have been working together for at least 15 years on related topics.
http://www.asmconsortium.net/deployment/guidelines/Pages/default.aspx
Might be something in there that help.
Specifically display standards and alarm rationalisation ("every alarm should have a unique action", if there is nothing the operator on shift can do about an alarm it should be journaled).
ZombieEngineer
Determine the decisions that need to be made, how they operate, what information they need to do that and how to best display that information. Nothing is worse than trying to operate a plant with a control room designed by someone who has no clue how it operates or what operator needs and as a result has a lot of cool but useless displays. Case in point - I worked on a control room design where the designers laid out nice sets of digital displays. Uncluttered, clear and totally useless during transients because you had no idea what key parameters where doing; unlike analog gauges where you could estimate max / min / average from the gauges' fluctuations.
I'm a consultant - I convert gibberish into cash-flow.
Check out Joel on Software blog. http://www.joelonsoftware.com/articles/BionicOffice.html He has a nice series of articles on designing his new office space for programmers. Much of it is not relevant to your exact situation, but you will find some good ideas (desks that can be lifted into a standing position for instance.)
... what you're asking for sounds more like Dr. Strangelove.
get a Directv box for that as you can use a RF remote to control the box with out line of site.
These are just my experience. I've worked in SSN maneuvering (reactor plant control) rooms, and at Fermilab, the Experimental Areas, D0, CDF control rooms and now days, the Main Control Room.
Keep the racks sealed, front and back as much as possible, to keep dust in the room from getting into the electronics. You might (or not) be amazed at how much dust a control room can collect over a year.
Include a fair amount of sound deadening foam. again, it's remarkable how loud pcs can get, way moreso if you have to add fan cooled crates and such.
Stay away from trackballs unless you can get *really* *awesome* trackballs, they tend to collect crap inside and are usu. kinda a PITA to clean.
Beware of the temptation to put in q00l tracklights, as they have a tendency to cause a lot of glare. Keep bookshelvs away from the consoles as much as possible.
Have your operators face AWAY from the hallway. Consider keycard access to keep distractions down. Spend the money on comfy high-backed chairs. Kitchen immediately adjacent to the control room. Bathroom nearby. Consoles should go UP, not out; it's easier to look up than turn your head and it _will_ make a difference.
I would not curve your workstations; skooching from one point to the next in a curved layout requires a unique trajectory for each endpoint; straight layouts are easier to run your chair along.
Here's mine http://www-visualmedia.fnal.gov/VMS_Site/gallery/stillphotos/2006/0000/06-0022-30D.hr.jpg kinda big, and i'm in the picture! Looks old and clunky but we manage.
HTH
Plan for change. Especially as this seems to be much more about flash than function. The aesthetic is at the whim of fashion more than usability. So Expect to change it often or to scrap it and start over on short notice.
At the last company I worked for the ops center was on a raised floor, but with carpet tiles. Some enterprising tier 1 folks discovered that at the very back of the room was a 3 foot tall crawl space. At some point they brought in those cheap folding lounge chairs, and left them with jackets under the floor. I think that 3rd shift was very very comfortable for them.
No you didn't, you still owe me $20 you need to pay before you can actually say you bought it.
Yup. No, he desperately needs to hire an ergonomics person as soon as possible! Especially if he gets only one shot at it and has no experience he should hire someone with experience and the right education for the job. The cost if something goes wrong (quite likely) when someone uneducated and unexperienced runs things is minimal compared to what a few days worth of consulting time from a ergonomics individual wold cost.
A datacenter in Sweden. Originally a civil defence command central, 30m underground in a nuclear secure bunker: http://www.bahnhof.se/pionen/gallery/ A guided tour: http://www.youtube.com/watch?v=qwlATf9xse4
Use LED-backlight LCD TV as artificial windows?
The "Death Wears Bunny Slippers" patch comes from the notion that the people who are supposed to start WWIII and end the world might just be wearing fuzzy pink bunny slippers when they do it. That’s because once you are underground and locked in your 15×10 ft capsule for 24 hours with another officer the uniforms come off pretty darn quick. It’s all sweats, PJs and anything else in the comfy clothes category after that, at least until your relief crew arrives the next day.
What hardware you have doesn't matter much. Any run of the mill dual headed desktop will do what I need - a few browser windows, a dozen xterms, and some email up in the corner. Anything else is just for show.
What's really valuable when I'm on point is documentation. I need to know:
What does this server do?
What's the procedure to rebuild it?
Who built it last time, in case I hit trouble?
Who's the business owner?
How do I reach them? A cell number would help.
What's the escalation path when they're AWOL?
Where are the contact numbers for our transit providers?
What are their SLAs?
Where's a map of our network, in case I have to creatively reroute traffic in ways that OSPF won't?
Is it up to date?
What services are exposed in the DMZ?
Where's the ticket requesting this port be opened?
Are there supposed to be 100,000 different IPs connecting here, or just three?
Where is the password vault?
What's the procedure to update the password vault if I have to change one?
Being able to find these things quickly will make me a much happier sysadmin than any creature comforts, excluding caffeine.
If you want to get into creature comforts in a windowless bunker, make it lighting. I don't want it bright, but it should be well designed to cover the space well. Good warm triphosphor fluorescents with high frequency electronic ballasts are much much nicer than the old cold ones with 120Hz flicker from magnetic ballasts. Color rendering index matters. That makes a good base lighting for the workspace. Then get a few of those industrial HID grow lamps, and have them light a big picture of a forest scene covering an entire wall. Or actually grow plants under them, and don't pay too close attention to what else people plant when you're not looking. Careful not to make it too bright, but the sun-like spectrum will break up the monotony of the fluorescents. Add some bright halogen task lights for when you need to see something well, without having to flood the whole room.
Raise the ceiling as high as you can. Rip out the ceiling tiles. Suspend the lights on cables. Let the ductwork show. Paint it all black. I hate living in a box. Exposing all the HVAC and cabling breaks it up, and I actually like the minimalist industrial aesthetic. If you want a softer look, hang some tapestries up amongst the machinery.
And a sofa.
Call it "the NOC". People go 'Whoah'.
Cwm, fjord-bank glyphs vext quiz
You will never find competent employees that will be able to monitor networking infrastructure, and also want do to/do security monitoring and fire alarm monitoring/camera monitoring.
This should be 2 separate rooms and tasks. Treated 100% different and separately. If you want feel free to put them under the same budget I guess (another stupid idea) you can build the 2 rooms adjoined at the same secure location if you want.
Cup holders.
When I saw this on Slashdot, I immediately thought back to these guys in Sweden who built an underground data centre... the pictures look cool... maybe some ideas to take away from here. I'm sure this was covered on Slashdot too at the time?
Yawn.
If you were trying to impress me try automating the entire thing and show how traps and triggers dispatch techs automatically. You could even show how the system "self-heals" as that is the buzz of this year.
Having a reduced costs follow the Sun tech group with good communications and even video conference is far more effective and cost efficient than build a bunker with local staff.
Show how you predict issues and not wait for failure.
Do it Bender-style: with blackjack and hookers. The clients will love it.
Make sure there's a "boss key" that flips things back to something that at least looks work related, so they can look busy when visitors come in. Yes, that includes their own managers, but so what? Presumably there's a way to monitor what they're doing, and if they need the boss key when they're not on break, they'll get busted sooner or later, [snip]
I suggest having the boss key set up so that when it is pressed, a very futuristic sounding stream of bleeps and bloops is emitted (in addition to the eye candy). It would improve the ambiance for the visitors, and the managers would know who is taking too much me-time.
http://royal.pingdom.com/2008/11/14/the-worlds-most-super-designed-data-center-fit-for-a-james-bond-villain/ Dimly lit, lots of blue and green accent lighting. Lots of organics, running water, etc. And keep it a few degrees below normal room temperature.
Have this playing on one of the monitors 24x7, with the audio up to full volume.
http://www.youtube.com/watch?v=y2WTARyipwU
Also, the Happy Happy Joy Joy song!
My company now hosts are servers in a massive data center (it was in the /. story of the top 10 largest data centers a few months ago) and, when we were touring the data center prior to moving in, they had a room that sounds exactly like what you are describing you want to make. The aspect regarding visitors is this, their is an observation platform, outside of the room, behind it, where there is a large window that you can see in. This is one of the electronic windows where, when you flick a wall switch, the window changes between clear / normal and between a completely hazed white semi transparent window. When it's turned to off, all you can see is light from the room. You cannot even tell exactly where the screens are. This can work wonders since, mostly you want this room to be kept private so it helps prevent people from just peering in but, additionally, since it's behind the room and since it makes no sound to activate, I don't think the staff will immediately notice that someone is there looking in. This makes for a great demonstration tool and at the same time allows you to look in on the staff in person without letting them know but at the same time, without hiding it from them.
The CERN SPS control room had carpet on the walls. This to stop us hurting ourselves when it all got too much and the headbanging started.
There were also jokes
Down with categorical imperatives
Use fiber optics to transmit the light. There are some sky scrapers that do this (and use it to grow plants too) to help decrease power consumption. The only ones I know of are in Japan but it may be worth looking into.
I'll meet you at the intersection of "Should be" and "Reality"
electrically height adjustable desks allowing a range of positions from sitting to standing. Worth every penny in my opinion and will save you massively in productivity gains. Also training people in how to sit for their backs not for how they are used to. we should move around much more than we do. In combination with good ergonomic chairs and height adjustable monitors you can produce exceptionally effective environments.
like this one or that one.
hi ! on this subject if folks gonna be in this room for many hours. gotta think of things like drinks, food, bathrooms ya that sort of thing
for me it's what i would want if i,m gonna be in this kind of room for 8 to 10hrs a day.
also the fake windows idea in a previous post is a good idea to gootta make it feel like a office not some concrete prison!
well that's my 2cents..
ttyl
What you will pay me to design?
And ye shall know the truth, and the truth shall make you free.
John 8:32(King James Version)
I know... Water and critical control systems. But they are nice and relaxing... Like windows into another world.
I've worked in a control room before, I can give you tips from my experience. If there will be more than 1 person there, make it possible for each person to have their own lighting levels and ventilation. Having people work long hours in an enclosed space like that, you are probably going to get a lot of employees that get migraines and back problems (its the nature of the job) so just keep that in mind. If you have all the employees sitting close together with 1 vent, 1 light switch and then one of them shows up with a Doctors note that says the lights have to be dim and the temperature bellow 60 at all times you're going to have drama. -- Giant screens with alarms displayed are stupid. They are just for show, and every manager that thinks they want to look concerned will use the screen as an excuse to stop by and bug your workers on a regular basis. "whats that flashing red one!!" "That's the fan alarm on the hed3 router, they have one on order, don't worry there's a backup fan."But couldn't it fail!" "If it does, we'll go stick a window fan in there" "But!!" "Get out please" etc... Trust me, this shit happened all the time to us and we finally took the screens down. -- If you DO have to have siren type stuff and flashing lights (like the spinny red kind and the tornado style alarms) keep in mind the room they are in and keep the volume and light levels appropriate. I was working in a facility which had an observation room that was about 10ft by 10ft, they kept the room very dark for some reason. One night I'm sitting there, it's dead silent and a sound not unlike God grinding his heal into my ear drum began while 3 police stile red spinning lights went off. The alarm went on for about 30 seconds and then god spoke "THIS IS A NATIONAL WEATHER SERVICE BULLETIN. THERE IS A SEVERE STORM WARNING FOR THE FOLLOWING COUNTIES" I damn near shit myself. -- If you want it to look cool, I recommend a negative airflow system. You get really nice clean air to help your employees to stay awake and healthy and neato vacuum gauges on the walls displaying the rooms negative pressure. Also, in the event of a gas leak, terrorist attack or nuclear holocaust, they'd live that much longer. You could also get a thumbprint lock for the door pretty cheap. They are worthless and easy to fool, but they look pretty cool. Lastly a white board I think is essential. You need a way for each shift to write things down for the next shift. "This alarm keeps going of. Jim said to call him at home when it does and he'll log in to reset it" etc... Stuff like that is essential. Make them write a date next to each message so you know when somethings old and can probably be erased.
At work, I use a network monitor I wrote that displays network traffic as randomly-places rectangles. The number of rectangles per refresh cycle is proportional to the packet count (as reported by /proc/net/dev). The height is relative to the download rate, and the width is relative to the upload rate. At a glance, a trained eye can tell what kind of traffic is going through the system, and that can indicate what the machine's doing at that point. To the untrained eye, it just looks awesome, especially when projected onto a wall at the end of the office.
You do not have a moral or legal right to do absolutely anything you want.
I worked in a NOC for several years and see a lot of things here that hit home for me. One I did not see was the sharing of keyboards. Everyone talked about the chairs and that is important, but i HATED sharing a keyboard.
We kept rubbing alcohol by the gallon and rolls of paper towels to cleanse everything between shifts, but I'd have loved to have "My Own" desk/space, that no other nasty folks touched.
We kept smoked glass between us and the looking glass. Turned out we had the only TVs in the office building and on 9/11 we had 3000 people trying to watch the news as we were trying to deal with mass outages in NYC. Once one person holds a door open to let a friend in, the entire office streamed in. It was bad.
Close bathrooms, smoking area, coffee mess, lockers if there is no or not enough space to store personal items, bookshelves for reference material, management team that works facing me instead of behind me or down the line would have been nice to avoid shouting over people's heads.
ball pit.
IMAGE VERIFICATION IS EVIL!
Plants
Incandescent Lighting
Comfortable Chairs
Art on the walls
More Plants
What an interesting coincidence that you would ask!
My company is soon to build a complete launch facility
for a large variety of rockets. Along with state-of-the-art
satellite communications, a research & testing department,
flight & safety monitoring and an innovative think-tank for
next-generation proposals, there will also be medical,
recreational and banking provisions for families who
will actually live on-site. Whilst reality doesn't
always transfer to our hopes, think "Cape Canaveral".
Hey, Slashdot, what are your suggestions on how we
should build this?
If it's no trouble to you, I'll have the cake and let you eat the Cake.
I feel fantastic, and I'm still alive.
Having worked in several such control rooms as a railway dispatcher, what made me mad was that the morons installed the computers under the desks (dozens, also every person working there had 6 monitors) with lots of ventilators and cold air coming from the AC in the room below. They could just as easily have put the computers in the room below, since they were in enclosed, locked metal boxes for security reasons, that way all the noise and cold air under the desk would have been avoided.
OTOH the noise-cancellation walls, floors and ceilings were OK.
If many different people sitting in the same seats, with varying weights and habits, the (expensive) seats were usually broken after 3-4 months.
One other thing, if they run a blocked, single app all the time, lefthanders need still a way to change the R/L mouse buttons, we didn't have that, it sucked for lefties.
Also, since the machines were totally blocked from accessing them, even changing a keyboard or mouse needed a certified tech, who had to be called from home during the night and holidays, that really sucked, how hard would it be to place connectors on the desk.
Single Point for Failure -- What could possibly go wrong?
Keep Doing Good.
I work in a NOC, 12hr rotating shifts, and we work 24/7/365 (and 366 on leap years :)). What I recommend is very comfortable fully adjustable high back chairs with a head rest.
As for monitors, you need to give them fully adjustable mounts, with nice high quality wide screen displays (Some may say standard aspect ratio, but seriously, widescreen would make my job SO much easier). 4 Monitors would be good too, 3 is good, 4 would be far better. Give them a powerful computer, and make sure they are kept up to date hardware wise, plus, if these people are technical, and you're giving them the keys to your castle, make sure they have admin rights on their own machines so they can fix minor issues without calling your IT Disservice Desk. Give them relatively unfiltered internet, night shifts get VERY boring. Make sure every telephone has a corded and cordless headset for comfortable use, holding a handset to the ear for long periods is hopeless, and wireless is good for short calls, but you need wired for anything long.
As for lighting, task lighting that the users can control is very important, as well as dimable overhead reflective lighting is important. As for desks, adjustable hight is important for comfortable use, and make them BIG. As for your video wall, make it a good size, and have it easily changeable if someone notices something important is missing, give them TV, hell, if you can, give them 2 'tvs', one for news, one for something else, and give them the ability to change the channels if they want to watch a hockey game at night, or if the olympics are on, etc, etc, etc.... Make sure the enviromental controls are good, and not constantly changing. Give them the ability to play music for the whole operation center, entertainment is BIG when working long shifts.
Give them a lounge to relax in before shift, or on lunch, or after shift. I think thats everything I can think of for now
It had to be done... It had to be said...
ustomizable T shirt is very cool, everyone focus, street, party, everything you can! To do the first himself. Waazula is dedicated to the customization service e-commerce platform, 3D personalized production, beautiful things, very cool
I was in charge of a small control room for the military. Obviously I won't talk about the security aspects of it, but I'll gladly tell you of some of the more practical aspects of it.
1. The biggest factor right off the bat would be the proper environmental control. Our room was a small room off of a larger main room. When the AC went out, even in the winter time the room became unpleasantly warm very quickly. A seperate AC until controlled in each room independently is important. Ventilation was also an issue. Too many adult males on a bad diet in close quarters. Use your imagination.
2. Cleanliness became an issue, since it was in a secure area staffed by a large number of people the levels of dust became an issue. Unless you have cleaning staff that is cleared to come in the area, your staff is going to have to have access to cleaning supplies and effective vacuums.
3. Sound levels were an issue. A number of desktops, plus network gear, plus radio and com gear resulted in a quite loud environment even when the equipment was not in use. Sound dampening on the walls and quiet computer systems are a must.
4. Lighting levels were a problem. The room had been converted and not purpose built. With no outside windows and normal height ceilings led to some really bad lighting and lowered the morale of the people. High/vaulted ceilings are must and the ability to change brightness level will go a long way to helping out people with cycle clues.
5. Break areas and bathrooms facilities will be a problem. If they are inside the secure areas then you run into cleaning again, if they are outside then it becomes a difficulty of getting people in and out of the area.
6. Multiple monitor setups are a must, but don't over do it on video walls they hardly ever get used. Pick up those cool robot desk arms for your monitors so maximize desk space.
I used to work for a large insurance company in Chicago. The director charged with building our NOC in 2000 basically traveled throughout the country visiting other large IT organization's NOC's and took the best ideas and made them work for us - and it did resemble 24.
Take a large crescent shaped room with a 30' or more ceiling. The video wall was three different sections (this is important for separation of displays and multiple tools at the same time). The display units were high end rear projection systems that were each hooked up to computers that drove the display and were roughly 3'x5' each. Of course there's no seam or separation between the screens. Any group of screens can be used to display anything you want (1 screen, 2, 4, 6, all, etc). Pretty basic stuff nowadays, but it was great ten years ago. The left and right banks had three screens stacked on top of eachother, by either 4 or 5 wide. The center bank was 3 high by either 8 or 10 wide.
Three rows of crescent tables with low walls in front separating them, and minimal separation between workspaces - you want people in a NOC to work very closely with eachother, especially in case of an outage. Each station had two or three LCD screens mounted on articulating arms, but not to be stacked on top of eachother like those trading desks you see with 6 or 8 LCD screens at them. That would be too tall, and you couldn't look over the top to see the main video wall without standing. The room sat close to 50 people. Around the edges of the room are various cabinets, printers, personal storage for the three shifts of employees that work in the NOC, etc. Of course high end chairs are important as others have noted. Lighting is also equally important. You have to be very careful with making sure it is as close to natural lighting as possible. The lighting we used was recessed and inset so that no lightbulb shone directly out or down on the people - it made it less harsh, but still very bright in the room based on a good design. Wireless headsets are important, and also minimizing speakerphones and any other distracting noise.
Behind the rows of tables at the back of the crescent in the donut hole section if you will is an enclosed room large enough to sit 30 people comfortably with power, phones, and network connections to cover it. The walls facing the NOC are floor to ceiling glass, and it has connectivity to the videowall of the NOC so that displays from their can be sent to the meeting room as well. It has every high end normal conference room tool you could need - multiple video conferences, smartboard, integrated microphones and speakers, etc. Everything was hidden inside builtin cabinets made of high end wood. This main room is the situation room. During a large outage, 2nd and 3rd level staff will work out of the room in conjunction with the NOC teams. Directly upstairs from the situation room is another identical room, also with floor to ceiling glass walls looking out to the video wall of the NOC. This upper room was reserved for senior and executive management use during a large outage. Engineers and Executive management have different needs during an outage and require separate spaces and separate functions, although constant information does need to feed between the two. The upper room was more of the showpiece room. It had a motorized curtain that you could press a button on the wireless control panel to open and close. The entrance from the building going up to the second floor board room does not give anything away for what the NOC itself looked like, so once everybody was assembled in the room and the button was hit, it never failed to impress first time visitors. They would always leave their chairs at the conf table and walk right up to the glass wall to look down at the people working in the NOC and see what was displayed on the board.
It was an extremely impressive setup. I am now in sales and visit customer sites on a daily basis and I have yet to see something that even approaches what this
Will his tortured mind give in to it's uncontrollable desires? Can he withstand the temptation to push the button, that even now, beckons him ever closer? Will he succumb to the maddening urge to eradicate history, at the mere push of a single button? The beautiful shiny button. The jolly candy-like button. Will he hold out, folks? Can he hold out?
You need to have control, and lots of it. Everybody, everything. Have loudspeakers that constantly blast out things like "This is the control room, baby! Let's have some control here!". Hire some dude to stand around with his arms crossed just staring at people. Occasionally he asks "What the heck do you think you are doing, mister?", then nods sternly and wanders off.
This is going to be a massively expensive exercise...
I don't want to misread your post, but assume you're asking for input from those "in the trenches," so to speak, which I heartily applaud. Spend some "quality time" with the project architect and/or interior designer. If they've been chosen well (not always a safe assumption), they should be able to tell you - in plain English, not "designer-speak - exactly what they've taken into account and their prior experience designing 24/7 secure facilities. You'll be surprise about the things you haven't considered (which they have). OTOH, if they sound like someone who designed the CEO's house (and nothing like what's planned), sound the alarm bells. Good luck with your project.
-z
I have to say that the general proposition -- "Ideas for a great control room?" -- makes me a bit nervous.
If I saw a post on Slashdot asking "Ideas for a Formula One race car?" or "Ideas for brain surgery?" I think I could be forgiven if I wondered if the folks asking the question/s were qualified to do so.
Jim
...just like the real 24, and have the place on lockdown until the mole has been identified. Other companies have (ho-hum) fire drills; you can have hostage situations. Make like the clients or visitors are a visiting UN delegation, with questionable documentation.
A couple of decades ago, I worked on the bottom right-hand screen in that NOC, which you can't see from the angle that picture was taken at, and later on a couple of desktop systems. When we were developing the applications that ran on the desktops, they weren't yet production systems, they were data collectors running in a lab where they had easy access to lots of internal databases and we could reconfigure them easily, because it was really a system designed to give the network planning whatever network data they needed . When we first deployed them on the NOC floor, because some of them had found our data to be useful, we still used the departmental system as the data collector. Which was fine until the day somebody plugged in *two* coffee pots for a party, and the system informed us that our lab didn't really have a production-quality electrical system :-) (Then we got to go buy some new servers and have people run long cables for us.)
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
On every screen there needs to be something that indicates that the screen isn't frozen. Like a ticking clock or something. You don't want to find out 5 minutes late that something is going on. We have a 4 position ticker in the general style of a clock for our DCS. If nothing is changing how do you know that the screen isn't frozen?
Oh Crap, I'm an optimist.....
Yeah. People need to be able to sit comfortably, and need to be able to see whatever screens they need to see without eyestrain or neck strain.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
A couple of decades ago, I got to work on the AT&T NOC in New Jersey. One of the important things they had there was a pair of large television screens, which initially both played CNN, and eventually (when the local cable TV network got it) the Weather Channel. When you're running a world-wide network, or even a US-wide network, it really helps to know what's happening out in the real world that might affect you. Hurricanes and blizzards were the most relevant, but if something bit happened that got lots of people to make phone calls to a given geographical area, that could be important because it made a big impact on our calling patterns.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
I think Microsoft already thought of that. It's called Alt-Tab.
My Sysadmin Blog
Also, start looking into alternative workstations - investigate iPads and other tablets. Most monitoring and maintenance software is web-based these days, no reason not to surf your network from a comfy couch or worktable. Keep standard workstations for heavy lifting - stuff that requires a lot of typing or multiple displays.
All of the solutions thus far are terrific, and I heartily endorse both the purchasing of really, REALLY nice furniture and the installation of as much free food and caffeinated (and non-caffeinated!) beverage solutions as possible to support the staff. Also, pack a storage room with some simple folding cots, surplus Army blankets, and washable pillows: during multi-day emergency response ops (we did one a few years ago for a snowstorm), the ability to tap out and take a nap for a bit will be highly prized. Remember to provide space for said nap, (optionally) earplugs/eye-masks for getting to sleep, and a laundry service to make sure everything gets sanitized after use.
Something that I considered quite intelligent for the secure monitoring facilities at work was the use of electronically-tintable glass for all of the viewing windows into the SOC. There will be times when your staff are working frantically on an emergency just as a tour group is coming through, and the last thing you want is anyone worrying about what's visible and what isn't and whether any of that is sensitive. It's far easier for the tour guide to see the red light next to the door and the glass tinted and explain that the SOC is in lockdown right now than it is to explain why an employee is viewing hacker sites with porn popups in order to snag extra information on the current worm.
Another good step was the installation of a large conference room right next to the SOC, with a full videoconferencing solution built into the room and table. When the excrement hits the spinning blades, one of the first things the executives like to do is open a bridge call to keep everyone updated. By making the videoconference (with individual mics at each seat at the boardroom table) in a room adjacent to the SOC *but not actually inside it,* the SOC employees are free to continue working without listening to the executive blather, but are easily able to pop out to the conference room and update the execs or just jump on the call from their desks to pass in update information. Even just a separate large conference room with a board-room table, speaker phones, and a projector/videoconference system (I like an HDTV with cable feed and a Mac mini with a good mic and camera for a quick solution) would be fine.
Finally, since you mentioned monitoring physical security systems like fire and patrols, you should look at some of the Radio-to-VoIP gateway solutions out there. That will enable field techs and patrol officers to communicate easily with the SOC without requiring separate radio comms setups, and you'll be able to write scripts to automatically notify them of issues by sending text-to-speech updates via their radios.
Here's a good example of an important control room. Generation 101: How PJM Operates and Dispatches. This is the control room that controls the power grid for the eastern United States. The presentation covers what the people there do. The slide "Dispatch Operations" shows the operating positions, and the next slide shows a more recent picture since the displays were upgraded.
This is a very organized operation. There are five positions, and they have specific responsibilities. Each position has a number of screens of its own, and the positions are placed close to the big wall screens most relevant to them. The transmission operator is in front of the transmission screens, the generation operator is in front of the generation screens. Those are the two people who directly run the grid. The shift supervisor, master coordinator, and reliability engineer sit further back on a raised level. There's a viewing gallery in back, behind windows, where visitors can watch but can't bother anybody. Interestingly, there are curtains for closing off the viewing windows, and they're on the control room side.
If you want to see some of the displays they are looking at, the data is available in Flash format. There's a economic system involved in power generation, with bids for power, so all the market players have to be able to see the data. The interaction between the control room and the bidders is complex. The control room does have the option of ordering "non-economic operation", where they tell generators what to do instead of merely sending price signals, but they only do that in emergencies. In more serious emergencies, they can order "conservative operation", which means all generators come on line and provide power to meet the load, regardless of cost. That's very rare.
Note that this is an operating center, not a response center. There's a routine workload. Over the course of a day, generators are started up and shut down as the load changes. The two people in front mostly handle that. The three people in the back row are there mostly to deal with problems as they come up. The physical layout reflects that. A data center or a security monitoring center has a completely different workflow, and may need a completely different physical layout.
A nautical themed noc would be soo cool. Screens with fishes swimming around represent monitored objects. Fast fishes mean lots of data and slow fishes mean not so much data. Upside down fishes mean whatever the fish represents is toast.
Blend MRTG style graphs over the fishes skin textures.
Big fishes can have a bunch of little fishes following them to represent a constellation of related objects.
Fish tanks with dirty water mean the hvac and or power distribution system is screwed up.
Fish tanks with cracked glass and water pouring out means the server room is currently being flooded.
The presence of electric eels electrocuting a fish means a PSU is crowbaring.
Fishes that appear cramped and unhappy in their fish tanks mean disk space is low.
With this setup you'll be able to respond in a moments notice whenever something fishy occurs.
I've been reading through lots of the posts here and there seems to be quite a few instances where people have spent a lot of time and money designing systems that ultimately don't get used. My suggestion is this: Start simple. Get a working system up and running and do it with budget to spare. Once your team has been using it for a while ask them what they want.
Unexpect the expected!
Having at least 2 Projectors each capable of running either the same or different desktops can be very nice. you could even have them set up to their own computers or better yet to a giant kvm switch so anyone's computer can be displayed for everyone to see. break rooms/vending machines/gyms are nice, and lockers are even better to store everything for work in. ever consider a simple courtyard with a few benches?? might be nice to relax for a few or to have lunch under a tree or a cig. i'm not a fan of particle board walls, so keep it open with lower walls if any to separate desks. I find that sometimes having multiple people working at one desk can inspire more collaboration and creativity if they work tightly together.(based on projects, not personality) Just my 2 cents....
Secondly : NO - do not put all your eggs in one basket. Keep the bunker as stand-by short-term fall-back. Not only is this your resilience (including all those system changes that cause grief) but you can also run training exercises down there without impacting on the normal system. Also spending a short time in the bunker helps people focus on issues, objectives and methods a lot more - which counts when there is 'an emergency'.
Thirdly : A secure but non-hardened environment is much cheaper, 'normal' and easier to alter than a bunker. It is FAR better to give people the tools they need to manage the everyday noise and have the necessary grades of experience of staff (=quantity) to learn the practicalities under supervision than to lock a apprentice in a box with flashing lights and coloured buttons hoping that they will not make too many mistakes.
You really have to think about what colors you want on the walls. Color psychology is quite true.
If you want your people to be quick on reflexes, choose red walls. The downside is: there may be some increased aggression and a lot of testosterone.
If you want them to make the best informed decisions possible, albeit a bit slower you should choose green. It will also lower testosterone levels quite a bit. It would, however, lower their ability to react fast in the event of an emergency because they think a bit longer and deeper.
Well, I might have a way, but it only works on a semi spherical planet in a vacuum.
Really, a football table. Like soccer. You may not like to watch it on TV but it's an excellent way to blow away some cobwebs and unlike most games you do it standing up.
My company used to have a true Command Center approach, with a fancy multi-level desk featuring rows of PCs on the bottom and screens for each on the upper shelves, redundant power, fire suppression, hot line phones, monitors to display all the security cameras, and of course electronic locks on the doors to restrict access.
It worked well for a while. We used this setup to process truly massive amounts of data and trouble-shot some amazing stuff. But after a while hardware goes obsolete or dies.
That whole room and a mirrored facility in another state have all been replaced with virtual machine versions of the same systems. The secure room isn't needed because the actual VMs themselves live in much smaller secured server rooms. The workers running the VMs work in normal cubes you'd see in any office, albeit with an extra screen to help juggle more than one thing at a time.
The old secure room is now just a break room.
We have found that virtualizing everything has massively reduced our hardware spending and issues, because VMs don't typically suffer from the same kinds of wear and tear and dust bunnies and the like. Uptime has gone from days to years, on Windows stuff, not linux. Part of that is via better hard drives and some redundancy. Basing things on dedicated servers versus scattershot hardware has also reduced the load on our IT team.
On the productivity side, it has the added benefit of letting workers VPN in and control the same systems they can run from their desks. The VPN works anywhere there's an internet connection. I've run the show from a tethered cell phone, wifi I borrowed, or from my home internet the same as if I was there. OK, yes, you can VNC or remote desktop too, but those interfaces are usually somewhat different than the "real thing" people use every day. With a VM, the VM is the real thing and works the same no matter which client viewer is being used. There is no obstacle to doing the same work the same familiar way.
I'll be spending my Labor Day doing that for at least part of the time.
Sig for hire.
Seriously, I have seen so many cases where people think they can do interior design themselves, and get it all wrong. I am not talking about decorating your bathroom. I am talking about functional, human friendly spaces where people need to spend working time.
"Glass cubicle walls will cut down on noise like a cubicle would, but does not give as much of the feeling of being in a box as standard cubicles. They allow unobstructed view of the video-wall and you can write on them with grease pens."
They also allow an unobstructed view of things you really don't want to stare at such as bright lights.
My experience is from the dev cubicle floor, not a NOC, but it applies at least as much to monitoring a screen or screens as to staring through the screen into the code-realm. Bad placement plus a low ceiling meant that I had a bright fluorescent fixture showing just above the cubicle corner. I tried sunglasses, moving the monitor, etc. and ended up roofing that corner of the cube with cardboard, just so I didn't have that light stabbing my eye while I was trying to dive into my code. Glass cubes will exacerbate this into a no-escape situation. Nobody's going to want to keep staring at a view which is actually painful.
Even if you don't go with glass, before you sign off on that build, have a short person and a tall person test out every station for glare within scope of view. Be prepared to hood some of those lights to keep them out of people's eyes.
Right now, most security equipment is NTSC video; which means low resolution, crap recordings. But HD is here -- it's just that the multichannel DVRs and switches and so forth aren't really taking over from the crap NTSC stuff the way they should, even though it's been a couple years now. The difference between video and HD for security is *amazing*. I'm still stuck with video, and we've had a couple of incidents where the video should have helped... but you just can't make out details worth spit. With HD... whole 'nuther ball game -- we've got one HD monitor / recorded I cobbled up. It's amazing. Did I mention NTSC video sucks? Yeah. Sucks.
If I were you, I'd tell your company to wait a few years. Seriously. Otherwise you're going to be on the trailing edge of basically crappy security gear -- everyone else's control center will have the good stuff, and you'll still be trying to expense off a zillion bucks worth of junk.
I've fallen off your lawn, and I can't get up.
After my experience in satellite control centers, I have seen both good and bad. Video Walls. are a cool idea, but not always practical. I won't explain to much, but as these systems not usually are pieced together based on requirements, thena bout version 3 or 4 they start to become usable as the user and complained enough. There is a lot more to consider in it all. Message me if you want more info, but there is so much info. Air handler placement alone can cuse problems. I now have a ringing in one ear. But if you do nothing else to talk to techs and users. (blurry eye tonight can't chase the typo's)
Don't make them share kitchen space with other office workers in the same building. It's just not fair on anybody. Give them their own space and their own fridge/freezer, microwave and sink. I've worked in two companies where I had to share a kitchen with NOC/SOC staff and in my experience the night shifts and weekend shifts degrade into a dorm-room mentality where they think that it's okay to raid the freezer and leave 2 days of dishes in the sink. I'm pissed off of going to get my monday lunch and finding no clean dishes to heat it, or that my lunch disappeared over the weekend.
You can create the greatest or cheapest control room and it will fail unless you known the process. The company, it's culture and it's people are not going to change (much) just by you building a control room.
How are they handling crisis now?
What's the impact of a failure?
It's not about chairs and monitors: it's about how a company is most effective. Using big bucks to create a control room is just a waste of money if you do not know how it is going to be used.
Don't talk to us:talk to the people how are actually doing the crisis-handling at this very moment.
If you think I'm taking this too serious, then why did you get the idea you needed a bunker-like control room
Because we don't have a choice in the matter. If we did, trust me, the place would have windows.
Approved and escorted visitors are not a threat.
Others who we don't fully trust e.g. prospective clients, can view via CCTV or an observation deck, both controlled by the operators.
Do you or your partner snore? - Visit www.snoring.com.au
I hear you. This post (which I am *thrilled* made it to the front page) is largely to obtain a second opinion for what we've been advised by 'experts', who ultimately aren't spending their own money, they're spending ours.
Trench advice greatly appreciated.
Do you or your partner snore? - Visit www.snoring.com.au
I don't want to cast aspersions (or worse!) on your experts, but in my experience most of those
people (especially architects) never go back to see how what they built is working out, what they did
wrong, what could be improved in their next project, etc. I'm sure there are some who do, but it certainly
is not standard practice, so you're wise to ask the slashdot crowd for real experience.
Having participated in designing control rooms, and been dependent on the people actually doing the monitoring, it's safe to say that I know how NOT to do this.
First of all, I was hoping you're not doing a show-piece. Making a functional control room that doubles as a way to show how cool you are doesn't really mix. I've seen this done a few ways.
First, there is the adjacent room with glass windows to the control room. Highly disturbing for the people doing the monitoring. Then, the one way mirror. Even more disturbing, as you get paranoid after a while. Lastly, video surveillance. Not doing any wonders for the people working in the control room either.
Not knowing what you're monitoring, or the workload, this is usually a job that requires a lot of concentration. Ideally, there should always be more than one person there. If something bad happens (the reason why you have monitoring in the first place), you don't want people panicking. Having seen the same old alarm for the 200th time, there could be a tendency to ignore it. Being able to consult with someone at 3 AM (other than the person on-call), is important.
While it's nice to see everything in full working order, the important thing is to make people notice if there's something wrong. (Even better, early warnings before things go bad.)
You should have one large monitor showing things that have gone wrong, and it should be the focal point in the room. If at all possible, make the system give a sound signal (or make the lights and/or monitors blink) to get your attention. The surrounding monitors can show how your systems are doing. You just don't want to miss that critical error. It's bad for you, your systems and your customers.
You are going to depend on the monitoring equipment. Duplicate the systems, have all kinds of redundancy. You don't want to miss those important alarms because the monitoring system is down or unavailable. This means that you need people who can administer the system and modify it if you have equipment that is not supported by your monitoring software. These people will be vital in ensuring that all systems are monitored correctly.
Guessing that the people in the control room won't have a deep knowledge about all the systems they are monitoring, you will have a number of people on-call.
The two most annoying things for a person who is on-call are: false positives and missed positives. That's right, while you get grumpy about unnecessary wake-up calls, you get angry if there is a missed important alarm.
I have worked with systems that have a very high number of real alarms, hundreds each day. The people in the control center will get reluctant to call for the 20th time that night. They will hope that the problem goes away, or they will deliberately misjudge alarms so they don't have to disturb the people on-call. This will escalate if you have a high degree of false positives.
Finally, the last thing you will want to avoid is confusion. Keep short, up-to-date, easy to read instructions for each and every system. What is the check-list for every known system/alarm? Who do you inform? How do you inform them? How do you escalate a problem if the person on-call is not answering?
So, not going into the discussion of which kind of soda/chair/toilet paper you will need, here is my list of advice:
* Try to avoid making the control room a show-piece.
* If possible, always have more than one employee in the control room.
* Focus on catching alarms and problems.
* Make audible or strong visual alarms if possible.
* Do what you can to avoid down-time on the monitoring system.
* Have personnel that can administer and modify your monitoring system.
* Make sure that all systems are monitored, and monitored correctly.
* Find a way to avoid false positives.
* Don't ignore alarms.
* Document all procedures.
2) Housekeeping & Janitorial
Don't only think what, but also think how. There's nothing more fun than trying to solve a critical time sensitive problem to the background music of a noisy vacuum cleaner. The control room at our refinery has a novel remote compressor with some air ducting that runs under the floor to vacuum ports at the wall. Rather than drag a vacuum around the cleaner will come in with essentially the tube and handle, and plug that tube straight into the wall.
The end result is the only part of the vacuum cleaner you hear is the person walking and the gentle sound of dust rolling up the pipe. While I have no hard stats on the results I'm sure this has saved not only the sanity of operators but also the lives of cleaning staff.
As a employee in the Danish television industries, I'm used to working in rooms with a lot of screens and a lot of comotion.
See if you can get a chance to get a peek into a TV production lorry. They're made for this kind of thing.
However, they may be a bit cramped - but just expand the idea.
Cheers and good luck
That could be the big red button!
Do we get consultancy fees for our opinions/expertise? This is a "massively expensive" commercial venture....
No sig today...
Funny? This should be +5 insightful.
Our "control centre" at work was basically built by us and the network guys, money went on decent monitoring software and nice big hi-res screens, all controlled from a single workstation with three graphics cards (soon to be replaced by a single ATI); we cam in massively under-budget because, hey, form follows function. We were told to spend the rest of the money making it look more like a movie set, otherwise the directors wouldn't be impressed - so we spent a couple of hundred quid on halogens, one of our guys cooked up a homebrew COM-controlled lighting system and we now literally have a "panic button" where enter + scroll lock on the num pad turns off the main lights and kicks in a bunch of OpenGL screensavers full of pointless 3D bar charts, spectrum analysers (on a screen labelled "Continuous Raster Attenuation Particulates), random beeping noises. Most new techies that see it burst out laughing at the contrived stupidity of it all.
The best bit? The eight hundred quid left over went a long way to paying for a whole bunch of delicious scotch at a bar around the corner as we got pissed and laughed our arses off. Biggest grin was from the slightly sozzled manager who smirked as he signed off the bar bill as a "consultancy fee". Fun times.
Moderation Total: -1 Troll, +3 Goat
We have such an control center. First of all, you need a centralised fault management tool, so that everybody can observe their own part of the system. Alternatives are Netcool, TeMIP, etc (there are many many more I am sure). You should have a screen as big as possible. Mitsubishi has some solutions. Putting on some fancy graphics that are updated every x minutes would be nice. Video wall is just for the show, it does not help anything. But it is really a show. You can open CNN to show that you are always watching the world :) Our solution is, room is always dark such as a cave. It is not nice but, in an illuminated room, big screen don't seem nice. You can make the operators desks so that all lines seem to be on large stairs. It makes the place like Houston control center.
You have the chance to build a cool control room and what do you come up with? Some Lame TV-Show?
Man, there is only one thing to do: Take a deep breath, put in the DVDs and build the freaking bridge of a Star Destroyer!
That you even have to ask is totally unbelievable!
Go for a Star Wars look, with the operators sitting in trenches, and the supervisors on a plateau above. Coolest ever!
You can't correct anyone here. Slashdot is a failure non-nerd website now. They're trying to justify the existance of Slashdot by passing some lame stories and topic to discuss. Does anyone register to Slashdot anymore? Why register to see something a few minutes in advance? As far as we know, 'lewko' is just another troll like Roland Piquepaille.
I'll repeat it for effect. Go tour the CR for utility or rail. They do it right.
I guess smoking should be mandatory, and the armed guards should have the option of using rubber bullets. Oh, and robots! Lots of autonomous killer robots!
The Russians have won. They have made the world a cesspool of distrust, greed, fear and hate.
I've worked in the industry for a while. In the 90's even helped a big multinational write some software for their NOCs...
Most of the NOCs I've been in have been real caves. No windows, subdued lighting, big video walls.
Try to use indirect lighting as much as possible, don't make it harsh etc... one of the best ways to do this is to use fake windows. There are companies that will install a lightpipe; basically, there's a lens / collector on the roof, a reflective tube through the building, and a frosted glass skylight or window in the room. It has the advantage of being extremely green, and follows natural lighting cycles.
A more traditional approach is to simply install a full spectrum light behind the window 'facade'.
At the company I'm at now, we run a 3-4 man NOC... and have at least 15 screens lol! Generally, at each station, at 'desk' level there's a standard CRT, then at standing eye level there's a larger plasma displaying a visualization of some component of NOC operations. One might be VPN geographic status (think big map with red and green blinkies), another is system status (think big list of servers with red and green blinkies) and so on... then, above that and visible throughout the center, is a larger screen showing either a rotation of all the other visualizations or highlighting the critical issue du jour. As others have noted, it's the software that matters. Get or build the right software to monitor correctly. With the right software, this setup has the advantage of being very visually appealing, as well as functional. Our NOC looks out on our server room, so we installed the equipment in the same cabinets we use in the server room, just the half height models above the desktops. The company we get them from sells glass and smoked glass doors (we use smoked glass for the upper screens to cut down on ambient glare), and will even silk screen our logo on the doors for us... 3 name or logo changes over 7 years, and every time we've shipped the doors back to them to have the names scraped off and reapplied!
That man pro'ly got more pussy rubbing on his walls just because of all that he did when this was reported on Slashdot like back in 20.1.2008.
He could go out and buy any kind of peice of shit Recreational Vehicle, rip-out the interior of that RV
down to the bare walls, then throw all his apartment inside to make the most awesome roaming landcraft that
money could buy.
ENGAUGE!
Sharks with frickin' lasers!
Best way to get everything right is to order desingn from company that specializes for control room design. Yokogawa is pretty good.
Special suggenstions for computer hardware:
- Monitors from Eizo. They just make the best monitors for control rooms, medical imaging, etc. http://www.eizo.com/global/
- Matrox graphic cards are really good for control rooms. It's their specialty and they exel in it. You can get multi monitor worstations that are silent http://www.matrox.com/graphics/en/
Dyslexics have more fnu.
If Solid Snake could hide in a janitors cart not a cardboard box - he'd go anywhere.
In my unfortunately extensive experience with this over the decades, the problem is not so much James Bond sneaking in, its "trash bags" full of laptops and HDTVs sneaking out. Even in the "best cities". Everythings gotta be locked down, even/especially in a security theater environment. And all employees need at least one lockable drawer for their purse, cellphone, spare change, insulin syringes, whatever.
You really need to understand what security theater is before you plan your "cool security system". Good security can be annoying, so sometimes people do the security theater thing and install annoying things mistakenly thinking that must make it secure. All the biometric hand scanner, card reader, and DNA sampler will do is annoy the janitor as he hauls his bag-o-laptops out the door.
The worst thing you can do is contract out the janitor to a random collection of temporary illegals. Then anyone can/will just walk in, block the door open, empty the offices, and no one thinks it odd. At least if you have a direct employee, at least HR had a chance to screen them and at least one person on site might know whom they are full access to the entire facility.
"Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
Most of the time, when such a project is undertaken, technical and technological factors get the first, and often the only, priority. What is left out is the human factor–how people with interact with, and use, the environment. There has now been considerable work done on this by human factors engineers. So before doing anything else for your control room, read Kim Vincente's book, "The Human Factor", especially the parts where he deals specifically with control rooms. Making the correct human factors decisions can reduce accidents and deaths resulting from poor design choices.
Reference: Vincente, K. (2003). The Human Factor. Revolutionizing the way people live with technology. Canada: Alfred A. Knopf. Amazon: http://www.amazon.ca/Human-Factor-Revolutionizing-Live-Technology/dp/0676974902/ref=sr_1_1?s=books&ie=UTF8&qid=1283779722&sr=1-1
I think Microsoft already thought of that. It's called Alt-Tab.
Ok, so instead of having to tell your visitor "it's OK for them to be checking out Slashdot because they're off-duty at the moment", you'll have to explain him "it's ok for us to run a Microsoft OS, because despite its appearance, this place is not really controlling a nuclear power station (or rocket launch, or ...)"
There is a lot good advice posted. Much depends on the operations that will take place in the room. Most control rooms have daily cycles consisting of the day shift with maintenance working on things and interacting with the operators, a swing shift where things quiet down, and a graveyard shift that consists of long hours of quiet and talk among the operators to pass the time. When a process upset or emergency occurs the operators must respond quickly, but with a well-run process upsets don't happen very often.
I have spent too much of my life in control rooms, paper industry and power plant. Control rooms existed before computers, consisting of expanses panels of controllers and switches. Since there were so many controls, at least half of them could only be reached by standing. The operators were trim and in good shape. They frequently spent their time sitting down because there was nothing to do, then immediately came to their feet when action was required. Humans function much better standing up. They think better and it is best for them physically, the two go hand in hand. Many control displays work by touch. Proper display design is an elite craft. Arrange the displays so that normal operator input occurs standing and dealing with the display at eye level. People will object that keyboard input is required and keyboards have to be horizontal. I don't know your processes, but the vast majority of keyboard interactions involve display selection, alarm interaction, and numerical entry (setpoints, etc.). Sometimes tags get typed, it isn't frequent or common and almost always results from bad display design. Anyway, numerical entry can easily be handled by a vertical keypad. It certainly doesn't need to be horizontal. Display selection should primarily be handled through the displays, using proper design. Alarm interaction needs its own small keypad beneath the numerical one. In the future, voice recognition will be used with the displays, but right now it would be a gimmick.
By the way, my company provides simulations for operator training. Contact me if you are interested. Start-ups, shutdowns, upsets, alarms, tags, etc. All the usual suspects.
To show it off you can take a tip from the NAP of the Americas. They have a large conference room immediately behind the operations center with the adjoining wall being entirely "snap glass" lcd panels. Super-cool effect for showing off the ops center with all of its high-tech video monitoring equipment. Touch a panel on the wall and the fourth wall of the room immediately becomes clear to show the ops center with all of the cool network traffic monitors, etc. Touch it again and the wall goes white.
FWIW, I highly recommend taking the tour there. Standing next to the pair of terabit routers that feed the root servers for ICAAN and Verisign is pretty cool. They don't really look any different than any other large router chassis, but c'mon, it is a terabit router at the top of the internet. Ok, maybe you have to be a nerd to appreciate that... Even more geek-cool is the "data exchange" where you can get on to the Internet at full wire speed for cheap. Just toss a cat-6 cable over the wall and plug in to the routers: boom, gigabit connection right into the backbone of the Internet. Pretty much every major and minor Internet provider has a presence in the exchange, and they have zero last-mile costs so you get really competitive pricing. And since all of the undersea cables terminate there, you are basically one hop away from most of South America and several places in Europe. It is pretty cool to see a route trace that shows 3 hops between you and a server on another continent. Or to ping any major search engine and get sub-millisecond ping times (because that's one of their server farms, right down the hall there...).
A Swedish company called Banhof can perhaps give you some inspiration. They have monitoring and servers in an old nuclear safe bomb shelter in Stockholm. Well worth a look. Pictures: http://www.bahnhof.se/pionen/gallery/ Video: http://www.youtube.com/watch?v=qwlATf9xse4 http://bahnhof.se/video/movie1.mp4 http://bahnhof.se/video/movie2.mp4
Make it easy to use and work every time.
Crestron DigitalMedia for switching
Crestron DVPHD-PRO (or DVPHD-CUSTOM-GB for annotation) for windowing/scaling
(Extron has equivalents for both of these but I can't vouch for their ease of use)
Runco WindowWall to maybe fake some windows and keep sanity. Others have mentioned plants and other creature comforts - if you don't feel like you're in a cave you'll be a little more comfortable for a longer period of time.
Baswaphon or some other acoustic apsorption to keep the chatter and clicking from being heard all over the place
Light light light. 6500K light might be 100% neutral depending on who you ask, but never looks natural. Go for a warmer color if possible.
In my experience employers that directly or indirectly turn a blind eye, ie who hire against national/local labor policies, already have deep problems anyhow. I wouldn't work nor recommend anyone work for such businesses.
For legit businesses, most housekeeping staff I've known are decent hard working folks often looked down upon by the Management and IT guys if even recognized at all. Most won't steal nor want to be implicated because it costs them their job or the contract instantly. No investigation period.
I brought the fact they block open the doors from my experience of having to account for disabled door alarms and other bypasses to areas meant to contain sensitive customer info. Its a real problem solvable by having shredders & screen savers internally to the command center.
In my experience its the "entitled" IT that typically steal. What happened to the old parts locked up in some drawer from all that upgrading last month? Its now in somebody's home system or ebay'd without a trace.
As for low skill employee, contractor petty theft (phones, change, purses) that is an entirely different realm of security.
However being wiped out of laptops and HDTVs either in one fell swoop (stupid criminals who have to fence a big score) or trickling them out (smarter but still stupid criminals who open up an opportunity for surveillance) is not the issue.
The financial loss is not the IT management concern. Equipment should already in plan to be replaced and insurance covers real theft.
However explaining to management the downtime, that the laptops went out the door without encrypted partitions, that the serial numbers of equipment were never inventoried is a bigger theft -- IT guys ought to know this as stock and trade and if they were paid big money for no real work -- this is basically ripping off the stockholders/owners. IT & management took "big" money for not doing basic diligence in their work.
Anyhow most every large building has a door -- the smokers door -- that never latches and just needs a few tugs or leverage to pop open. Its just a matter of observation and turning doorknobs. I'm sure the building rent-a-cop does that right?
Those criminals that try to score used electronics probably already need rehab. Else want to fence them for cash for getting presents for the wife or girlfriends.
No criminal retires on the fortune accrued by stolen used electronics from a business.
If they were organized they would hijack a trailer headed to bestbuy or frys and live well for a few months or 1/2 a year.
Not just a little fountain something of significant size with a good amount of falling water. The extra natural moisture in the air makes the air smell better and is easier to breath, than simply recycled AC with a humidifier. It helps regulate the temp a little. Also lots of plants. And when people talk about good lighting use natural color balanced light, or more simply florescent fixtures known as grow lights. And make sure maintanence isn't allowed to cheap out on replacement bulbs later on and switch your staff back to cheap standard flourescent when bulbs get changed out. Or your hard work is for knot.
If there's a possibility of a crisis that will call for flat-out effort for days on end, I'd suggest:
I've seen Ops Centers installed by professionals which were worse than useless. Sure, they were big and plush and shiny, and had a large glass partition behind them so that the executives could be walked past it and be impressed (which was probably a big clue), but they were manned by the collected deadwood of the enterprise and despite all their super-modern network-wide monitoring gear, they were usually the last ones to know about any outages.
In daily use, they usually ran a bunch of sports channels on the big screens and just sat there drooling for their entire shift. If it hadn't been for the fact that in one case, management had forcibly relocated the Network team to the little minibunker next door and demanded all calls to Networks go through the fancy Center, the rest of I.T. would most likely have just quietly cut them out of the phone network altogether and no-one would have been the wiser.
For what was paid for these places, they could have hired a warehouse, SFX team, and a bunch of actors for the one day a year the executive walked past the glass window and peered into the zoo, and come out way ahead.
I work in a 24/7 911 police, fire, and ems dispatch center. We just built a new center and used Watson Furniture for adjustable position consoles, check out their site at www.watsondispatch.com , lots of options as far as desk size, configuration, # of monitors, etc. I was on the design team and we basically gave them the dimensions of our room, number of consoles we needed, equipment used, and they were able to draft up a number of options complete with 3d drawings.
The Changing Face of Control Room Design
Handbook of Control Room Design and Ergonomics [Hardcover]
"Fish" (David B. Trout)
I work as technical support in an Emergency Ops Center for one of the departments at a State level. We don't run 24/7/365 but we do 24/7 when we need to be up. The EOC built was a repurposing of an existing office space, but other than sticking with most of the walls we had we got to build from the floor up. Here are some of the things we did that seemed to have worked out well:
1) Use a raised floor and/or drop ceiling. It's so much easier to run wires when you have some space to tuck them into.
2) We've got (1) high intensity projector unit (no need to turn off lights) & a few wall LCDs (2 in main room, 3 in meeting rooms) connected to one computer with a couple multiple output cards. These seem to be enough for most of what we need to display to the room. Everything else people look at on their station computers. Figure out how much you need shared displays for and don't go overboard. At some points it's just information noise.
3) Related to displays: It's not as much of a problem now, but LCDs over Plasmas if you're looking at long term use. I know EOCs that had monitor burn in or burn out after a year using plasmas. This is especially true on anything displaying news stations. Those ticker trackers are a killer.
4) Good multi-monitor support software like Ultramon is a god send.
4) Try to keep facility noise down. We've had to baffle our projector because the fan whine made it hard to hear in the seats under the projector.
5) To support what has already been said, comfortable furniture is a must. Chairs are important. We use a chair that is supposed to be guaranteed comfortable & safe for 24 hours. They're not cheap and they have the 20-30 page instruction manual to prove it. They're also the most 'borrowed' piece of equipment in the building. Of equal value you need to look at desk height, monitor placement, desk/table alignment and the rest.
6) Related to desks, put the computers out of the way. Due to an order error we got the wrong computers and we had to hang them. This became a big mistake and a lot of painful knees. Because of our purchasing process it was years before we could fix this.
7) Also related, don't discount giving empty desk space. People need room to work.
8) Move long discussions & meetings out of the main center. We've got (3) meeting rooms just off the EOC for these. Added so that participants can monitor the EOC, and rather than relying on high tech camera feeds, we put in windows and shades. They’re simple, inexpensive and don’t need much trouble shooting.
9) The comments above on fruit & snacks & coffee are all very good suggestions. Food is a comfort.
10) Make sure that you have enough people who know how things work and take the time to make things simple to work. Throwing tech at a problem is easy, but it takes time to find the right stuff & to put it together well so that you're common user doesn't need a 100 page manual to use it all.
Things we might have changed:
1) It might be nice to have a few standing workstations. Not necessarily for assigned positions but so that someone can use one for a while if they want to stand and stretch, or for people who just need to check one thing and then head out again.
2) We put our IT closet & UPS units just off the EOC because there was a nice space there at a convenient distance. But once it was in we had to put in extra work for noise baffling. Try to put noisy equipment a few rooms down if you can.
The best advice? Figure out your core functions for the room, ours was communications and information handling, and then build the room to facilitate this. Make sure people stay comfortable while they are using the equipment. Minimize distractions. And KISS, KISS, KISS.
check out evansonline.com for some good ideas
Anyhow most every large building has a door -- the smokers door -- that never latches and just needs a few tugs or leverage to pop open.
And that's exactly why you should let your staff smoke like chimneys while at their stations. C'mon, do we really think today's OSHA-pansy NASA could deal with a situation like Apollo 13? No way - real crises demand not just caffeine, but nicotine, too!
After all, what's the point of a controlled access area if you can't violate a few local laws?
For video wall you may want to consider Christie Microtiles which I'm learning about myself now. Command and Control is one of the main uses. The interesting part is that it is made of self-calibrating display "bricks" you can build into any shaped display you want. Each brick houses a long-life DLP projector with very high resolution and color reproduction. There is only 1 mm between bricks, and it is easily reconfigured. From what I have heard, the bricks sense each other so you do not have a long calibration session, it is a low maintenance setup.
Natural Lighting: Tubular Skylights
Attached breakroom with a big screen with relaxing natural landscapes and playing soft music.
Refrigerator stocked with (healthy?) snacks and beverages.
These things will help improve the mood of your staff, and lower stress. They are cheap, easy ways to improve performance.