Slashdot Mirror
Design and Evaluation of Central Control Room Operations
brothke writes "In aviation today, technically advanced airplanes present a unique paradox. Technically advanced airplanes, in theory, have more available safety, and the outcome should be that there are fewer accidents. But without proper training for their pilots, they could be less safe than airplanes with less available safety. The FAA found that without proper training for the pilots who fly them, technically advanced airplanes don't advance safety at all. The reason is that technically advanced airplanes present challenges that under-prepared pilots might not be equipped to handle." Read on for the rest of the review.
Human Factors in the Design and Evaluation of Central Control Room Operations
author
Neville Stanton, Paul Salmon, Daniel Jenkins, Guy Walker
pages
446
publisher
CRC Press
rating
10/10
reviewer
Ben Rothke
ISBN
978-1439809914
summary
Invaluable reference that can be used for the design, assessment, evaluation an operations of NOCs and SOCs
In the IT world, staff members are often expected to install, configure, maintain and support technically advanced software. Companies often buy huge infrastructure software, such as CRM, ERP, PKI, identify management, intrusion detection and more, without first understanding how to make them work in their complex environment. Management often is oblivious to the fact that just because they can buy and install the software that it will not work on its own. The reason why so many large software deployments fail miserably is that the IT staff often doesn't have the proper training, support and assistance that they need.
Human Factors in the Design and Evaluation of Central Control Room Operations is a fantastic book that shows what it takes to ensure support staff work and operate together, in a formal and efficient manner. The book integrates the topics of human factors and ergonomics to create an incredibly valuable tome. The book details the interactions between people and their working environment, and shows in depth how the work environment can and must be designed to reduce errors, improve performance, improve the quality of work, and increase the work satisfaction of the workers themselves.
While the book was written primarily for control room settings, it is relevant for those in IT if they have any involvement in remote support, security operation centers (SOC) and network operation centers (NOC).
While the book is of value to anyone involved in operation, those who will find the most value are those charged with the management and operations or large groups or operations. If they have management support to deploy the formal methods detailed in the book, they will find that they can create significantly higher levels of customer and end-user satisfaction.
The authors note that all SOC and NOC's have a common feature in that the people operating them are often remote from the processes that they are monitoring and controlling, and the operations function on a 24/7 basis. The many demands of remote and continuous operation place special considerations on the design of the SOC and NOC. The output of the book is that it can be used to effectively to design these operating centers.
The books presents a comprehensive and all-inclusive on the topic of human factors on the following 14 topics: competencies, training, procedures, communications, workload, automation, supervision, shift patterns, control room layout, SCADA interfaces, alarms, control room environment, human error, and safety culture. Each chapter includes extensive diagrams and flowcharts to show how the processes develop.
The book also provides a highly analytical approach to each topic. It details the required processes and procedures necessary to make each subject area work. The book is not only based on the four author's expertise; they quote heavily from other experts and their research.
Chapter 2 opens with the observation that the safe and efficient operation of operating centers and control rooms is dependent upon the competence of the operators working within them. It details how to create competence assessments to ensure that staff is capable of carrying out their tasks safely and efficiently by assessing their skills and knowledge. The authors stress that it is not acceptable for organizations to assume that their staff are competent based on only their exposure to training and experience. They suggest that organizations create a program to determine those competence levels.
Chapter 3 goes into detail about how to create effective training programs to ensure worker competence. The benefit of a trained worked is that they can yield higher productivity and provide better service. Well-trained workers often have better morale and produce less errors. The chapter details the importance of a training needs analysis to properly determine what needs to be in the curriculum.
Chapter 4 is on procedures and is particularly important to those working in a SOC or NOC. If consistent and repeatable procedures are created, staff can provide much a more effective and dependable levels of service. Even with the benefits of well crafted procedures, its development process is a complex one involving the identification of all of the tasks that require procedures, a judgment on the level of assistance required, identification of the type or format of procedure required, writing and reviewing the procedures, and obtaining approval for them.
The importance of procedures is underscored when the book notes research that 70% of accidents and incidents within the nuclear power companies occurred when workers failed to properly follow procedures. In the petrochemical industry, 27% of incidents were caused by situations for which there were inadequate or no procedures available.
The percentage of failed IT projects and large software rollout catastrophes is both staggering and appalling. No other sector but IT would tolerate such failures. A book like as Human Factors in the Design and Evaluation of Central Control Room Operations goes a long way to stop that. The book is a rare one in that it both provides all of the factors involved in the problem at hand, and then provides all of the details needed to obviate those problems.
Ben Rothke is the author of Computer Security: 20 Things Every Employee Should Know.
You can purchase Human Factors in the Design and Evaluation of Central Control Room Operations from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
Human Factors in the Design and Evaluation of Central Control Room Operations is a fantastic book that shows what it takes to ensure support staff work and operate together, in a formal and efficient manner. The book integrates the topics of human factors and ergonomics to create an incredibly valuable tome. The book details the interactions between people and their working environment, and shows in depth how the work environment can and must be designed to reduce errors, improve performance, improve the quality of work, and increase the work satisfaction of the workers themselves.
While the book was written primarily for control room settings, it is relevant for those in IT if they have any involvement in remote support, security operation centers (SOC) and network operation centers (NOC).
While the book is of value to anyone involved in operation, those who will find the most value are those charged with the management and operations or large groups or operations. If they have management support to deploy the formal methods detailed in the book, they will find that they can create significantly higher levels of customer and end-user satisfaction.
The authors note that all SOC and NOC's have a common feature in that the people operating them are often remote from the processes that they are monitoring and controlling, and the operations function on a 24/7 basis. The many demands of remote and continuous operation place special considerations on the design of the SOC and NOC. The output of the book is that it can be used to effectively to design these operating centers.
The books presents a comprehensive and all-inclusive on the topic of human factors on the following 14 topics: competencies, training, procedures, communications, workload, automation, supervision, shift patterns, control room layout, SCADA interfaces, alarms, control room environment, human error, and safety culture. Each chapter includes extensive diagrams and flowcharts to show how the processes develop.
The book also provides a highly analytical approach to each topic. It details the required processes and procedures necessary to make each subject area work. The book is not only based on the four author's expertise; they quote heavily from other experts and their research.
Chapter 2 opens with the observation that the safe and efficient operation of operating centers and control rooms is dependent upon the competence of the operators working within them. It details how to create competence assessments to ensure that staff is capable of carrying out their tasks safely and efficiently by assessing their skills and knowledge. The authors stress that it is not acceptable for organizations to assume that their staff are competent based on only their exposure to training and experience. They suggest that organizations create a program to determine those competence levels.
Chapter 3 goes into detail about how to create effective training programs to ensure worker competence. The benefit of a trained worked is that they can yield higher productivity and provide better service. Well-trained workers often have better morale and produce less errors. The chapter details the importance of a training needs analysis to properly determine what needs to be in the curriculum.
Chapter 4 is on procedures and is particularly important to those working in a SOC or NOC. If consistent and repeatable procedures are created, staff can provide much a more effective and dependable levels of service. Even with the benefits of well crafted procedures, its development process is a complex one involving the identification of all of the tasks that require procedures, a judgment on the level of assistance required, identification of the type or format of procedure required, writing and reviewing the procedures, and obtaining approval for them.
The importance of procedures is underscored when the book notes research that 70% of accidents and incidents within the nuclear power companies occurred when workers failed to properly follow procedures. In the petrochemical industry, 27% of incidents were caused by situations for which there were inadequate or no procedures available.
The percentage of failed IT projects and large software rollout catastrophes is both staggering and appalling. No other sector but IT would tolerate such failures. A book like as Human Factors in the Design and Evaluation of Central Control Room Operations goes a long way to stop that. The book is a rare one in that it both provides all of the factors involved in the problem at hand, and then provides all of the details needed to obviate those problems.
Ben Rothke is the author of Computer Security: 20 Things Every Employee Should Know.
You can purchase Human Factors in the Design and Evaluation of Central Control Room Operations from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
I'm not reading the review. If it's half as painful to read as the summary was, I think I'll save myself the agony.
Seriously, redundant much?
Fellow slashdotters - this is a book with tips for improving our own command centres!
One that hath name thou can not otter
"In today's world, technically advanced cars present a unique paradox. Technically advanced vehicles, in theory, have more available safety, and the outcome should be that there are less accidents. But without proper training for their drivers, they could be less safe than cars with less available safety. NHTSA found that without proper training for the drivers who drive them, technically advanced cars don't advance safety at all. The reason is that technically advanced vehicles present challenges that under-prepared drivers might not be equipped to handle."
This could explain some of the Toyota crashes. The drivers don't understand what they need to do to slow down and stop the car when the accelerator acts like it's stuck.
If I used a sig over again, would anyone notice?
My one and only car crash, due to my fault, is due to ABS. It was my first ABS car and I tried to pump the brake, and failed to stop before hitting the other car.
It was a minor fender bender, never mind any injury, but it burned it into my skull: slam the brake.
Fuck systemd. Fuck Redhat. Fuck Soylent, too. Wait, scratch the last one.
and it was all like 'beepbeepbeep', and I was like 'huh?' It devoured my airplane. And it was a really good airplane."
I designed a Nagios interface especially for Control Rooms. My program can be run on a large screen hanging on the wall and then display a list of problems. Of course has a nice web-interface for remote configuration ;-)
It is called CoffeeSaint.
www.vanheusden.com - home of Multitail, HTTPing, CoffeeSaint, EntropyBroker, rsstail, bsod, listener, nagcon, nagi
The authors note that all SOC and NOC's have a common feature in that the people operating them are often remote from the processes that they are monitoring and controlling, and the operations function on a 24/7 basis. The many demands of remote and continuous operation place special considerations on the design of the SOC and NOC. The output of the book is that it can be used to effectively to design these operating centers.
You need a big fridge, and a microwave.
Another common feature is at least of all NOCs I've seen is marketing wants the most stylish looking facility they can get, which is often/always completely at odds with the goals of an effective facility.
Common noc mistakes:
1) Everyone crammed in like sardines so "we can work together". Except that no one noticed that we don't work together. All it makes is a lot of noise and interference. No space to open a stack of manuals, closely related to no space for usable computer monitors (as opposed to the ones used only for show). Even worse, designers seem addicted to adding "static noise" masking generator, crappy elevator music, and/or a PA system for other departments blaring away. Thru careful work, its possible to include features to make it look like its ideal for cooperation, yet make actual cooperation impossible due to noise level etc.
2) Extraordinarily expensive big screen TVs / monitors / projectors on all the forward facing walls, that no one actually uses. Too small, too low res, no actual business purpose. This is a killer two ways, first of all its a huge capital expense that could have paid the salary for extra techs for years, which would have a measurable positive effect. The other way its a killer is you'll actually take people off productive work to "fix" the big screens so marketing is happy. Would anyone in the NOC have a problem doing their job if all the projector bulbs burned out? No, but marketing would freak out.
3) Second class citizen status. "Real" employees can have family pictures in their work area. The dogs of the "noc", not so much. This attitude flows thru the organization in many other ways, producing discontent. Promotion out of the noc becomes a goal, not to "advance" but just to get the hell out.
4) Constant over the shoulder monitoring. No matter if its marketing, or management, there seems an utterly desperate desire to perch over the NOC workers shoulders, either physically or virtually. A great employer-employee attitude if you are 17 years old and working at taco bell. Not a great attitude in a professional noc environment. There seems something inherent in all NOC management that makes them distrust their employees, that you generally don't see in most other departments. Kind of like having the ability to treat them as serfs inevitably makes it a requirement to treat them like serfs.
5) You know those 1970's "sunken livingrooms"? alive and well in the nocs of the world. How about the 1960s original star trek theatre in the round concept with a bridge in the middle? alive and well in the nocs of the world. Remember the set of "wargames" from 1983? Why can't a noc be designed that doesn't look like a throwback or parody? At least try something different, like a medieval dungeon or something?
"Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
The notion that technology saves the day without introducing new tasks, quirks, and procedures is naive thinking at its worst. The Office 2007 ribbon might be a fair example. People who memorized countless shortcut keys and menus for Office '97 through 2003 suddenly have to reacquaint themselves with the shortcut bar for n00bs, with no simply way to revert to the old setup. Fun times are had by all. Also, as noted above:
Management often is oblivious to the fact that just because they can buy and install the software; that it will work on its own
This tends to be responsible for about 90% of all tech woes for deployments I've ever encountered.
My late Grandfather used to tell me stories all the time of stuff that happened in the air traffic control tower at Charlotte-Douglas airport (he was the chief of the tower for 20 years). 99% of the time, human error is to blame, for stuff like military pilots trying to land at the wrong airport and not listening to comms to stuff like poor pronounciation over comms to pilots causing close calls. By the end of his career, he found that work ethic was more important than credentials, because laziness was the cause of a large majority of mishaps. It is difficult to train work ethic, and easy to impart knowledge.
Rewarding diligence and establishing a culture of consistency was their solution. I've seen it work in other industries over the years as well. But, when you think about it, isn't it just common sense to do it? Why do you need a book to explain that?
I bet you have an 8 player and a carberator too. As electronic fuel injection is for wimps.(/sarcastic)
Meredeces is one of the few companies selling cars with radar modified cruise control.
If you don't know what it is and what it does and how it works. When your car starts slowing down by itself oneday you can cause an accident. Just that feature changing a basic tool you use regularly requires training. If you are to stupid to understand why you shouldn't be laughing.
Merecedes was the firstcar company to ship air bags standard. If you want to see what will be in the next decades ever one elses vehicles look at Mercedes. I follow them not because I can afford one but because ford and nissian will soon be duplicating those features.
i thought once I was found, but it was only a dream.
Here's an example of just that - the new Moesk control center for Moscow's electric network.
Take a look at the pictures. This looks like a movie set for a Bond movie. The architects got completely out of control here.
Notice the suspended transparent bubble for top management. It looks like it retracts into the ceiling. The lower operator's platform has steeply slanted sides, no railings, and chairs with wheels. The huge room only has eight operator positions.
I'll bet that, within a year or two, the people who actually have to run the grid set up a "field control center" with about twenty people with PCs, cork boards on the walls, 2-way radios for talking to field crews, a conference/map table, and some printers. The real work will be done there. A few people will sit in the big room and answer questions for management.
... is quite involved and requires careful thought. Training and procedures are important, but the best UIs should make the next step(s) in a task obvious. A symptom of an overly complex, poorly thought out UI is the high level of training and checklists needed to identify the next step(s) or locate required data. Using the flight deck model, older airplanes needed a large amount of training (and a third crew member) because all of the instrumentation and controls were just mounted on a few panels, with no guidance as to which dials and knobs would require special attention under various different flight conditions. The modern flight deck makes use of flexible displays that remain quiet (dark) until they demand attention. Then, they are presented in a manner which suits the particular task at hand. Like an automated checklist.
It's not just an issue of ergonomics. Older aircraft, even up to the DC-10/747/L-1011 era, didn't have the automation to allow two-crew operations. Engines were mechanically controlled, requiring someone to monitor temperatures, pressures, oil levels, etc. and make adjustments to keep them happy and prevent them from exceeding parameters. Cabin pressurization required monitoring, as did the electrical system. Newer aircraft, by contrast, have things like FADECs (Full-Authority Digital Engine Controller), automated monitoring and load-shedding for the electrical side, automatic cabin-pressure controls, auto-tuning radios, etc. Taken together, these mean that the pilots don't have to spend as much time watching gauges and trying to make sure they don't exceed some critical parameter, or fiddling little knobs back and forth to keep a constant pressure differential, and can instead worry about flying and navigating.
To use a car analogy, imagine having to constantly monitor and adjust choke, mixture, and ignition timing while driving... wouldn't it be easier to have someone else doing that?
The meek may inherit the earth, but the strong shall take the stars.
I'll bet that, within a year or two, the people who actually have to run the grid set up a "field control center" with about twenty people with PCs, cork boards on the walls, 2-way radios for talking to field crews, a conference/map table, and some printers. The real work will be done there. A few people will sit in the big room and answer questions for management.
At a previous employer, for a Very Important Photoshoot for marketing, they hired college age models to staff our center in the pictures, apparently because the real personnel were far too unphotogenic. I believe the age of all the models added together still didn't reach the age of some of our old timers.
Kind of like how anytime you see a call center in marketing material, its always staffed by stereotypical beauty pageant white women, where in reality most (not all) call centers have been moved to prisons and 3rd world countries. I've often wondered what the prisoners and 3rd worlders think when they see those advertisements (other than the obvious, americans are idiots, etc)
"Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
Hey! The Cone of Silence was a required deliverable insisted on by management, okay?