Slashdot Mirror
Three Mile Island Memories
theodp writes "Thirty years after the partial nuclear core meltdown at Three Mile Island, Robert Cringely describes the terrible TMI user interface, blaming a confluence of bad design decisions — some made by Congress — for making the accident vastly worse. While computers could be used to monitor the reactor, US law prohibited using computers to directly control nuclear power plants — men would do that. So, when the (one) computer noticed a problem, it would set off audible and visual alarms, and send a problem description to a line printer. Simple, except the computer noticed 700 things wrong in the first few minutes of the TMI accident, causing the one audible alarm to ring continuously until it was shut off as useless. The one visual alarm blinked for days, indicating nothing useful. And the print queue was quickly flooded with 700 error reports followed by thousands of updates and corrections, making it almost instantly hours behind. The operators had to guess at what the problem was."
Never has the gravity of an accident (of any kind) been so exaggerated. Before or after.
"The agriculture ministry is not in charge of Gundam" - Japanese ministry official.
U.S. Navy reactor operators, the sort who served under Jimmy Carter in the 1950s, were selected primarily for their temperament. ... their Navy job--as at TMI--was to follow the manual. All knowledge was inside the book. So knowing the book was everything. Unfortunately knowing the book isn't the same as knowing the reactor. So knowing the book was everything. Unfortunately knowing the book isn't the same as knowing the reactor.
No. Just fucking no. There's a significant (and necessary) emphasis on following procedures and getting the books out for any planned change to the plant to make sure you're doing things right. But Cringely makes it sound like nuclear operators are just slightly trained mouth-breathers that only know how to look things up in the book and do what it tells them. I can't speak for the civilian training, but the Navy does NOT do things that way.
When something goes wrong, they depend on you having enough internalized knowledge about the plant, its controls, and its indicator systems to work out what's going on and (if necessary) do something about it. Once you've got stuff at least marginally under control, *then* you get the books out to check the applicable procedures to make sure you haven't forgotten something, and to figure out how to recover from whatever happened without causing any more problems.
The Navy puts a lot of effort put into making sure their operators know how and why things work the way they do. They would never have got to the 21st century with the track record they have if all they did was train people to look at the book.
[b.belong('us') for b in bases if b.owner() == 'you']
I don't blame the UI at all. I bleme the belief that the goal of an UI is to lower the required understanding (and thus salary) of the operators.
How the UI worked is irrelevant. Operators who understood what they were doing would have checked what needed to be checked, and taken the precautions the situation warranted, no matter what kind of warnings were lost because of a bad UI.
Alas, the way for an electric company CEO to get big bonuses isn't by spending more money on smart people, but cutting costs which makes the short-term investors happy. So they spend $50k on an idiot-proof interface, and hire an idiot. The problem is that Nature is a whole lot better of churning out interface-proof idiots than programmers are at making idiot-proof interfaces.
It's high time that Western society started valuing knowledge and understanding again, and not just ability to study for requirement tests. Reinstate the journeyman/master system and accredited guilds, and ditch college diplomas as the worthless piece of gilded paper they are.
If your user interface lags behind by two hours and the UI is the only way to find out about the extremely complicated and intricate details coming out of a myriad of sensors that are inaccessible to people for safety reasons... I suppose you might be entirely wrong.
In this case, yes, the user interface was necessary for the operators to do their job. Are you going to tell me that submarine operators should rely on their "gut feeling" rather than a measurement of external pressure or depth to determine whether the submarine is safe? These are jobs that can't be done by even the most skilled operator because the information is completely walled off from them for the safety and integrity of the facility.
As far as I can tell, you're advocating that we should hire psychics to determine the safety of the nuclear plant and pay them exorbitantly because spending a single dime on a good interface is wasted money. Sometimes, a $50,000 idiot proof interface is exactly what's called for, rather than intentionally using outdated technology and hoping a printer will provide information fast enough to prevent imminent disaster.
I wasn't there so I can't say Cringely is wrong about the government regulation of nuclear power, however, I have worked in the semiconductor industry which utilizes some of the deadliest chemicals known to man and their are mandated regulations from various government agencies, EPA, OSHA, etc., that result in the controls, interlocks, and containment systems used to make the industry safe. I'm also pretty sure that the issue in Bhopal was more a lack of regulation than a lack of respect for the dangers. There should have been powerful laws and inspectors to shut down the plant before it killed thousands.
Where we both do agree is on the belief that we can expect more Bhopal and economic melt down events due to bean counter management. Over the past 20 years I've noticed a managerial shift towards a focus on cutting costs and less of a focus on the technology and science behind the manufactured products. In the past two years I've engaged in heated debates with peers and managers over the purpose and focus of engineering resources. Its seems that decision makers are forgetting that the core of a technology based manufacturing corporation is the technology not the cutting of fixed costs by reducing head count, wages, service contracts, etc. Accounting and business management are tools to support the core skills, they are not the core themselves. When accounting and business management undermines the ability of a technology based business to develop and manufacture the core technology of their business you can expect a gradual degradation of the business until it is no longer viable.
Comment removed based on user account deletion
I think you got it backwards. They didn't want to withold information from humans or remove control from them, so they didn't automate enough and the humans in the loop got swamped with more than they could handle.
Sounds like it was engineered just right. Bean-counters often use "over-engineered" when something is built to withstand the rare but serious malfunctions. Instead, they'd rather things be built to be "good enough" to run fine most of the time. Problem is, a minor issue can become a critical one if you don't build your devices to withstand the rare but serious issues. For example, a failover server setup is 100% overbuilt...until the primary fails.
But it wasn't engineered this way to secure it against a partial meltdown. It was above average for reactor containment vessels actually in use at that time, and the average containment vessel would have failed. The only reason it was able to withstand it was that it happened to be on the final approach path of a former airforce base, and had originally been engineered to withstand a bomber crashing into it.
God, I wish I had mod points for you.
... how much nuclear power is involved with Centralia? Ummmm.... NONE! A natural resource (accidentally ignited by humans) has destroyed a town completely. Personally, I put Centralia on a higher level of "disaster" than I do TMI.
I live about 15 miles away from TMI and I have for 20 years. I've never felt unsafe or felt like I was in danger. People seems to enjoy comparing TMI to being a potential Chernobyl, but there's simply no way that the two can even be compared.
On the other hand, head up to Centralia, PA where the whole town has been demolished because of a fire that has been running through the ignition of a natural, coal vein. A fire ignited some coal, and now the whole town has been abandoned, homes have been razed, there are very few buildings to speak of, there are dangerous leaks of carbon monoxide and other lethal gases, the ground has swelled and cracked from the heat, and this fire is expected to last 250 years.
Now
The Overrated mod is for reversing inappropriate, positive mods, not for voicing disagreement with a post.
See, See. UI is important!!!!
I'm a nuclear engineer and I think the use of the term UI for the control room is somewhat 'simplistic'. I personally think a major issue was over design in a certain area (redundant alarms), and lack of safety systems that would prevent the core from melting even with a LOCA in place. It was two hours after the shutdown when the fuel melting began at TMI-2. This was a scenario where the operators couldn't understand what was happening. Now from an operator's perspective (who sits in the operator room) you're not looking at a "UI" in the traditional CS sense. Here is an image of a control room: http://www.ornl.gov/info/ornlreview/v38_1_05/images/a11_controls_full.jpg The events leading up to the disaster started on the secondary side (non-core) leading to a LOCA (Loss of Coolant Accident). For those unfamiliar with the term "secondary side". The secondary side of a Nuclear Power Plant is similar to that of any power generating plant, meaning the secondary side does not contain the reactor core.
Right. If I need a nuclear reactor managed, I'll call you. Good to know the old talent of understanding exactly what the state of a nuclear reactor is by looking at a rock isn't lost. I'm just going to go and plug myself into my other computer now and manipulate it with my mind. Screens and command lines are for pussies, I can feel what it's doing well enough.
Admit it. You post strawman arguments as AC so you get modded Insightful for refuting them, rather than Troll
I used to work in the nuclear power plant operator training industry. Believe me, whatever else those operators were, they were not cheap. The CEO could not skimp on salaries and hire idiots. In fact, in a time when $40K was an excellent salary, the training costs per operator was more than $1 million.
On the other hand, there were cultural obstacles. In Europe (Sweden), they hired engineers with masters degrees to become nuclear plant operators. In the USA, they were mostly high school grads who were union members and promoted from running older coal plants. Union politics, not merit decided who got promoted. They were not the best and brightest. Of course in Sweden they also attract the best and brightest to be civil servants. Can you imagine that happening here?
There are always plenty of suggestions as to where society should apply its best and brightest. It is much harder to place the worst and dumbest. Consider the bottom 25%. They have to have jobs. No matter where you assign them, the public will in some way be depending on those jobs being done well. So filling jobs becomes less of a question of rational allocation of resources, but more a matter of attractiveness and recruiting.
A plant operator must stand there and do nothing but monitor year after year, yet react swiftly and accurately in those rare seconds of pure terror, and then have the whole world second guess how well they did it. In addition, they have to do shift work for 24x7 operation. Most people think that it is a hell of an unattractive job. I think that the plant owners do a hell of a job trying to find and retain the best people they can get, and to enrich the jobs to make them less boring. It takes much more than deep pockets to succeed.
So you tell me. You play CEO and tell me how would you convince Google engineers to quit Google and become operators, and how many of the lower quartiles you would assign to invent Google. Convince those bright college students that they don't want to be environmental scientists, but nuclear power plant operators instead.
This. Most of the US civilian nuclear power industry is, to say the least, heavily influenced by the military nuclear power industry and the cult of personality surrounding Admiral Rickover. If nobody is in control, nobody can be held accountable when the fan hits the shit.
Er, in what way is that "nobody is accountable" attitude reminiscent of the nuclear Navy? They're obsessive when it comes to accountability. Every time I saw any fecal matter hit a rotary device, they were pretty damn rigorous about getting to the bottom of it and finding out who did what.
[b.belong('us') for b in bases if b.owner() == 'you']
tmi2> sshutdown -r now
sshutdown: Command not found.
tmi2> halt -c
halt: invalid option: -c
Try `halt --help' for more information.
tmi2> help halt
help: Command not found.
tmi2> shut it down, damnit!
shut: demand not found.
tmi2> assume nuclear defense position
assume: Command not found.
tmi2> stick your head between your legs
stick: command not found.
tmi2> *%&*^&$
[from system: system going down for meltdown NOW!]
[from system: assume nuclear defense positon]
[from system: stick your head between your legs and kiss your ]
***line down***
Thats just wrong. For something as dangerous and deadly as a nuclear reactor, you practically want a monkey to be able to figure out what they need to do.
You DO NOT require someone with a PHD to make the plant safe. You practically want the plant to be idiot proof and scram at the first blush of trouble.
By making it require (rare) operators that understand the plant as a systemic whole, you make them irreplaceable, and from a design for long long long term safety point of view thats just wrong. Over time, understanding of large complex systems at plants degrades, and with a plant lifetime of 20-50 years you will see whole generations change in the lifetime of the plant.
Chuck