Slashdot Mirror
Failure Is Always an Option
Logic Bomb writes "The New York Times has a short but elegant op-ed regarding the different perspectives of engineers and managers and the role that plays in accidents like the space shuttle Columbia disaster. It's the sort of article you'll nod all the way through, then print and leave anonymously on your supervisor's desk. Any tech managers in the Slashdot crowd might have some interesting comments on how the right balance is struck." Henry Petroski has written several good books on engineering and failure.
In the case of my last software project, engineers who worried about bugs that the software may have suffered during design were ineffective in getting it properly inspected before launch.
"No boss, I have no idea where that article printed out 15 times and strewn across your office came from........ It looks like a good article though."
"I can not bring myself to believe that if knowledge presents danger, the solution is ignorance" - Isaac Asimov
By Homer Hickam
When I go to the Cape and watch the Shuttle being launched, I still get a lump in my throat watching it soar. Even though I no longer work for NASA, its thunder affirms my dreams for spaceflight. Still, when I put emotion aside, I can't ignore my engineering training. That training and my knowledge as a 20-year veteran of the space agency (and also a Vietnam vet) has led me to conclude that the Space Shuttle is NASA's Vietnam. A generation of engineers and managers have exhausted themselves trying to make it work and they just can't. Why not? Because the Shuttle's engineering design, just as Vietnam's political design, is inherently flawed.
Much has been made of the report produced by the Columbia Accident Investigation Board (CAIB). I've read newspaper articles that called it "scathing." Hardly. Its tepid recommendations probably had Shuttle managers who made poor decisions dancing with relief. It gave them a pass by proclaiming "culture" made them do it.
I don't believe there's a NASA culture. There is, however, a Shuttle cult. It is practiced like a religion by space policy makers who simply cannot imagine an American space agency without the Shuttle. Well, I can, and it's a space agency which can actually fly people and cargoes into orbit without everybody involved being terrified of imminent destruction every time there's lift-off. With some reservations, written in the politest language, the CAIB recommended to keep Shuttles flying but with more inspections, more bureaucracy (an outside safety agency), and more money. But piling on more inspections, people and dollars won't make the Shuttle safer. Neither will the safety sensitivity training that will probably be dumped on top of the overworked, disillusioned NASA engineers. My God, they've already dedicated their very souls to keep the Shuttle flying safely! The truth is, no amount of arm-waving about "culture" can fix a flawed design.
Take a look at the Shuttle stack and what do you see? A fragile spaceplane sitting on the back of a huge propellant tank between two massive solid rocket boosters. The Shuttle has to sit right in the middle of all the turmoil of launch because we once believed it would be cheaper to bring back those engines and rebuild them than to build new ones. That has not proved to be the case -- far from it -- but it has left us with a crew sitting in the most vulnerable position possible in terms of design. Simply put, had that spaceplane been on top of the stack, the destruction of Challenger and Columbia wouldn't have occurred. The CAIB ignored this flawed design and that makes their conclusions suspect: no amount of inspections or condemning another NASA generation to worry over this thing will solve it.
So let's get practical. We can't just shut the thing down. We need the Shuttle to finish the space station and also to keep the Russians and Chinese from dominating space. I'm not willing to see that occur while we dither. Human spaceflight is important to this country. But the Shuttle is as safe as you're going to get with what's in place today. Let's put some tough engineers in charge, fly it 10 more times over the next four years with hand-picked crews to finish the space station and meet our international obligations. Then close the program and replace it with expendable launchers and a shiny new spaceplane. And, this time, put it on top.
The problem is that people are afraid that if the shuttle stops flying space exploration will stop. Public support will wane and funcing will slow. I happen to disagree but there are many in the space program who do not.
"I can not bring myself to believe that if knowledge presents danger, the solution is ignorance" - Isaac Asimov
I have to admit up front that I am biased against NASA on primarily ethical grounds. To me, there's one basic valid purpose of government, and that's to defend the individual rights of its citizens. In the U.S., this is the principle upon which the Constitution and Bill of Rights is based, and the primary ligitimate activities of government, the police, courts, and defense, are inferrable from that.
Everything has an opportunity cost. The money spent on NASA could otherwise be spent elsewhere, such as aiding the homeless or better road infrastructure, and preferably on something the person earning the money (the taxpayer) himself chose.
Sure, it's nice to be able to explore space and determine facts about physics and cosmology, and test theories against empirical information, but I think at some point the costs associated with expanding the realm of science to more obscure areas, in the shorter term, are too high. And, yes, I know the argument that expanding basic science can lead to invention that benefits the individual, but personally I'd put more faith in the ability of industry to use the money making targeted investments while hiring scientists, than effective production from NASA. At some point I think we have to say the money can be better spent than knowing more about the behavior of some unreachable binary star. Eventually, that information will likely come anyway, as a function of better theoretical models. Why do we need it now, assuming it isn't primarily to give a Ph.D. something to play with?
NASA exists in an enviroment that offers none of the efficiency advantages of modern industry.
- No effective competition
- No way to inexpensively prototype or proof-of-concept things and test them in the intended deployment environment
- Few efficiencies of scale from being able to buy parts widely used and commoditized
- Little economic justification for the expense, even in the instances where the mission is "successful"
- No realistic, market-driven benchmarks for the performance of the managers or engineers
In the end, I don't feel that NASA is an optimal way to spend money, and since it's at least in part my money, I should be able to make this decision. Perhaps some kind of opt-in "NASA" checkbox, like I've seen opt-in "environmental" checkboxes on tax forms. I'd be content with that.
~ Whence do you come, slayer of men, or where are you going, conqueror of space?
I don't know who said it, or if I have the quote right but I'll attribute it to a 1950's era Ford Engineer:
"The most effective safety device for a car would be a 6 inch metal spike, attached to the steering wheel, pointed at the driver's chest."
Surely that would be cheaper than today's airbags, and I agree that I'd be inclined to drive a little more cautiously... Safety doesn't have to cost money, but it will cost something. In this case It would take me much longer to cross the city in my car, traveling at about 40km/h...
www.jmagar.com
-
Did you read the Investigation report?
Hindsight is 20/20, but that doesn't mean that we should wear blinders when looking towards the future!
The Management team _actively_ canceled requests for information pertaining to the impact. See page 153 of the PDF.
The management team also didn't follow their own procedures, they didn't meet every day (they were supposed to).
I was impressed by the engineers at Boeing (I think that was the company) who elected to research the impact and footage of it over the weekend even when management told them not to.
Read the report. Section 6.3 (DECISION-MAKING DURING THE FLIGHT OF STS-107) is extremely interesting and points out Eight seperate missed opportunities to find out more information about the problem.
There were also some engineering related issues - the engineers using test software that wasn't designed to analize an impact nearly that large, and other issues - but it really comes down to a lack of the management team accepting that there could be a real, out-of-family problem on the mission.
Sticks and Stones may break my bones, but copyright will always protect me.
It's worth thinking about what would have happened if the damaged Shuttle had been images by USAF ground cameras, and it became clear that re-entry was going to be a disaster. The shuttle and crew would have been stuck in orbit, with worldwide publicity, while NASA tried to come up with a fix. They probably wouldn't have succeeded. On-orbit rescue using Atlantis has been discussed as marginally possible, and on-orbit patching has been suggested, but most likely, they wouldn't have worked.
Think of the PR fallout. Seven astronauts stuck in orbit for most of a month, with constant TV coverage, followed by their deaths on worldwide TV. That would have been career-ending for most of NASA's top management. Letting them crash saved the jobs of top people at NASA.
Worst case, a rushed launch of Atlantis could have resulted in losing two shuttles. That would have ended the Shuttle program.
Henry Petroski wrote
>If engineers are pessimists, managers are optimists about technology.
Is this the difference between programmers and engineers?
Fred Brooks, in The Mythical Man-Month (go read it!) argues that programmers are optimists. We work with pure thought-stuff, so of course it should work the way we think it will. Bzzt. But that optimism drives projects. Who'd start a big project knowing how many stomach-churning bugs, random external changes, stupid feature requests, irrelevant but deadly external bugs, dependencies and just plain stapler misfires would come up? How many projects, open or closed source, would have started if the actual development timeline had been known in advance?
This reminds me of a decent book I read about a year ago, called To Engineer is Human . It discusses the role engineering failures play in our many engineering successes.
Interesting read, though the author tends to drone on and on a bit. He makes some great points, though, not the lest of which is that (gasp!) engineers are not perfect, and thus, failures will happen. And guess what--most of the time, we learn from those failures!
--- Standard disclaimer applies.
Hickam is on track, but I'm not sure we need spacecraft with wings. Wings are only useful on airplanes. By definition, spacecraft are not airplanes. NASA has thrown away too much money pursuing winged spacecraft for their own sake, rather than dealing with the issue of getting people to and from space. They might as well try to make a submarine that can fly. Probably do-able, but: why?
Let's decide that we will do two things:
1) Any human space travel beyond LEO will start from LEO in spacecraft built in LEO and that return to LEO. If we do that, we will never need to spend money trying to build airplane-spacecraft hybrids.
2) Let's use big expendable boosters to get hardware to LEO, and smaller expendable boosters to get people to LEO. Put the people in modern versions of the Apollo or Gemini craft (the so-called "Big" Gemini was an appropos solution)>
And, let's also decide that the main reason to build a space station in LEO is to serve as a construction yard and a gas station for trips elsewhere. Let's put aside the quaint notion that the reason we need to be in space is to "do science".
-- Slashdot: When Public Access TV Says "No"
I've been involved in engineering literally all my life. My dad was an engineer and as a small child I remember going to work with my dad and being in awe of all the stuff he had to 'play' with. I never wanted to be anything else! Unfortunately, in the scheme of things we are the workers, the ones who toil withput credit. The managers take all of that. In the 1980's as a contract engineer I built a Boston FM radio station from scratch (WFNX), yet they didn't even see fit to invite me to its sign on party! When I asked why, I was told: "You were paid well for your work, isn't that enough?". They actually believed they paid me too much to make their property worth many millions morethan it was before. Needless to say from that time forward, I did only precicely what they paid me to do (and what they asked me to do), nothing more. Part of the problem is we ALLOW ourselves to be treated in this way! The plumber, electrician or auto mechanic don't. Why do we? I think one answer is UNION. They realize there is respect and safety in numbers. Are we too good, too elite to do the same?
fair enough, point taken. :)
i personally have a bit of a different philosophy. to me, a focus on success presupposes a focus on points of failure.
i can develop an algorithm to process foo and focus intently on getting all the little details just right (focus on the success of the algorithm). but if i fail to perform appropriate checking at critical points, then that algorithm may very well be useless in anything but an academic environment.
focusing on failure begets failure, sure. but my point was that a focus on failure *analysis* is very much a vital part of the design process, at least as much so as a focus on successful implementation.
good luck with your studies, and don't forget to analyze your points of failure
Can we? Good question...
A couple maybe-relevant personal opinions:
1. As pointed out by others in this thread, basic space research has had a bunch of other benefits in other industries. From a body politic perspective, I'd say we've benefitted overall from it, so it seems a net-beneficial exercise, at least to me.
2. NASA does a horrible job qualifying the "why" of these programs. I think they need to point out benefits other than space just being a Cool Place To Explore.
3. If you're saying that NASA needs a little more oversight and thought in deciding which programs to fund and how to manage them, I absolutely agree with you.
Pundits have claimed that the parallel launch configuration of the orbiter and external tank are a design flaw. Hog wash. The size of the orbiter precludes an inline configuration. If you want to fly "spam in a can" on top of a larger rockets then welcome back to 1960!
Now that a major risk of known to be debris strike, to avert it why not have the shuttle ascend right side up? The shuttle currently flies upside down for two (lousy) historical reasons: to simplify the manuver for RTLS abort, and a for line of sight radio link with the ground and antennae on the nose of the orbiter. There are no groundstations down range anymore. RTLS will not be made any more insanely risky than it already is by having the orbiter stack roll 180 degrees to an RTLS attitude. The shuttle already rolls to heads up after 5 minutes of flight in order to acquire the TDRS satellite for tracking and communications. Doing so in flight through the lower atmosphere should have added benefits. Tank debris will tend to fall away from the orbiter instead of into it. The lift provided by the orbiter wings should improve performance.
an ill wind that blows no good
At this point the X-Prize gives me more hope than NASA.
I knew one nice lady (a rarity in both respects) whose entire job consisted of going from meeting to meeting carrying a stack of paper at least a foot high. And this was a project that was about 1000 times smaller than the shuttle project.
Here are two articles (part 1 and part 2) about the history of flying submarines. Great stuff. It's in Russian, so you will need to use the fish or just check out the photos.
Future Wiki -- If you don't think about the future, you cannot have one.
When engineers and managers clashed over the 1986 Challenger launch, the managers pulled rank.
What a dark, yet utterly true statement. Do the NASA and contracting company managers sleep well today knowing that in 1986 their decisions cost lives?
Edward Tufte, author of some amazing books on information display, wrote in Envisioning Information on the Challenger disaster. Looking at the materials prepared by engineers, he saw that they had correctly correlated temperature with O-ring failure. Yet their materials, hastily prepared during the 11th hour, failed to convince managers to abort the launch. Tufte shows a design of a simple graph that shows temperature on the abscissa and burn-through on the ordinate, and any manager could draw a line through the points and extrapolate out to the bitter cold Florida day that cost the shuttle.
Having my own share of bad managers, I have to wonder, would it have made any difference?
That's the major flaw with government underwriting a space program. You have to get public support for it. Let private enterprise underwrite it, and all you need is commercial interest. That's a MUCH easier beast to summon.
You can tell a great deal about the character of a man by observing those who hate him.
Oh, because you've forgotten the basic truisms:
Private sector ALWAYS GOOD.
Government sectore ALWAYS BAD.
I've tried to say it here and other places before, with mixed reactions:
The government has no monopoly on stupidity. The success of the cartoon strip, "Dilbert" proves that business has its share - across the board.
I used to imagine that Scott Adams worked for my employer, but when he revealed that he worked for Pac Bell, I decided that the stupidity he writes of was universal to the high-tech industry. Then I found that my sister-in-law, who works in a doctors' office, finds it "meaningful," and others in non-high-tech say the same. Stupidity appears universal.
(No doubt some of you will apply it to this post, as well.)
The living have better things to do than to continue hating the dead.
I think that the author was really off the mark.
First, the engineer/scientist comparison is incomplete. There is a third category, the inventor. He can often be one of the two, often he is all three.
The engineer leverages science to build useful creations. The scientist researchers the way the universe works, he often cares nothing for invention, only knowledge.
The inventor really doesn't CARE about science OR engineering. He just wants something that works and is happy when it does. If it fails, he will invent something better. He'll use science and engineering if it furthers his goals.
The beueracrat is of course the forth factor that tries to get engineers, scientists and inventors to serve some other goal. Sometimes the public well-being, sometimes his own. Most often he serves his bosses well being in pursuit of his own which may or may not correspond to the well being of an organization (like the public interest).
Part of serving your bosses best interest is not making him look bad. When you ground your project, your project looks bad irregardless of whether it's the right thing to do. It causes the schedule to slip, and somewhere up the line the big boss is staking his reputation on it. Thats how you get to be the big boss, making promises and coming through.
The truth is that failure is a part of success. Risk is a fundamental part of achievement and risk will ALWAYS produce failures at some point.
I am disspointed at the nature of Columbia's failure. However, in such a game as space travel, risk is an incredible factor. Despite an incredible effort to systematically mitigate risk, you will have failures.
Whether it's from the managements perspective or the engineers, failure will inevitably occur. The prime risk for the managers is that NOTHING would get done if they did EVERYTHING the engineers wanted to. The perfect system isn't created, it evolves. And evolution NEEDS failure to point out mistakes.
In this case, the managers were wrong. Their stonewalling and mindless dedication to schedule produced the death of a crew and the loss of a multi-billion dollar vehicle. In some other case, it could be an engineer who used the wrong unit system or an engineer that pendantically freeted over an issue that ultimately wasn't that important.
The lesson is to seek balance. And of course, even when you have balance you will have failures. Unfortunatly, for NASA, their failures are always VERY unforgiving.
-------- -------- Support Wesley Clark for president!!!