Slashdot Mirror


Failure Is Always an Option

Logic Bomb writes "The New York Times has a short but elegant op-ed regarding the different perspectives of engineers and managers and the role that plays in accidents like the space shuttle Columbia disaster. It's the sort of article you'll nod all the way through, then print and leave anonymously on your supervisor's desk. Any tech managers in the Slashdot crowd might have some interesting comments on how the right balance is struck." Henry Petroski has written several good books on engineering and failure.

18 of 479 comments (clear)

  1. Fail? by Matrix272 · · Score: 5, Insightful

    Was it Thomas Edison that said, "I haven't failed. I just found 10,000 ways that didn't work."?

    --
    "It's better to have a gun and not need it than need a gun and not have it." ~ Christian Slater, True Romance
    1. Re:Fail? by gujo-odori · · Score: 5, Insightful

      Say what?!

      If engineers find problems in a project, you think the answer is to "innovate our management skills?" Is that really even English?

      Don't look now, but Dilbert and The Way of The Weasel is making fun of PHBs, it's not a management blueprint? Well, actually, it is a rather good guide to managing successfully, but the key is to read what's in the book and *not do* those things.

      If engineers on a project, whether it's hardware, software, or something else, come to management and say "We have found problems with this project that will have a negative impact on its quality or possibly cause it to fail" the answer is not to sweep those concerns under the rug or blow them off. The answer is "OK, what do you need to fix those problems so that this project will succeed and reach its full potential?" When they tell you, obviously, there may be a cost/benefit tradeoff between some of the items, but basically you have to send them out to fix the problems so that the project will succeed.

      If engineers tell you "You can't do X for Y amount of money, it's just not possible," you should listen to them. Knowing what can be done, and for what price it can be done, is their job.

      If the engineering team comes to you and says "This project is so broken that it can't succeed, the best thing we can do is scrap it and do a total redesign," then you had better listen good. They are probably right, and the ass they save will be your own. The money sunk into the project is gone; don't make it worse by throwing good money after bad.

      Being committed to quality and excellence in a project are not "old thinking and old views that hold us back." They are the things that make projects successful. That's my company has a successful product, is growing fast, and is making money. Is yours?

      Aside to those who modded the parent Insightful: I never believed it before, but now I'm convinced that (some of) the mods really are on crack.

    2. Re:Fail? by SillySlashdotName · · Score: 4, Insightful

      Why isn't there a moderation "disturbingly sick, but funny"?

      --
      Acts of massive stupidity are almost never covered by warranty. --me.
  2. Safety always has a price by shoppa · · Score: 5, Insightful
    The NY Times editorial has a good perspective in the manager vs engineer battle, but in the end we will never have a pefectly safe mode of travel (on or off earth) because Safety Costs Money.

    Now that money may be in the form of lower gas mileage in a car, or in the form of hundreds of unmanned test flights before putting a human in, or obscene safety margins.

    But to pretend that anything is ever perfectly safe is to ignore the fundamental economic issue that at some point you have to stop putting money into safety concerns and just fly the damn thing.

  3. This is annoying. by Prince_Ali · · Score: 5, Insightful

    On a project the size of the space shuttle thousands of safety concerns will be brought up. Not everyone of them can be fully investigated. They have to pick and choose based on what is most urgent. Yes, there will be accidents, but otherwise the shuttle would never get off the ground. Hindsight is twenty-twenty, and you can say they should have investigated further all you want, but the fact is that there were many other concerns that seemed just as urgent, and some that seemed even moreso.

  4. No Money == Failure by Matrix272 · · Score: 4, Insightful

    After you make such significant strides in space exploration in the late 60's and early 70's, then have your funding cut by almost every President since Nixon, you're bound to start taking short-cuts and missing things. Remember... space is still deadly. In my book, when you're dealing with something that could very easily kill you, you don't short-change yourself. The problem is that when you have no money to spend on things you need, and a time limit to do certain things, you don't have any other choice.

    The problem NASA has right now is trying to convince the rest of the country that what they're trying to do is worth spending the money on. Why worry about what Saddam can do if we could all just move to Mars (for instance)? On the other hand, funding was cut because nothing significant was happening... but nothing significant was happening because funding was cut. It's a vicious cycle.

    --
    "It's better to have a gun and not need it than need a gun and not have it." ~ Christian Slater, True Romance
  5. You still can't prove a negative. by rdewald · · Score: 5, Insightful

    I have spent the last few days reading the entire CAIB report and I have to agree that Mr. Petroski is right on target with his observations.

    Simply put, the problem was that the engineers concerned with the safe re-entry of the orbiter after the foam strike were put in the position of having to prove a negative. Management wouldn't pay attention to them until they could prove that the strike was *not* safe.

    They couldn't prove or disprove the notion that the foam strike had caused critical damage until they got the images, but they couldn't get the images without first proving they needed them to assure the safety of the re-entry.

    There had been a number of previous foam strikes, many of them involving this same piece of foam (the left bipod ramp), and all of those shuttles had landed okay, so management believed that this foam strike was similarly okay just because they had gotten away with it so far.

    No science. No analysis. Just an assumption that if they had gotten away with ignoring this problem so far, they could continue to ignore it. The schedule was king, not safety.

    Engineers know well that "getting away with it" is not evidence of reliability. Managers, at least in my experience, tend to be proportionately successful in their careers to the extent that they can spin "getting away with it" into a career advancement tool.

    This is really why the orbiter was lost. This is really why the astronauts died.

    Denial is deadly.

    --
    The best way to do is to be.
    1. Re:You still can't prove a negative. by ckd · · Score: 4, Insightful
      There had been a number of previous foam strikes, many of them involving this same piece of foam (the left bipod ramp), and all of those shuttles had landed okay, so management believed that this foam strike was similarly okay just because they had gotten away with it so far.

      Yeah, sounds familiar. "We've had O-ring erosion due to low temperatures before, but it's never caused a real problem, so we can launch." IOW, they learned nothing from Challenger.

  6. Re:NASA's Vietnam (From today's Wall Street Journa by Guano_Jim · · Score: 5, Insightful

    This is the same Homer Hickham about whom October Sky was made, I'm assuming?

    It would be nice if more people listened to engineers instead of politicians when it came to science projects, wouldn't it?

  7. Old story same sad ending by Crashmarik · · Score: 5, Insightful

    This is always the case it has been for a very a long time. The problem is not NASA's culture so much as the culture of the society around NASA.

    The article Misses the big points. When the Challenger blew up blame was apportioned to the engineers that built it not the congressmen who insisted the engines be built in utah. When software is shipped before its ready, blame goes to the programmers that were working 90 hour weeks not the sales people that promised the customer whatever they wanted to hear. When a heartvalve fails blame goes to the inventors that made a device that saved lives, not the insurance companies that wouldnt pay for a proper solution.

    Yes managers are willing to take risks, its rare they ever have to pay the price for failure.

  8. Unfortunately, many quality procedures go awry... by TimTheFoolMan · · Score: 4, Insightful

    Sadly, many higher-ups see the solution in CMM, or other quality programs that produce reams of paper, but those same top-level managers ignore the economics of trying to develop too much, in too little time, with too little money. I manage the development of custom software projects for a Fortune 100 company, and at the end of the day, the sales dweeb sells whatever he has to to make his commission, and the engineering group is left with impossible constraints. CMM would probably work well if the entire company bought into it, but I've not seen that yet.

    Likewise, NASA sees us (the public) crying about cost overruns and the return on our investment. Ultimately, that comes back down to the line-level managers at NASA, where no matter what the good intentions, the pressures of $$$ and time will always apply.

    We need to decide if space travel is worth the cost (done properly, and left to engineering minds to decide what "properly" means), or worth the risks of doing it at lower cost. Like my company, NASA has squeaked by on luck for quite some time.

    In my experience, the luck ALWAYS runs out.

    Tim

  9. Re:Management's decision not to image by kin_korn_karn · · Score: 4, Insightful

    I think you hit the nail on the head.
    Managers at that level never do anything because they think it's right, they only do what will cover their asses, and they have no conscience about it whatsoever.

  10. Pragmatic vs. perfect safety by coyote-san · · Score: 5, Insightful

    NASA isn't getting criticized because it doesn't have perfect safety, it's getting nailed because it has TWICE ignored clear evidence of significant problems and failed to perform even cursory investigations until after the loss of an orbiter and crew.

    There was clear evidence of problems with the O-rings before the Challenger was lost. NASA had somebody produce some really cryptic plots, but nobody bothered to really investigate whether the cooler weather on some of these launches might have an influence. It takes a real genius to reduce this to dipping an o-ring into a glass of ice water, but any competent investigator should have been able to reduce the data to plots of damage vs. various independent variables such as temperature at launch or overnight lows.

    With Columbia, the arrogance of management is far more stunning. It KNEW that the insulation had flaked off, it KNEW that the insulation had caused surface damage in the past, and it KNEW that some areas on the leading edge of the wing are much more vulnerable to damage than others because of access points. It could have test fired foam at wing mockups at any time, just to have hard proof instead of just hunches that the foam could never cause significant damage to an orbiter... yet it did nothing.

    This testing is expensive, of course, but it's really not that much when compared to the cost of a normal launch (isn't that approaching a billion dollars per launch now?), or the various costs associated with the loss of an orbiter and crew. It's akin to failing to spend $10 to check something on your car even though you knew that a mistake would mean that the car would erupt into a fireball and kill everyone inside if you're wrong.

    --
    For every complex problem there is an answer that is clear, simple, and wrong. -- H L Mencken
  11. Re:Management's decision not to image by jeffy124 · · Score: 5, Insightful

    No, I beg to differ.

    Assume NASA did attempt to evaluate the damage and it revealed the Columbia to be a death trap. Yeah, there will be media coverage had it become necessary to send up a repair crew or something.

    But there would be an Apollo 13 type effort. Atlantis could go up with a minimal crew and pick up the Columbia crew. Maybe do it in two flights. Leave the Columbia in space until repair becomes possible. Not possible? They'd find a way.

    Or, engineer a solution on the ground and figure out a way to get that solution up into space and istalled. Again, an Atlantis crew would head up with the necessary materials and perhaps be the ones to do the repair job. Sounds like the Hubble, doesnt it? Also impossible? They'd find a way.

    Engineers are quite capable of great things, and you seem to be underestimating the potential of great thinkers. When JFK made his "before this decade is out" challenge, everyone at NASA thought "No way! You've got to be kidding." But then the people who would do it got thinking of ways they could and they came through.

    --
    The One Rule Of Chess You'll Ever Need: Don't play someone who carries a kit in their bookbag.
  12. Shuttle == Marketing by phliar · · Score: 4, Insightful
    I think the real disservice NASA managers have done is convincing this country that the Shuttle is just like an airline flight, safe as houses. Teachers and tourists fly on them! Scheduled flights every month! Whee! Utter crap.

    But try as I might, I can't lay 100% of the blame on them: they see the budget for aero and space research being cut (more tax-cuts for the wealthy!!!) and they know they need to get public opinion behind them. That means the Shuttle must fly, and it must be a media spectacle.

    The truth of the matter is:

    • much of the "research" that is done on Shuttle flights could be done just as well by unmanned missions; and
    • "reusable spacecraft" is an oxymoron at the current state of technology (even ignoring pork boondoggles like Morton Thiokol in Utah) .
    Time to ax the Shuttle program. Give NASA some real money. Move the little experiments to the various LEO launches on small vehicles. Use heavy lift rockets like Energiya and Ariane while NASA designs and contracts out a US design, perhaps an updated SaturnV or something. To hell with jingoistic crap like "giving up the space race to the Russians and Europeans" -- let's not cut off our noses to spite our faces.

    And let's not forget that space travel for humans is still very much an experimental thing. "There be dragons -- expect to die!" There still will be no dearth of volunteers for astronaut positions.

    --
    Unlimited growth == Cancer.
  13. Re:Can it really be fixed? by Viking+Coder · · Score: 4, Insightful

    It comes down to two fundamental beliefs:

    - The people are smart enough to govern themselves.

    - Capitalistic forces will always find the optimal solution to any problem.

    You're wrong on both counts.

    If you put it to a vote, everyone in the United States would have to worship Jesus Christ, and the Death Certificate of Elvis Presley would be declared invalid. And universities would get funding money for "astrological research." The world is too complex to let all of our decisions be made by people who merely BELIEVE in things. It's far, far better to try to elect a government that will make the best decisions they can. Sometimes they make bad moves, and sometimes they make good moves. Your primary role as a citizen in the U.S. is to make sure your government is run in a way that you agree with. Not that you necessarily agree with all of their choices, but that the process works.

    How would you make national freeways? How can capitalistic forces balance the rights and freedoms of the individual versus the needs of society? People are not smart enough to research which lipstick manufacturer pours less toxic waste into the ocean - and to boycott the one that dumps more. They just aren't. And hoping that "concerned citizens" and the media will help achieve that optimal solution is pure foolishness. For one, media is run by corporations. The best way to achieve that balance is to give the power to make those decisions over to a government, and keep your government in check.

    I'm glad that we as a society don't directly vote for government funding. I think we would make HORRIBLE choices. For one, we would probably vote away our national debt ("why should we pay?!"). We would probably stop aid to Afghanistan ("feed Americans, not Afghans!"). We would probably chop public schools ("I have a right to raise my kid like I want to - in Catholic schools!"). We would probably stop AIDS research ("Why should we pay to find a cure to a disease they got by sinning against God?"). We never would have gotten involved in the European Theater in WWII ("what have the Nazis ever done to us?"). There would be no national archive ("who cares about old books?"). The results of the Human Genome Project would be patented and copyright by [insert major corporation here] ("Why should taxpayers pay for something that a private company is perfectly willing to do?"). Hell, there would be no public domain! ("You mean someone could make pornography with Mickey Mouse in it! Hell no! Let Disney hold the Copyright forever, so we can PROTECT THE CHILDREN!")

    Never underestimate the stupidity of a crowd. I, for one, am glad we don't live in a true Democracy.

    --
    Education is the silver bullet.
  14. Re:NASA's Vietnam (From today's Wall Street Journa by lommer · · Score: 4, Insightful

    God - do we have to have this argument AGAIN on slashdot?

    The problem with private enterprise is that it expects rewards from its funding - rewards that generate $$$, not scientific knowledge or nationalistic pride, but cold hard cash. The problem with space is that there is as-of-yet, no viable way to make $$$ out there. Tourism is the only industry that's already made a start in space, but its first steps were shaky, it relied on a publically-funded infrastructure, and it has yet to progress any further. As for mining, there is nothing up there that we can't get down here for cheaper. Some might point to the He-3 resources on the moon, but these are not needed at all except for in undeveloped nuclear fusion technology.

    I suppose there is one commercial industry that has been succesful in space: the sattelite communication/telecommunications industry. However, private interests are not going to progress beyond the sorts of sattelites we are currently flying, let alone go anywhere near manned flight on there own.

    In conclusion, I would argue that private interest is not an "easier beast to summon." In fact, I would say that it is much more difficult to raise funds for space exploration through private means than it is to get public support. A space race with China would generate the neccesary support very quickly, and we might start seeing some projects come to fruition rather than being nearly completed only to be scrapped for going over-budget, and then being restarted a few years later to satiate the military-industrial complex. The only alternative that I can see having any success in space other than publically-funded programs is philanthropy. If some very rich people got together and started offering more prizes similar to the X-prize, we could see some actual development. It worked in aviation, the only thing holding it back for space is that the prizes need to be that much bigger to make it worthwhile.

    In short, space exploration's only hope lies in publically funded programs or philanthropic rewards, not in the commercial exploitation of resources that don't exist.

  15. No Saturn V Saaturn VI! by Bohemoth2 · · Score: 4, Insightful

    The data is lost huh? Then build a Saturn VI!

    here's my case:

    1. Cryogenic turbo pump design and reliability has improved significantly since the early 70's

    2. all the data we need is just lying around in space museums and outdoor rocket gardens. i think i saw something on the net that had an SV laying on it's side. not to mention recoverd apollo capsuls.

    3 Materials technology both metalurgical and especially composite is well in advance of what they had available in the 60's. All we really neeed is the dimensions of this stuff

    4. our sensors and digital control devices are much more accurate and faster reacting and can process more I/O.

    5. the stages could be made reusable due to advances in materials technology giving us higher strength and lighter weight. with our miniscule electronics we could also have "smart" stages that could recover themselves to pre determined points on the globe.

    6. the payload could increased because of he abovementioned wieght savings and improvements in the turbopump/engine design.

    Thus we would have a Saturn VI instead of a Saturn V.