Slashdot Mirror


Murphy's Law Rules NASA

3x37 writes "James Oberg, former long-time NASA operations employee, now journalist, wrote an MSNBC article about the reality of Murphy's Law at NASA. Interesting that the incident that sparked Murphy's Law over 50 years ago had a nearly identical cause as the Genesis probe failure. The conclusion: Human error is an inevitable input to any complex endeavor. Either you manage and design around it or fail. NASA management still often chooses the latter."

274 comments

  1. Mark my words by zerdood · · Score: 5, Funny

    Someday all decisions will be made by machines. We'll just sit back while they do all the work. Then, no more human error.

    --
    My sig would have been a lot cooler if /. didn't filter out HTML tags 0.o
    1. Re:Mark my words by Jeffery · · Score: 1

      then the machines kill us all, and error will cease to exsist :) (as long as the machines don't use windows)

      --
      President Bush Supporter
    2. Re:Mark my words by zerdood · · Score: 1, Insightful

      Why does everyone have this Crightonesque fear? As long as competent humans program the machines, they will be made unable to harm humans.

      --
      My sig would have been a lot cooler if /. didn't filter out HTML tags 0.o
    3. Re:Mark my words by wiggys · · Score: 4, Insightful

      Except, of course, that we programmed the machines in the first place.

      When a computer program crashes it's usually down to the human(s) who programmed it, and in the rare occasions it's a hardware glitch and it was humans who designed the hardware, so we're still to blame either directly or indirectly.

      I suppose it's like the argument about whether bullets kill or the human who pulled the gun's trigger.

      --

      Sorry, but my karma just ran over your dogma.

    4. Re:Mark my words by VistaBoy · · Score: 1

      Who will make the machines? Humans. With error.

    5. Re:Mark my words by theparanoidcynic · · Score: 1

      Yes, but machines are programmed by people. Your average hacker is lazy, impatient, tired, horny and may or may not be intoxicated. They fuck up. That's why we have bugs.

      --
      Only in a Slashdot fantasy can a Slackware install turn into several hours of sex . . . . .
    6. Re:Mark my words by zerdood · · Score: 1

      The machines will be able to diagnose and change their own code if they find bugs in it. Wouldn't that be cool?

      --
      My sig would have been a lot cooler if /. didn't filter out HTML tags 0.o
    7. Re:Mark my words by Anonymous Coward · · Score: 0

      But humans working on complex machines will make mistakes! I think AI is one of the most complicated ventures humans will ever undertake!

    8. Re:Mark my words by Anonymous Coward · · Score: 0

      All future erros will then be of the blue screen varity. Posting anonymously because slashdot has tried to silence me.

    9. Re:Mark my words by penguinoid · · Score: 1

      Yes, but that does not mean we will agree with them. For example, the Terminators decided to eliminate human error.

      --
      Don't waste your vote! Vote for whoever you want, unless you live in a swing state it won't matter anyways
    10. Re:Mark my words by _Sprocket_ · · Score: 1

      You would think an article that outlines several failed autopilot systems might indicate a fundimental flaw in that thought process.

    11. Re:Mark my words by sgant · · Score: 1

      Ah, so the "Murphy Law" prone human programmer will program the machines that will be made to make decisions for us.

      Ah, but first, we must program a machine to find a competent human to program the decision-making machine!

      --

      "Leo Fender was in a 'state of grace' when he designed the Stratocaster." -- Paul Reed Smith
    12. Re:Mark my words by EvilTwinSkippy · · Score: 1
      Computers cannot make decisions. They can perform computations. They can evaluate formulas. They can even pen new algorythems. But their decision making power ultimately comes down to flipping a coin.

      It can be a very heavily weighted coin, but it is a coin nonetheless.

      --
      "Learning is not compulsory... neither is survival."
      --Dr.W.Edwards Deming
    13. Re:Mark my words by Anonymous Coward · · Score: 0

      Competent humans, huh? Look up Therac 25, my friend. And if your response to that is that the humans weren't competent, I'd ask you if it takes a few deaths to judge if someone is competent or not?

    14. Re:Mark my words by NonSequor · · Score: 3, Insightful

      If you're expecting this to result from the development of human level AI I wouldn't bet on it. In order to solve problems not predicted by its creators it will have to make some leaps of intuition the way humans do when they solve problems. The ability to propose original solutions also introduces the possibility of error. An AI will also have to rely on inductive reasoning in some situations and there is no reason to believe that a computer can avoid making any false inductions. I suspect that human level AIs will be able to do a lot of things better than us, but they will have at least some of the same flaws we do.

      --
      My only political goal is to see to it that no political party achieves its goals.
    15. Re:Mark my words by j0yb0y · · Score: 5, Funny

      Let me restate what he said,

      Someday all errors will be made by machines. We'll just sit back while they do all the work. Then, no more human error.

    16. Re:Mark my words by WormholeFiend · · Score: 3, Funny

      I suppose it's like the argument about whether bullets kill or the human who pulled the gun's trigger.

      Color me medieval, but I prefer the following analogy:
      Crossbows don't kill people, quarrels kill people.

    17. Re:Mark my words by pixelpusher220 · · Score: 1

      Obligatory:
      <humor>
      Are you implying that Microsoft programmers are *human*?

      Taco...remove this infidel!
      </humor>

      --
      People in cars cause accidents....accidents in cars cause people :-D
    18. Re:Mark my words by iluvgfx · · Score: 1

      I agree. Computers are as smart as the people who program it.

      --
      ...imagine...create...express...share...enjoy...
    19. Re:Mark my words by Anonymous Coward · · Score: 0

      There aren't any completeness theorems about bullets.

    20. Re:Mark my words by Anonymous Coward · · Score: 0

      Have you been near my cube today? Straw poll of my coworker votes me matching that description today.

      Add hungover to that list and it's me. Not a pretty sight.

    21. Re:Mark my words by Anonymous Coward · · Score: 0

      I read that as squirrels kill people. Didn't make sense, but it was funny nonetheless.

    22. Re:Mark my words by Anonymous Coward · · Score: 0

      you cant shoot squirrels out of a crossbow...

    23. Re:Mark my words by Justin+Clarke · · Score: 1

      Unless they make an error! :)

    24. Re:Mark my words by Anonymous Coward · · Score: 0

      No, you need to put them into a canon...

    25. Re:Mark my words by rapcomp · · Score: 3, Insightful

      Where do you plan to find competent humans?

      --
      Does this look like the face of concern?
    26. Re:Mark my words by Anonymous Coward · · Score: 0

      i would never put a squirrel in a canon, because they're not photosensitive, and i don't know of any photolab that will process a squirrel.

    27. Re:Mark my words by mattyrobinson69 · · Score: 1

      unless several different (as in programmed separately, by different people) programs are developed, which vote on a strategy - this way, the chances of human error are reduced greatly (unless there's a bug in the vote counting program, but then there's the possibility of redundant clean-house voting programs, which would also suffer from the same problem.

      hmm, talking bollocks on slashdot is too complicated for me - i'l leave the future of the world in the hands of people like bill gates and [insert other mediocre programmers/good buisiness men here]

    28. Re:Mark my words by Anonymous Coward · · Score: 0

      Murphy says the diagnostic code will have the bugs.

    29. Re:Mark my words by hackwrench · · Score: 1

      So, deciding what is spam or not is a matter of flipping a coin? Wow!

    30. Re:Mark my words by Anonymous Coward · · Score: 0

      Ah, these young whipper snapper 'coders', brought up nowadays on junk like Visual Basic...sigh...

      Already been done, to death, long, long ago...it's called "Core Wars" :-) All the dinosaurs on /. should recognize this one...:-)

    31. Re:Mark my words by Tablizer · · Score: 1

      Someday all decisions will be made by machines. We'll just sit back while they do all the work. Then, no more human error.

      Thank You for the compliment, Dave. Now please move away from that air-lock.
      - HAL

    32. Re:Mark my words by Tackhead · · Score: 2, Insightful
      > Why does everyone have this Crightonesque fear? As long as competent humans program the machines, they will be made unable to harm humans.

      Funny. When I read the article, I had exactly the same sentiment, but for the opposite reason:

      "As long as humans build/program the machines, the machines will fail/crash before they can kill too many people" :)

    33. Re:Mark my words by WolfWithoutAClause · · Score: 0, Offtopic
      Actually a relatively young bunch of guys in America died of CJD. Turned out they'd all eaten scrambled egg mixed with squirrel brains- a local delicacy.

      Presumably, the Squirrel had had CJD, and they'd caught it.

      So, those tree rats can kill if sufficiently provoked! :-)

      --

      -WolfWithoutAClause

      "Gravity is only a theory, not a fact!"
    34. Re:Mark my words by feargal · · Score: 1

      Well that's *easy*.

      Write a computer program that analyses candidates' abilities and produces a list of the most competent humans...

      --
      "A goldfish was his muse, eternally amused"
    35. Re:Mark my words by Anonymous Coward · · Score: 0

      > Someday all decisions will be made by machines. We'll just sit back
      > while they do all the work. Then, no more human error.

      You are in error.

    36. Re:Mark my words by JohnFluxx · · Score: 1

      That sounds much better than humans.

      Humans: Look at some evidence, come up with a decision that seems sort-of best.

      Computer: Look at all the evidence it can, come up with the best decision under the constraints it has.

      Sounds like the computer will make the better decisions.

    37. Re:Mark my words by Anonymous Coward · · Score: 0
      " scrambled egg mixed with squirrel brains"

      That's disgusting!!!

      They deserved to die. And the "cook"!

    38. Re:Mark my words by admiralh · · Score: 1

      Putting squirrels into a canon won't work, because canons are much too simple. For them to be shot properly, you've got to use a full-blown six-part fugue.

      --
      Hopelessly pedantic since 1963.
    39. Re:Mark my words by Fallen+Andy · · Score: 1

      Not quite the same. Corner cases you see. When an engineer or programmer designs something they *try* to cover the cases where someone abuses things. Validating everything is kind of impossible, except for trivial engineering, and
      this led to nonsenses such as the "Viper" CPU which didn't have interrupts (non deterministic is
      evil you see (grins)). Needless to say that idea
      had the aerodynamic capability of the average
      porcine..

      But truthfully, there is no software industry version of
      the infamous Underwriter's Lab (and I want to watch
      those guys *do* those hideous things from a safe distance).(what happens to your vacuum cleaner when you accidentally poke it into the tropical fish tank etc. etc.).

      Often, at least when I started out, there was no guarantee that the hardware was even a good idea, let alone a good design. It just existed, and
      some fool in marketting had told a nother (grin)
      fool in management that we can make money doing
      some fool thing which nobody in their right mind
      would do. (Like port an operating system no fool in their right mind would use).

      OK, you've just been pointed to. Nobody notices that said pointee *doesn't* know anything about the CPU's instruction set, has never done a serious programming project more than 1000 lines of basic (sorry, BASIC) and nobody in the company
      masochistic enough to take on the project can help. (It helps to remember that mutual help a la
      mutual assured destruction aka the "Internet" didn't exist).

      So, what do you do. Er. You try hard to do the best engineering job you can (and feel foolish *all* of the time). Feeling dumb and dreaming of
      enlightenment is the Engineer's curse. Hence, the
      Buddhist jokes that circulate..

      Understand. The people doing projects at NASA are stuck with this curse 120% of the time, *and* all the red-tape from hell.

      I always wonder how they manage to do anything.
      Burt (Rutan) is set up pretty much as a "small" outfit. Hence, everyone is in a tight comm's loop and the usual small company "telepathy" kicks in.

      If you've never risked your career in a small outfit then you've never risked living...

      Bottom line: Bullets *do* kill.
      Some one (perhaps the dog) did pull
      the trigger.

      It might have been the hamster (see Dave Barry)

    40. Re:Mark my words by JuggleGeek · · Score: 1
      Computers cannot make decisions.

      Sure they can. Pretty much any computer can easily beat you or me (or both of us together) at chess these days. Time and time again. Computers make decisions in a manner much different from humans, but they still make decisions.

    41. Re:Mark my words by JuggleGeek · · Score: 1
      If you're expecting this to result from the development of human level AI I wouldn't bet on it. In order to solve problems not predicted by its creators it will have to make some leaps of intuition the way humans do when they solve problems.

      Many people tend to think of computers as a "us vs them" kind of thing. Can the best human play better chess than the best computer? That sort of thing.

      I see computers as a tool. I, as a human, can remember more things, do more calculations, be more productive, and do things I could not otherwise do, because I can use the computer to help.

      An AI that has all of the abilities of a human (and presumably none of the flaws?) is still a *long* way away, and may never happen. But AI development should be able to help *humans* make better decisions much sooner than that. Say, for instance, starting _now_.

  2. interesting but it's not really true by spacerodent · · Score: 4, Interesting

    while it's possible to always have a mistake, having people double check a project from the ground up will almost always find the problems. Nasa's current difficulties arise from scattered teams that all only check their parts rather than having fully qualified teams that go over the entire vehical. The fact that the whole thing is usually designed by committee and in several pieces then assembled at the last minute probally helps facilitate error. The Saturn V rockets and other technology we used to land on the moon had hte capability of being far less relyable than today's technology but we still managed to use them for years without error.

    1. Re:interesting but it's not really true by jmmcd · · Score: 1

      ok, but how are they supposed to test all their parts together in advance of the "real" launch?

    2. Re:interesting but it's not really true by Moby+Cock · · Score: 4, Informative

      Its an oversimplification to say that older technology was used without errors. In fact, its just downright incorrect. Appolo 1 and Appolo 13 both suffered from catastophic failures. Furthermore, the next generation of space vehicles, the shuttle, has had two very significat disasters and reams of other failures.

    3. Re:interesting but it's not really true by EvilTwinSkippy · · Score: 4, Insightful
      I'm still trying to figure out why the Apollo formula of contractors with Nasa oversight doesn't seem to work anymore.

      Then I remember Apollo 1, that killed 3 astronauts, and Apollo 13, that nearly killed 3 more.

      To invoke Heinlien, Space is a harsh mistress.

      To invoke Sun Tsu, success in defense is not based on the likelyhood of your enemy attacking. It is based on your position being completely unassailable.

      --
      "Learning is not compulsory... neither is survival."
      --Dr.W.Edwards Deming
    4. Re:interesting but it's not really true by Anonymous Coward · · Score: 0

      the saturn 5 never had a failure

    5. Re:interesting but it's not really true by HeghmoH · · Score: 0

      I have not RTFA, but it seems to me that having people double-check things qualifies as acknowledging human error and designing around it.

      --
      Mod down posts with a "Free Mac Mini/iPod" sig, they're spam!
    6. Re:interesting but it's not really true by Anonymous Coward · · Score: 0

      if they had run diagnostics on the circiuts they would have seen the switch was installed backwards by the resitance differences, they could also have simply traced the whole circuit looking for errors and would most likey have seen the switch reversed. The mistakes that I'd say are unavoidable are more in the realm of totally overlooked things like the door on the Apollo capsule that got everyone inside burned to death. Simple errors are not just "luck", they're sloppy engineering.

    7. Re:interesting but it's not really true by rabtech · · Score: 2, Informative

      No, it isn't. The Saturn V rocket was the most complicated and largest system ever built by man at the time and launched without a SINGLE failure for its entire operational life. The vehicles and satellites it carried had problems but the rocket itself never failed.

      --
      Natural != (nontoxic || beneficial)
    8. Re:interesting but it's not really true by GR1NCH · · Score: 3, Insightful

      I think this goes along with the saying: 'If you make something idiotproof, someone will build a better idiot'. Sure maybe they could have designed the accelerometers so that they couldn't be installed backwards. But then again what else might have failed. I guess in the end it all comes down to econnomics. What does the cost-benifit analysis say? Is it better to keep checking and double-checking, or to just send it out like it is. Now, I can understand cost-benifit is a little bit difficult when you are talking about a space probe. But chances are you could keep redesigning and rechecking a probe for 50 years and then something you never thought of will come up and all your plans will go to hell.

      Maybe we should just make lots of cheap crappy probes, and hope and expect most to fail instead of one really good expensive one with the hope that it will suceed.

    9. Re:interesting but it's not really true by Wizzy+Wig · · Score: 5, Insightful
      ...having people double check a project from the ground up will almost always find the problems...


      Then you double check the checkers, and so on... that's the point of the article... humans will err... Like Demming said... "you can't inspect quality into a process."

    10. Re:interesting but it's not really true by Moley · · Score: 1

      not really true??? Even using formal methods errors are bound to creep in. I agree with the article that on large projects human error is simply unavoidable! That's why NASA use fault tolerance systems, and that's why it's a good idea to use fault tolerance in combination with formal methods for critical systems.

    11. Re:interesting but it's not really true by eggoeater · · Score: 3, Informative

      The fact that the Saturn V rockets never blew up doesn't mean they never had problems! There were plenty of things that went wrong. Even in the movie Apollo 13, one of the Saturn V engines malfunctioned during take off. We survive failures in rockets and other critical pieces of technology due not only to pragmatic design but also redundancy. (Also, think about the design of airplanes...triple redundancy on hydrolic lines.)
      Also, there was some kind of semi-critical problem in EVERY SINGLE Apollo mission except Apollo 17, the very last one.

    12. Re:interesting but it's not really true by Control+Group · · Score: 5, Insightful
      No, it is true. It's the "almost always" in your statement that's the key. It's simple statistics, really. Assume that a well-trained, expert engineer has a 5% chance of making a material error. This implies that 5% of the things s/he designs have flaws.

      Now suppose this output is double-checked by another engineer, who also has a 5% chance of error. 95% of the first engineer's errors will be caught, but that still leaves a .25% chance of an error getting through both engineers.

      No matter what the percentages, no matter how many eyes are involved, the only way to guarantee perfection is to have someone with a zero percent chance of error...and the chances of that happening are zero percent. Any other numbers mean that mistakes will occur. Period.

      I remember reading a story somewhere about a commercial jet liner that took off with almost no fuel. There are plenty of people whose job it is to check that every plane has fuel...but each of them has a probability of forgetting. Chain enough "I forgots" together, and you have a plane taking off without gas. At the level of complexity we're dealing with in our attempts to throw darts at objects xE7 kilometers away, it is guaranteed that mistakes will propagate all the way through the process.

      --

      Reality has a conservative bias: it conserves mass, energy, momentum...
    13. Re:interesting but it's not really true by orac2 · · Score: 2, Interesting

      from scattered teams that all only check their parts rather than having fully qualified teams that go over the entire vehical.

      Your sentiment is correct, but your details are a little off. For example the Saturn V rocket was built by "scattered teams" (and committees were heavily involved, despite the mythology around Von Braun)-- the first stage was built by Boeing, the second by North American, the third by Douglas Aircraft, the Instrument Unit (the control system) by IBM, the LEM by Grumman and the CSM by North American, and so on, all the way down a huge chain of sub-contractors. But Apollo had brilliant technical management: it was pricey, but did do an amazing job of system integration.

      It's when you try for cheaper missions that having one team take a spacecraft from design through operation is important: this was done on the Mars Pathfinder mission to great success, but wasn't done on other "Faster, Cheaper, Better" missions, to great falure, as demonstrated by, well, take your pick.

      --
      "Just once, I'd like to meet an alien menace that wasn't immune to bullets." -- The Brigadier, Dr. Who
    14. Re:interesting but it's not really true by gammygator · · Score: 2, Insightful

      Finding problems is a good thing but I've found that nobody likes to be told their baby is ugly.... and if they're far enough up the corporate food chain... good luck getting 'em to listen.

      --

      No Nyarlathotep, No Chaos
      Know Nyarlathotep, Know Chaos
    15. Re:interesting but it's not really true by mikael · · Score: 2, Interesting

      Just like the case in which the airport crew assigned to clean an aeroplane put some masking tape over the air pressure sensors, but forget to remove it. Or rather, as the airport was badly lit and the masking tape wasn't noticably different from the skin of the aircraft, nobody noticed this small defect. Until the pilots came in to land the aeroplane that is, then it became a large problem.

      --
      Vintage computer adverts: http://www.vintageadbrowser.com/computers-and-software-ads
    16. Re:interesting but it's not really true by olderchurch · · Score: 2, Funny

      Makes me think of the quote from Armageddon:
      Rockhound : You know we're sitting on four million pounds of fuel, one nuclear weapon and a thing that has 270,000 moving parts built by the lowest bidder. Makes you feel good, doesn't it?

      And it will have flaws, no matter how often and thorough you check.

      --
      Disclaimer: This opinion was created without the use of any facts
    17. Re:interesting but it's not really true by 0123456 · · Score: 3, Informative

      "The vehicles and satellites it carried had problems but the rocket itself never failed."

      No, but it came damn close. The 'pogo' problem on one of the launches, for example, almost lead to the loss of the Saturn V: if I remember correctly it would have broken up in a few seconds, but one of the engines shut down due to excessive forces and that saved the rocket.

      The sad thing is that by the time we launched the last Saturn the worst of the bugs had been resolved, just in time to stop flying them...

    18. Re:interesting but it's not really true by _Sprocket_ · · Score: 3, Informative


      I'm still trying to figure out why the Apollo formula of contractors with Nasa oversight doesn't seem to work anymore.


      Take a look at Chapter 5 of the CAIB Report. You might be especially interested in Section 5.3 - "An Agency Trying To Do Too Much With Too Little." And since you're comparing Apollo era NASA with today's program, look at diagrams 5.3-1 and 5.3-3. In short, the Apollo program enjoyed considerably more funding.
    19. Re: interesting but it's not really true by gidds · · Score: 2, Insightful
      Assume that a well-trained, expert engineer has a 5% chance of making a material error. This implies that 5% of the things s/he designs have flaws.

      Now suppose this output is double-checked by another engineer, who also has a 5% chance of error. 95% of the first engineer's errors will be caught...

      That doesn't follow. It's only true if the two errors are completely independent, which is a very big 'if'. In practice, the chances are that some types of error are more likely than others, and that the processes/standards/ways of thinking which are common to both will also affect the types of errors they make. All of which makes it more likely that if one engineer has made a particular mistake, another engineer might make that same mistake as well. So the second engineer will catch less than 95% of the first engineer's errors -- maybe a lot less.

      Of course, things are far better when there's little or no commonality between the two engineers -- different companies, processes, methods, approaches, and cultures will all help to make their work independent, and help to reduce the errors that get missed. Anyone know if NASA does anything like this?

      I believe it's common practice in some mission-critical situations to use three different systems, each built from the ground up by three entirely separate groups of people, with nothing by the specification in common, for exactly this reason.

      --

      Ceterum censeo subscriptionem esse delendam.

    20. Re: interesting but it's not really true by Control+Group · · Score: 2, Insightful
      I was being overly simplistic, admittedly, but I think the "model" (to put on airs) I used illustrated my point adequately: as long as there is a percent chance of an error being made at every step of the process, an error will eventually be made.

      Obviously, the trick is to minimize the odds, but you can't eliminate them.

      --

      Reality has a conservative bias: it conserves mass, energy, momentum...
    21. Re:interesting but it's not really true by DerekLyons · · Score: 1
      Its an oversimplification to say that older technology was used without errors. In fact, its just downright incorrect. Appolo 1 and Appolo 13 both suffered from catastophic failures.
      Indeed. And it's interesting to look at the root causes: Apollo 1- poor design, poor managment, poor craftsmanship. Apollo 13-poor managment (the cockup regarding the thermostat switch) and foulups by the workers on the floor (damaging the tank that would later explode because of the damage and the switch).

      While it's fashionable to blame the NASA in the present, and worship them in the past, some of the same attitudes and problems are clearly present in the past when you study it with open eyes. (For example, on no fewer than three occasions, they continued with a moon landing, despite significant problems. Problems which by NASA's own rules required an abort of the landing attempt.)

    22. Re:interesting but it's not really true by sphealey · · Score: 4, Insightful
      I'm still trying to figure out why the Apollo formula of contractors with Nasa oversight doesn't seem to work anymore.
      Two reasons. First, outsourcing requires more and better project managers and technical managers than insourcing. Many organizations learned this to their sorrow in the 1980s; many more are going to learn it around 2006.

      Second, the stable of competent contractors that existed in the 1940-1960 time frame is gone. North American, Grumman, McDonnell, dozens of others that could be named have been absorbed into 2-3 borg-like entities. The result is less competition, less choice, less innovation, few places for maverick employees to go, and in the end worse results from outsourcing.

      sPh

    23. Re:interesting but it's not really true by Anonymous Coward · · Score: 0

      of course, the Heinlein statement was really 'The Moon is a Harsh Mistress" the title of one of his books.

    24. Re:interesting but it's not really true by OwnedByTwoCats · · Score: 1

      Apollo had two gross errors in twelve launches: the Apollo 1 fire that killed Grissom, White, and Chafee, and the Apollo 13 explosion.

    25. Re:interesting but it's not really true by Rei · · Score: 2, Interesting

      You're saying that there was no way to see the risk of *fire in a pure oxygen atmosphere*?

      They certainly couldn't have tested the circuit here. The circuit detonated a ballistically launched chute. Not exactly the sort of thing you want to be doing in the lab on a fully built craft you're about to launch. Yes, they could have tested the integration of the chute with the craft electronics, but then to justify that, you'd need to justify testing *every* integration made on the craft. And you people already complain about how much NASA costs - what, are you just looking for more fuel to throw on the "NASA is too expensive" fire?

      Ah well. Let the NASA-bashing thread continue. God knows slashdot has enough of them, always participated in by those who have never designed a rocket or probe in their life....

      --
      Did you really name your son "Robert');DROP TABLE Students;--"?
    26. Re:interesting but it's not really true by EvilTwinSkippy · · Score: 1
      +1 insightful.

      Onward to Wayland-Yuatani

      --
      "Learning is not compulsory... neither is survival."
      --Dr.W.Edwards Deming
    27. Re: interesting but it's not really true by OwnedByTwoCats · · Score: 2, Interesting

      The flight control computers on the Shuttle are an interesting example. If you have one fcc, and it dies, then you lose the vehicle and crew (vehicle is unflyable without the fcc). If you have two fccs, and one starts producing erroneous results, you don't know which one to trust. If you have three computers, you can survive any one of them failing, but then a second failure causes you problems.

      If you have four computers, there's an outside chance that two will fail, and you will have to choose between two As and two Bs, and if you choose the wrong one, vehicle and crew are debris.

      So the shuttle has five computers. The fifth runs software developed independently from the software running on the other four, and breaks two-two ties.

      Probably the most reliable large software project ever. But horribly expensive in $/SLOC...

    28. Re:interesting but it's not really true by orac2 · · Score: 1

      They did test the accelerometers, using a centrifuge. But the test harness was wired backward too. Didn't read the article did you?

      There's certainly a balance to be found between too much testing and too little. But -- as the Columbia Accident Investigation Board report harshly criticised -- at NASA, the pendulum had swung too far from physical testing towards simulation, extrapolation, and pure guess-work.

      Let the NASA-bashing thread continue.

      NASA does appear to be working on its management issues. But until it proves it has succeded in making significanct progress, it has, sadly, lost the benefit of the doubt. NASA is a public program, and if NASA management or Congress won't provide the requisite oversight, then we should be grateful that people like Oberg (a 22-year NASA veteran) are holding NASA's feet to the fire.

      --
      "Just once, I'd like to meet an alien menace that wasn't immune to bullets." -- The Brigadier, Dr. Who
    29. Re:interesting but it's not really true by JollyFinn · · Score: 1

      I already trust my computer. My computer has no business 'wondering' whether it trusts me or not.
      Your sig, happens explain this more than enough. Computers should never trust humans.

      --
      Emacs is good operating system, but it has one flaw: Its text editor could be better.
    30. Re:interesting but it's not really true by shotfeel · · Score: 1

      The sad thing is that by the time we launched the last Saturn the worst of the bugs had been resolved, just in time to stop flying them

      Sounds like most soft- and hard-ware. About the time the bugs are ironed out, its time for the next version with new bugs.

    31. Re:interesting but it's not really true by shotfeel · · Score: 1

      And building in fault tolerance costs money.

      NASA really doesn't have much room to manuver. If they build in too much tolerance, the program costs too much and will never get funded. Not enough and you risk catastrophe.

      Said another way, any time cost becomes a design contraint, some other variable (i.e. safety factor) has to give.

    32. Re:interesting but it's not really true by shotfeel · · Score: 1

      Now suppose this output is double-checked by another engineer, who also has a 5% chance of error. 95% of the first engineer's errors will be caught, but that still leaves a .25% chance of an error getting through both engineers.

      OK, nitpicking here. My statistics are a little rusty, but I think you got that wrong. You calculted the joint probability that both would miss a flaw. What you want to calculate is the probability that they will both miss the same flaw.

      So there's a 5% probability the first one misses it, but a 95% chance that it will be caught by the second person. So, 5%-(95% x 5%) leaves only a 0.25% chance that both will miss it.

      Either way though, your point is correct. There will always be a finite possibility of failure.

      Should also acknowledge that there are also a lot of other assumptions that we're probably both aware of.

    33. Re:interesting but it's not really true by Mark+of+THE+CITY · · Score: 1

      I have heard a claim that Wally Schirra, in response to a press conference question along the lines of "What are you thinking about, sitting on the rocket, waiting for launch?" His reply, except for the nuke part, was essentially that attributed to "Rockhound" in the parent post.

      --
      The clearance system sounds logical. It is not. It is completely arbitrary. -- John Bolton
    34. Re:interesting but it's not really true by legirons · · Score: 1

      "I think this goes along with the saying: 'If you make something idiotproof, someone will build a better idiot'."

      The article mentioned accelerometers whose orientation could only be determined by x-raying the component -- regardless of the people involved, you'd imagine that that would cause the odd problem or three

    35. Re:interesting but it's not really true by Control+Group · · Score: 1
      So, 5%-(95% x 5%) leaves only a 0.25% chance that both will miss it.

      Er

      Isn't that what I said? Or am I missing a subtle distinction between your statement and mine?

      (Bear in mind I freely admit that all my statistics knowledge is based on two semesters in college, I could easily be wrong)

      --

      Reality has a conservative bias: it conserves mass, energy, momentum...
    36. Re:interesting but it's not really true by GR1NCH · · Score: 1

      Very true. One would think that a relatively inexpensive fix would be to stamp the part 'this way up'.

    37. Re:interesting but it's not really true by jafac · · Score: 1

      Hell, Apollo 11 had some failures that nearly cost the mission (and loss of crew). Some false error conditions reported by the flight hardware on descent, and a very slim margin of fuel consumption prior to landing, they came within less than 10 seconds of depleting the fuel alloted for landing, but they were too low to abort.

      --

      These are my friends, See how they glisten. See this one shine, how he smiles in the light.
    38. Re:interesting but it's not really true by Performaman · · Score: 0

      But did it ever have a catastrophic failure?

      --

      I have gas, but my car uses petrol.
    39. Re:interesting but it's not really true by Anonymous Coward · · Score: 0

      The book's title is "The Moon is a Harsh Mistress", not space.

    40. Re:interesting but it's not really true by Anonymous Coward · · Score: 0

      Even in the movie Apollo 13, one of the Saturn V engines malfunctioned during take off.

      The movie was just that: a movie. This actually happened in real life. Your comment would make sense if they used a working scale model for the movie, and it failed take-off (but it was all computer graphics).

    41. Re:interesting but it's not really true by JuggleGeek · · Score: 1
      while it's possible to always have a mistake, having people double check a project from the ground up will almost always find the problems.

      That's like saying that the weather can be reliably predicted by having a 2nd weatherman double check the forcast. Sorry, but in real life, there are too many unknown variables, and it just doesn't work that way.

      Considering what they are have done and are doing, I think NASA has done incredible things with a better safety record than would be expected. It *is* rocket science, not Boiling-Water 101. Yes, mistakes happen. I believe that is unavoidable.

    42. Re:interesting but it's not really true by JuggleGeek · · Score: 1
      One would think that a relatively inexpensive fix would be to stamp the part 'this way up'.

      It could go right over the part that says "Made in China".

  3. Cost Effective by clinko · · Score: 4, Interesting

    It's actually more cost effective to allow for failures. You build the same sat 5 times and if 4 fail in a cheaper launch situation, you still save money.

    From this article:

    "Swales engineers worked closely with Space Sciences Laboratory engineers and scientists to define a robust and cost-effective plan to build five satellites in a short period time."

    1. Re:Cost Effective by Anonymous Coward · · Score: 0

      Speaking of cost effective, wasn't Jimbo trying to save some money for Wiki? This is going to put him over budget.

  4. Murphy's corrolary by draxredd · · Score: 0

    Murphy's law is rocket science

    --
    --- Back to the trees, back to the trees !
  5. Let's get serious. by Anonymous Coward · · Score: 0

    I sincerely doubt that NASA engineers ever set out to intentionally fail. Nice troll though.

  6. Good Point by RAMMS+EIN · · Score: 5, Insightful

    ``Human error is an inevitable input to any complex endeavor. Either you manage and design around it or fail.''

    This is a very good point, and I wish more people would realize it.

    For software development, the application is: Just because you can write 200 lines of correct code does not mean you can write 2 * 200 lines of correct code. Always have someone verify your code (not yourself, because you read over your errors without noticing them).

    --
    Please correct me if I got my facts wrong.
    1. Re:Good Point by Spy+Hunter · · Score: 1

      There is a third option which is often overlooked, even though it is nearly always applicable: Reduce the complexity of your system! It can be a lot harder to design a simple system to solve a complex problem, but it's the only way to truly defeat Murphy's Law. It may be that some problems simply can't be solved by a simple system, but I'll wager that the set of those problems is a lot smaller than most people realize.

      --
      main(c,r){for(r=32;r;) printf(++c>31?c=!r--,"\n":c<r?" ":~c&r?" `":" #");}
    2. Re:Good Point by Anonymous Coward · · Score: 0

      Complex problems do not have simple solutions.

      The "simplicity" cry is a cop-out.

      Would you rather build a web browser using "simple" machine code, or use a higher level but more complex language?

  7. That's right by nels_tomlinson · · Score: 3, Funny

    That's right, blame it all on the Irish. After all, it's not like anyone else ever screwed up...

    1. Re:That's right by Ford+Prefect · · Score: 2, Informative

      Murphy's law has nothing to do with the Irish.

      Anyway, has anyone else ever thought an article is based on a Slashdot post one made? I was thinking about the exact same similarities last week. :-)

      --
      Tedious Bloggy Stuff - hooray?
    2. Re:That's right by Anonymous Coward · · Score: 0

      Yeah. Actually, he forgot Poland.

  8. That is NOT correct. by Puls4r · · Score: 4, Insightful

    >>Either you manage and design around it or fail. >>NASA management still often chooses the latter.

    This is hindsite at its best, and is the classic comment by beareaucrats who have no concept of what cutting edge design is about. F1 race cars, Racing Sailboats, Nuclear Reactors - NO design is failsafe, and NO design is foolproof. Especially a one off design that isn't mass produced. Even mass produced designs have errors, like in the Auto Industry. It is a simple fact of life that engineers and managers balance Cost and Safety constantly.

    What you SHOULD be comparing this against is other space agencies that launch a similar number of missions and sattelites - i.e. other real world examples.

    Expecting perfection is not realistic.

    1. Re:That is NOT correct. by EvilTwinSkippy · · Score: 2, Informative
      If you are going be sheer number of launches, body count, payload capacity, or cost effectiveness, the Russians have us beat hands down.

      Sure we've been to the moon. But we haven't done a damn bit of fundimental research since then. (A lot of improvements to our unmanned rocket technology have been bought/borrowed/stolen from the Russian program.)

      --
      "Learning is not compulsory... neither is survival."
      --Dr.W.Edwards Deming
    2. Re:That is NOT correct. by _Sprocket_ · · Score: 2, Insightful


      This is hindsite at its best, and is the classic comment by beareaucrats who have no concept of what cutting edge design is about. F1 race cars, Racing Sailboats, Nuclear Reactors - NO design is failsafe, and NO design is foolproof.


      But this isn't about design. It's about implementation. In each of the examples, the failure occurred because of incorrect assembly of key components.

      Having said that - there IS an issue of design brought up by the article. That is, the design of a system should not allow for catastrophic configuration. In several examples, failure occurred when sensors (accelerometers) were installed backwards. Those devices should have been designed with some sort of keying system that only allows installation in the intended configuration. Heck - one of the accelerometers' configuration could only be determined after x-raying the device!
    3. Re:That is NOT correct. by orac2 · · Score: 4, Insightful

      This is hindsite at its best, and is the classic comment by beareaucrats who have no concept of what cutting edge design is about.

      You only get to play the hindsight card the first time this kind of screw-up happens. If you actually read the article you'll see that Oberg (who isn't a beauracract but a 22-year veteran of mission control and one of the world'd experts on the Russian space program) is indicting NASA for having a management structure that leads to technical amnesia: the same type of oversight failure keeps happening again and again.

      Oberg is not alone in this. The Columbia Accident Report despairingly noted the similities between Columbia and Challanger: both accidents where caused by poor management but what was worse with Columbia was that NASA had failed to really internalise the lessons of Challanger, or heed the warning flags about management and technical problems put up by countless internal and external reports.

      Sure, space is hard. But it's not helped by an organization that has institutionalised technical amnesia and abandoned many of its internal checks and balances (at least this was the case at the time of the Columbia report, maybe things have changed).

      And if you really want to compare against other agencies, NASA's astronaut bodycount does not compare favorably against the cosmonuat bodycount...

      Sadly, your post is a classic comment by slashdotters who have no concept what effective technical management of risky systems looks like. (Hint: not all cutting edge designs get managed the same way. There's a difference between building racing sailboats and spaceships. This is detailed in the Columbia accident report. Read it and get a clue).

      --
      "Just once, I'd like to meet an alien menace that wasn't immune to bullets." -- The Brigadier, Dr. Who
    4. Re:That is NOT correct. by khallow · · Score: 1
      This is hindsite at its best, and is the classic comment by beareaucrats who have no concept of what cutting edge design is about. F1 race cars, Racing Sailboats, Nuclear Reactors - NO design is failsafe, and NO design is foolproof. Especially a one off design that isn't mass produced. Even mass produced designs have errors, like in the Auto Industry. It is a simple fact of life that engineers and managers balance Cost and Safety constantly.

      This advice better applies to yourself. Why does NASA use "one off" designs for all of its work (eg, the Space Shuttle, space probes, etc)? And NASA doesn't balance cost and safety. First, they don't practice even the most rudimentary of cost controls. Namely, NASA still uses "cost plus" as the basis of most of its contracts. Second, NASA has come up with elaborate safety measures and procedures that were routinely bypassed by those who supposedly were performing the inspections.

      Incidentally, most commercial nuclear reactors should not be "one off" designs. Ie, build a few prototypes and then get a stable design. That's just my opinion, but I think it explains part of what went wrong with US nuclear power in the 60's and 70's.

      What you SHOULD be comparing this against is other space agencies that launch a similar number of missions and sattelites - i.e. other real world examples.

      Why? We lower our standards of performance to government bureaucracies when we do that. They are inherently inefficient since survival of the organization doesn't depend on success. Scaled Composites, for example, has demonstrated a suborbital craft capable of barely reaching space for a cost of around $25 million. In comparison, NASA developed and flew three X-15 prototypes with similar capabilities for a cost of $300 million in 60's dollars (which incidentally was considered a cheap program).

    5. Re:That is NOT correct. by at_18 · · Score: 2, Interesting

      Scaled Composites, for example, has demonstrated a suborbital craft capable of barely reaching space for a cost of around $25 million. In comparison, NASA developed and flew three X-15 prototypes with similar capabilities for a cost of $300 million in 60's dollars (which incidentally was considered a cheap program).

      With the small difference that Scaled Composites is benefitting from 30 years of technology advancements. I don't think that an equivalent company of the 60s could build three SpaceShipOnes for $300 million.

    6. Re:That is NOT correct. by 0123456 · · Score: 2, Interesting

      "F1 race cars, Racing Sailboats, Nuclear Reactors - NO design is failsafe, and NO design is foolproof."

      Not true: there are failsafe nuclear reactor designs that even a genius couldn't manage to melt down, let alone a fool... you just have to design them with safety guaranteed by the laws of physics, not the control systems. General Atomics built a lot of them decades ago, and the Chinese are developing modern versions today.

      Good design prevents a heck of a lot of problems. If nothing else, you'd have thought that by now engineers would realise that if you design something so it can be fitted backwards, sooner or later it will be.

    7. Re:That is NOT correct. by _Sprocket_ · · Score: 2, Insightful


      In comparison, NASA developed and flew three X-15 prototypes with similar capabilities for a cost of $300 million in 60's dollars (which incidentally was considered a cheap program).


      You've got good points. But you're being unfair on this one. Even the Rutan notes that the X-15's capabilities far outstrip Spaceship One. That, and X-15 provided some of the basic building blocks in aero and astronomics on which Spaceship One could be built. Furthermore, Spaceship One enjoyed numerous high-performance off-the-shelf materials that didn't exist during the X-15's time. Comparing the two provides some interesting historical perspective. But the two programs are apples and oranges.
    8. Re:That is NOT correct. by Dun+Malg · · Score: 2, Interesting
      If nothing else, you'd have thought that by now engineers would realise that if you design something so it can be fitted backwards, sooner or later it will be.

      Hah! Engineers are the most intelligent bunch of idiots you'll ever find. The problem with engineers is that often their own cleverness and/or familiarity with the item they're designing blinds them to the viewpoint of someone who's "not clever" or totally new to the item. With (for example) the classic non-reversable, yet perversely symmetrical accelerometers, it probably never occured to the engineer designing them that someone could "not know" which end goes up. Sometimes it looks like just plain stupid engineering, like with a particular telephone PBX control system I work with. It has two expansion slots, Slot 1 and Slot 2. When you want to add only one expansion card, where do you put it? Slot1? No, that's too obvious. You put it in Slot 2. If you out a second card in later, that goes in Slot 1. At first I thought it was just an error in labeling the slots on the cabinet, but then I noticed that the circuit board itself is marked the same way! I'm sure there's a perfectly rational reason for it that makes sense only to the engineers who designed the system.

      --
      If a job's not worth doing, it's not worth doing right.
    9. Re:That is NOT correct. by DerekLyons · · Score: 2, Insightful
      If you are going be sheer number of launches, body count, payload capacity, or cost effectiveness, the Russians have us beat hands down.
      Well,part of one out of four isn't bad. Let's examine these in detail shall we?
      • Sheer number of launches - This is the only one that the Russians 'beats' the US on, mostly because their hardware is unreliable and short lived, thus requiring frequent replacement. So far as manned flight go however, they've actually flown less. (87 Soyuz flights vs. 113 Shuttle flights alone.)
      • Body Count - The Shuttle (alone) has carried to, and returned from, space nearly twice as many people as the entire Russian space program.
      • Payload Capacity - Not counting vaporware like Energia, or booster not in production like the Saturn V... We find there is actually little difference between currently available payload weights.
      • Cost efectiveness - well, comparing apples to oranges invariably leads to odd results. Soyuz is mostly cost effective because it's has extremely low performance. (And because all of the R&D and infrastructure was paid for by somebody else.) They closest American program you can compare Soyuz to... Is Gemini.
    10. Re:That is NOT correct. by OwnedByTwoCats · · Score: 1
      Why does NASA use "one off" designs for all of its work (eg, the Space Shuttle, space probes, etc)?

      NASA usually doesn't use "one off" designs its work. There were five shuttles built. Voyager one and 2, and the two mars rovers are examples of "build two, so that its more likely that one will work".

      The commercial launch business also has a few standard designs and tested optional configurations that get used over and over again.
    11. Re:That is NOT correct. by Rei · · Score: 1

      I can't give you stats for Genesis, but the Space Shuttle has 2.5 million parts, including 370 kilometers of wire, more than 1060 valves, over 1440 circuit breakers, and more than 27,000 thermal tiles and insulating blankets. Even the simpler one-use systems, built by the US or Russians, had hundreds of thousands of parts. Genesis probably had at least as many parts as your car, and didn't have the benefit of decades of mass production behind it.

      I find it insulting that people expect perfection from NASA. Getting into space, travelling through it, and getting back to Earth safely are *incredibly* hard work.

      You want more "checks and balances"? Then expect a higher price tag. You want a lower price tag? Expect more failures. You can't have everything.

      PS - Nasa's bodycount compares very favorably to the Soviet bodycount, especially when you compare the number of manned launches. Heck, there were cases where the Soviets launched people on craft that had not had a successful launch to date.

      --
      Did you really name your son "Robert');DROP TABLE Students;--"?
    12. Re:That is NOT correct. by orac2 · · Score: 1

      I find it insulting that people expect perfection from NASA. Getting into space, travelling through it, and getting back to Earth safely are *incredibly* hard work

      No-one is expecting perfection. What we do expect, and have a right to do so, is to expect a management structure that does not actively prevent already learnt solutions to known problems being applied.

      There's a difference between doing something risky and simply being reckless.

      --
      "Just once, I'd like to meet an alien menace that wasn't immune to bullets." -- The Brigadier, Dr. Who
    13. Re:That is NOT correct. by Rei · · Score: 1

      PBMRs are interesting, but they're not failsafe. People said the same thing when they made nuclear reactors use water as a moderator; it was something to the effect of "Look, the faster the reaction rate, the more water will boil off, slowing the reaction rate - what could go wrong?"

      I find it scary that they plan to not have a containment structure for PBMRs. The main risk is that they use graphite as a moderator. Graphite moderators burn, and burn incredibly dirty, as we saw in Chernobyl (among other nuclear disasters, of which there were far more than most people know about). One big "safety feature" of PBMRs, supposedly, is that if the helium leaks out, air will act as coolant. But in the presence of air, the graphite can burn. Furthermore, if there was a steam pipe explosion that penetrated the reactor core, you'd get hot steam on the graphite, which generates hydrogen (not good!) and can start a fire.

      I like the PBMR design. I really do. But lets not get cocky about it.

      --
      Did you really name your son "Robert');DROP TABLE Students;--"?
    14. Re:That is NOT correct. by shotfeel · · Score: 1

      As is mentioned in the article, it is about design, not assembly.

      "From the NASA report, it seems that the accelerometers had to be X-rayed to determine the internal up-down orientation of their sensors, which reportedly were described incorrectly in the technical drawings."

      There are two design flaws right there. First, you shouldn't have to X-ray it to determine orientation, second that the drawings are incorrect. So everything was assembled according to the techinical drawings.

      So the company that built the sensor built it according to spec, and the person who bolted it onto the craft also did so according to the specifications.

      Maybe we're just arguing about semantics, but I'd say all the flaws occured during the design process, not the implementation or assembly stages.

    15. Re:That is NOT correct. by wjwlsn · · Score: 1

      No steam. PBMR coolant (helium gas) feeds a gas-turbine directly. This is different from the previous generation of HTGRs, which used intermediate boilers to generate steam for a steam-cycle turbine.

      --
      Getting tired of Slashdot... moving to Usenet comp.misc for a while.
    16. Re:That is NOT correct. by Rei · · Score: 1

      What "already learnt (sic) solution" are you referring to?

      --
      Did you really name your son "Robert');DROP TABLE Students;--"?
    17. Re:That is NOT correct. by orac2 · · Score: 1

      That stripping missions down to the point where meaningful independent technical oversight is lost kills probes.

      --
      "Just once, I'd like to meet an alien menace that wasn't immune to bullets." -- The Brigadier, Dr. Who
    18. Re:That is NOT correct. by Rei · · Score: 1

      PBMRs typically contain water circuits as well for emergency coolant. Even if you don't have an emergency coolant circuit, some PBMRs are being designed to have a water circuit for hydrogen generation to power fuel cells.

      --
      Did you really name your son "Robert');DROP TABLE Students;--"?
    19. Re:That is NOT correct. by Rei · · Score: 1

      Then get people to quit complaining about cost! You can't have everything.

      Cheap or low risk - take your pick.

      --
      Did you really name your son "Robert');DROP TABLE Students;--"?
    20. Re:That is NOT correct. by orac2 · · Score: 1

      It not one of my comments have I complained about cost. Nor did Oberg in his article. You're shadowboxing.

      --
      "Just once, I'd like to meet an alien menace that wasn't immune to bullets." -- The Brigadier, Dr. Who
    21. Re:That is NOT correct. by _Sprocket_ · · Score: 1


      There are two design flaws right there. First, you shouldn't have to X-ray it to determine orientation, second that the drawings are incorrect.


      I agree on that point. And I pointed it out myself. Such components should be keyed so they can only be installed one way.


      So everything was assembled according to the techinical drawings.


      Absolutely not. If the system was assembled according to the technical drawings, they would not have been installed in reverse. The mistake wasn't a design that failed to work as expected. The mistake was an implementation that didn't follow the system design.

      From the article:

      Similar hardware has also been installed on NASA's Stardust probe, which has already collected fragments of a comet and is headed back to Earth with them. Engineers have studied design drawings of Stardust, which has a parachute re-entry system using the same switches as Genesis.

      Investigators into the Genesis crash believe that Stardust is assembled correctly -- if the documentation is accurate. "While the switches are the same, the installation ... is quite different," a NASA manager told journalists.

    22. Re:That is NOT correct. by antares256 · · Score: 1

      I would suggest that you read some books on this subject to become better informed. "The Secret of Apollo" by Stephen Johnson is an excellent book on Systems Management, the design of ICBMs and the Apollo program.

      Another good book on the subject of the failures of FBC (of which I believe Genesis was a part of) is one by Howard McCurdy "Faster, Better, Cheaper: Low-Cost Innovation in the U.S. Space Program"

      Why does NASA use "one off" designs for all of its work (eg, the Space Shuttle, space probes, etc)?

      Of corse NASA uses "one off" designs for most of their probes. Do you know of any "Mars probe widget making device" that ANY company has made? Think about it for a second, NASA does R&D/Science. You may be able to build a generic chassis, but tradeoffs must be made when designing any space application. All spacecraft have different power requirements, mass requirements, science requirements, etc. Unless you were swarming Mars with 500 ANTS type probes, chances are you're going to do a one-time design.

      Several of NASA's endeavours have been one design, many craft including: the Satrun V, MER, and the Space Shuttle (after a fashion, internals were upgraded with better technology when Endeavor came along).

      The DOD's GPS constellation is another example of one design, many off the line. Simply because they needed ~30 satellites the same, thereby decreasing the cost by creating a manufacturing line.

      And NASA doesn't balance cost and safety. First, they don't practice even the most rudimentary of cost controls. Namely, NASA still uses "cost plus" as the basis of most of its contracts.

      Of course they balance cost and safety, otherwise the agency would have been abolished long ago. Your argument that they still use "cost plus" doesn't hold water because of the reality of the situation. The Air Force has been using this system for years. Cost Plus is a necessity when working in a R&D area. You don't know exactly how much a design is going to cost, you can give a ball-park figure for the design (Cost), but it may take some extra time to complete because of various requirements that weren't made clear (Plus). In general, all Gov't contracts that are Cost Plus will put a limit on the Plus. No company would design a new craft (air or space) on a rigidly fixed budget, simply because it is extremely difficult to give a cost estimate and be right-on. Cost Plus helps to ensure that companies desiging/building the product will still make a profit, except in certain circumstances, and that's why companies are in business.

      In addition to that, they use the Systems Management approach which controls the costs as best as possible. To make a change after the design is approved, the engineer must justify the change, and balance the change versus the cost of implementation and the time it would require to implement.

      Second, NASA has come up with elaborate safety measures and procedures that were routinely bypassed by those who supposedly were performing the inspections.

      No, the vast majority of bypassers were managers. Take Challenger. The main reason this accident occurred is because managers at Thiokol pushed the Mute button on the phone during a teleconference with Marshall Space Flight Center during the Flight Readiness Review (FRR). Marshall didn't hear the Thiokol Engineers' reasoning behind making the claim that it was too cold to launch. They just heard the Engineers saying "No-go", a pause, and the Managers saying "Go".

      If you go back farther, the problem can also be traced to normilazation of the error state. O-Rings had been burning through since the first shuttle launch, but is was never a problem because the 2nd O-Ring always held. The Thiokol Engineers had data stating that as the temperature got colder, there was more O-Ring erosion. But there wasn't any data saying what the coldest temperature should be. It can also be traced to a roll-reversal because of Marshall managers/engin

    23. Re:That is NOT correct. by khallow · · Score: 1
      Hmmm, I see I made a slight mistake in characterizing the Space Shuttles as a one-off design since the design of the future shuttles were improvements (and by definition not "one-off"). The other craft you mention, are all one-off designs. Ie, even though two or more may have been made, the design was one-time.

      The commercial launch business also has a few standard designs and tested optional configurations that get used over and over again.

      Name a NASA design for a space probe or launch vehicle that gets used over and over again. The Russians have the Soyuz space capsule for example and decades of design improvements.

    24. Re:That is NOT correct. by khallow · · Score: 1
      I would suggest that you read some books on this subject to become better informed. "The Secret of Apollo" by Stephen Johnson is an excellent book on Systems Management, the design of ICBMs and the Apollo program.

      Another good book on the subject of the failures of FBC (of which I believe Genesis was a part of) is one by Howard McCurdy "Faster, Better, Cheaper: Low-Cost Innovation in the U.S. Space Program"

      I'll do so.

      Of corse NASA uses "one off" designs for most of their probes. Do you know of any "Mars probe widget making device" that ANY company has made? Think about it for a second, NASA does R&D/Science. You may be able to build a generic chassis, but tradeoffs must be made when designing any space application. All spacecraft have different power requirements, mass requirements, science requirements, etc. Unless you were swarming Mars with 500 ANTS type probes, chances are you're going to do a one-time design.

      Hmmm, Boeing, Beal, and Lockheed Martin come to mind as potential answers here. After all, I imagine that most satellite chasis are adequate for most space probes. And I just don't buy your argument. I doubt that requirements differ between missions so much that common designs can't be developed. In other words, just because NASA doesn't create reusable designs doesn't mean they can't be or shouldn't be done.

      Several of NASA's endeavours have been one design, many craft including: the Satrun V, MER, and the Space Shuttle (after a fashion, internals were upgraded with better technology when Endeavor came along).

      The DOD's GPS constellation is another example of one design, many off the line. Simply because they needed ~30 satellites the same, thereby decreasing the cost by creating a manufacturing line.

      I'm not clear what the point is here. The Shuttle (despite my erroneous statements to the contrary) is not a "one-off" design. But neither is its design being reused in any significant way. The current GPS constellation wasn't a one-off design either, and while being the best run program you mention here, wasn't designed by NASA, but rather by the DoD. You can bet that further GPS satellites will reuse the design features of their predecessors.

      Of course they balance cost and safety, otherwise the agency would have been abolished long ago. Your argument that they still use "cost plus" doesn't hold water because of the reality of the situation. The Air Force has been using this system for years. Cost Plus is a necessity when working in a R&D area. You don't know exactly how much a design is going to cost, you can give a ball-park figure for the design (Cost), but it may take some extra time to complete because of various requirements that weren't made clear (Plus). In general, all Gov't contracts that are Cost Plus will put a limit on the Plus. No company would design a new craft (air or space) on a rigidly fixed budget, simply because it is extremely difficult to give a cost estimate and be right-on. Cost Plus helps to ensure that companies desiging/building the product will still make a profit, except in certain circumstances, and that's why companies are in business.

      Two things here. First, you don't address my argument. If NASA were a private company that couldn't balance cost versus safety, then it would probably go bankrupt (eg, be "abolished") sooner or later from the lack of cost controls or from lawsuits by aggreived third parties. But as a government agency with little oversight over cost or safety until people die? Your point isn't valid. Yes, NASA can and is existing quite well despite its inabilities to contain costs or deal with safety issues. No abolishment to see here.

      Second, no private firm uses cost plus contracts in R&D. If this was the "only" way something could be done, then they would do it that way. Cost plus is just a means to extract more public funds.

      Hmmm, there is one final demonstration of NASA incompetence in the area of cost and safety. It's

    25. Re:That is NOT correct. by khallow · · Score: 1

      Ok, I have looked up the Library of Congress codes for the two books you mention and intend to read them.

    26. Re:That is NOT correct. by JuggleGeek · · Score: 1
      Why does NASA use "one off" designs for all of its work (eg, the Space Shuttle, space probes, etc)?

      Are you claiming that they made only one space shuttle? Or are you claiming that each one they did make only flew once?

      Doesn't matter - you clearly have no business posting about NASA, as you don't know anything about it.

    27. Re:That is NOT correct. by khallow · · Score: 1
      Doesn't matter - you clearly have no business posting about NASA, as you don't know anything about it.

      Yes, I'm now aware of my mistaken use of the term "one-off". But your childish remark is annoying and irrelevant.

    28. Re:That is NOT correct. by JuggleGeek · · Score: 1

      Your claims that the shuttles are one-shot deals show that you don't have a clue. You may not like having that poitned out, but in fact, that is clear evidence that either you don't understand even the basics of the subject being discussed, or else that you are willing to tell blatent lies in order to support your argument.

    29. Re:That is NOT correct. by khallow · · Score: 1
      Your claims that the shuttles are one-shot deals show that you don't have a clue. You may not like having that poitned out, but in fact, that is clear evidence that either you don't understand even the basics of the subject being discussed, or else that you are willing to tell blatent lies in order to support your argument.

      I misunderstood what was meant by the term "one-off" and didn't bother to look up the term before I originally posted.

      My beef with the Shuttle is that as a vehicle, the design is a dead end. There won't be a sixth shuttle or a six vehicle that borrows major aspects of the Shuttle design. Parts like the tiles, solid boosters, etc can be reused, but the overall vehicle won't be upgraded. In comparison, the Soyuz capsule has been steadily improved over several decades and the design has been adopted by the Chinese as well.

  9. Isn't that... by computational+super · · Score: 3, Funny
    NASA management still often chooses the latter.

    Why be different than any other management?

    --
    Proud neuron in the Slashdot hivemind since 2002.
    1. Re:Isn't that... by EvilTwinSkippy · · Score: 1

      If a business manager fails, he gets fired. If an engineer fails, a few thousand people could die or be horribly injured.

      --
      "Learning is not compulsory... neither is survival."
      --Dr.W.Edwards Deming
  10. NASA deserves more chances at failure by Anonymous Coward · · Score: 2, Insightful

    Human error is an inevitable input to any complex endeavor. Either you manage and design around it or fail. NASA management still often chooses the latter.

    There's a contradiction in that above statement .. but I cant think what it is exactly. Along lines of human manages and designs the error handling dont they?

    That said nothing wrong with building in redundancy and failsafes

    In space probes redundancy comes at the cost of number of unique mission goals and financial cost.

    Sometimes you just have to eat the failure, thats what insurance is for. We in the public shouldn't always expect NASA to have 100% failure free (non human) missions and then extract harsh punishment on them which invariably gets passed down to engineers and not management decision makers.

    Wuith the current attitude NASA of old would have been shutdown in the first couple of years for wasting tax payer money. Luckily there was competition with Soviets.

    1. Re:NASA deserves more chances at failure by orac2 · · Score: 1

      No-one, least of all Oberg (a 22-year veteran of mission control), is asking NASA to have a 100% success rate. Space is harsh, unknown unkowns lurk, etc.

      What is is calling for is a management structure that allows solutions to problems that have occured before to be implemented properly. Columbia was destroyed for almost the same root causes that were exposed after Challanger. I don't think it's unreasonable to expect people to have elimiated those problems, and kept them eliminated.

      The Columbia Accident Board had some harsh things to say about "Cheaper, Faster, Better" and NASA technical oversight in general, you should read their report.

      --
      "Just once, I'd like to meet an alien menace that wasn't immune to bullets." -- The Brigadier, Dr. Who
    2. Re:NASA deserves more chances at failure by Rei · · Score: 1

      > Columbia was destroyed for almost the same root causes that were exposed after Challenger.

      Pray tell, what problems are you referring to? Certainly not technical - Challenger was caused by a failure of an O-ring on an SRB, while Columbia was caused by foam striking the RCC leading edge of the shuttle.

      Do you mean management issues? You'd have to take a really vague definition to get that. Challenger was caused mainly by two things: 1) the not having, at decision time, every study done on the SRBs (one of which determined O-ring failure at low temperatures, and 2) improperly utilizing statistics (they omitted successes from the graph of O-ring failures vs temperature, and only plotted failure-cases; when you include successes, you find that there wasn't a single low-temperature launch that was completely successful). Columbia, on the other hand, was mainly the fault of 1) improper assumptions about the impact foam would make on RCC (due to a lack of ballistics experience on the team), and 2) not elevating worker concerns up the chain effectively.

      How, exactly, are these related? BTW, if you want to bash "Cheaper, Faster, Better", don't complain about costs.

      --
      Did you really name your son "Robert');DROP TABLE Students;--"?
    3. Re:NASA deserves more chances at failure by orac2 · · Score: 1

      Have you read the CAIB report? The issues are managerial and frighteningly similar. I direct you to Chapter 8 of the report, summarized in the table of contents thus:

      8. History as Cause: Columbia and Challenger
      8.1 Echoes of Challenger
      8.2 Failures of Foresight: Two Decision Histories and the Normalization of Deviance
      8.3 System Effects: The Impact of History and Politics on Risky Work
      8.4 Organization, Culture, and Unintended Consequences
      8.5 History as Cause: Two Accidents.
      8.6 Changing NASA?s Organizational System

      The report is excellently written and present and organized and available as free download.

      --
      "Just once, I'd like to meet an alien menace that wasn't immune to bullets." -- The Brigadier, Dr. Who
    4. Re:NASA deserves more chances at failure by Rei · · Score: 1

      Don't just cite chapter headers; cite what you're referring to as being similar to Challenger, and give a reference if need be. I just discussed both the technical and management problems that led to each disaster; your goal, now, is to cite why I am wrong. That's how debates work.

      --
      Did you really name your son "Robert');DROP TABLE Students;--"?
    5. Re:NASA deserves more chances at failure by orac2 · · Score: 1

      Christ almighty -- are you really so lazy that you can't take it upon yourself to read the analysis of NASA, despite getting worked up enough about to spout off on /.?

      Fine, sluggo, here you go (I've highlighted the extra-relavent bits for those who require spoonfeeding):

      P. 195 CAIB report Vol 1.

      The Board began its investigation with two central questions about NASA decisions. Why did NASA continue to fly with known foam debris problems in the years preceding the Columbia launch, and why did NASA managers conclude that the foam debris strike 81.9 seconds into Columbia?s flight was not a threat to the safety of the mission, despite the concerns of their engineers?

      As the investigation progressed, Board member Dr. Sally Ride, who also served on the Rogers Commission, observed that there were "echoes" of Challenger in Columbia. Ironically, the Rogers Commission investigation into Challenger started with two remarkably similar central questions: Why did NASA continue to fly with known O-ring erosion problems in the years before the Challenger launch, and why, on the eve of the Challenger launch, did NASA managers decide that launching the mission in such cold temperatures was an acceptable risk, despite the concerns of their engineers? The echoes did not stop there. The foam debris hit was not the single cause of the Columbia accident, just as the failure of the joint seal that permitted O-ring erosion was not the single cause of Challenger. Both Columbia and Challenger were lost also because of the failure of NASA?s organizational system. Part Two of this report cites failures of the three parts of NASA?s organizational system. This chapter shows how previous political, budgetary, and policy decisions by leaders at the White House, Congress, and NASA (Chapter 5) impacted the Space Shuttle Program?s structure, culture, and safety system (Chapter 7), and how these in turn resulted in flawed decision-making (Chapter 6) for both accidents. The explanation is about system effects: how actions taken in one layer of NASA?s organizational system impact other layers. History is not just a backdrop or a scene-setter. History is cause. History set the Columbia and Challenger accidents in motion. Although Part Two is separated into chapters and sections to make clear what happened in the political environment, the organization, and managers? and engineers? decision-making, the three worked together. Each is a critical link in the causal chain. This chapter shows that both accidents were "failures of foresight" in which history played a prominent role.1 First, the history of engineering decisions on foam and O-ring incidents had identical trajectories that "normalized" these anomalies, so that flying with these flaws became routine and acceptable. Second, NASA history had an effect. In response to White House and Congressional mandates, NASA leaders took actions that created systemic organizational flaws at the time of Challenger that were also present for Columbia. The final section compares the two critical decision sequences immediately before the loss of both Orbiters - the pre-launch teleconference for Challenger and the post-launch foam strike discussions for Columbia. It shows history again at work: how past definitions of risk combined with systemic problems in the NASA organization caused both accidents. Connecting the parts of NASA?s organizational system and drawing the parallels with Challenger demonstrate three things. First, despite all the post-Challenger changes at NASA and the agency?s notable achievements since, the causes of the institutional failure responsible for Challenger have not been fixed. Second, the Board strongly believes that if these persistent, systemic flaws are not resolved, the scene is set for another accident. Therefore, the recommendations for change are not only for fixing the Shuttle?s technical system, but also for fixing each part of the organiz

      --
      "Just once, I'd like to meet an alien menace that wasn't immune to bullets." -- The Brigadier, Dr. Who
    6. Re:NASA deserves more chances at failure by Rei · · Score: 1

      Christ Almighty yourself - will you learn some basic rules of how to debate? You *don't* tell a person to debate with a publication that they don't think backs up your point.

      > Why did NASA continue to fly with known O-ring
      > erosion problems in the years before the
      > Challenger launch

      Absolutely nothing to do with the recent case.

      > and why, on the eve of the Challenger launch,
      > did NASA managers decide that launching the
      > mission in such cold temperatures was an
      > acceptable risk, despite the concerns of their
      > engineers?

      Engineers always have concerns, so this is really a non-sequiteur. The engineers, nonetheless, signed off in both cases that it was an acceptable risk.

      > The echoes did not stop there.

      Seing as they didn't start yet, I'd hope not.

      > Both Columbia and Challenger were lost also
      > because of the failure of NASA?s
      > organizational system.

      Unless you can cite specifics, I don't want to hear it from you.

      > both accidents were "failures of foresight" in
      > which history played a prominent role.

      All accidents are failures of foresight. You need a cite if you want to claim, however, that the "history" was related between the two accidents.

      > First, the history of engineering decisions on
      > foam and O-ring incidents had identical
      > trajectories that "normalized" these
      > anomalies, so that flying with these flaws
      > became routine and acceptable.

      That's laughable. Every rocket in the world has hundreds of known risk factors. If you don't "normalize" them, you'd never get to space. Most of them would require the redesign of the craft in question - for example, one "normalized" problem for the shuttle is that the engines aren't mounted directly linear with the bulk of the shuttle's mass, which increases the vibrational load, increasing wear on all parts and their risk of failure. Should they scrap the shuttle just for that? Every rocket has similar risk factors.

      > NASA leaders took actions that created
      > systemic organizational flaws at the time of
      > Challenger that were also present for
      > Columbia.

      Once again, without specifics, I can't debate this with you (remember, I want to discuss things with *you*, not with a paper that can't talk back. The paper isn't the one posting its opinions on Slashdot, you know.).

      > Connecting the parts of NASA?s organizational
      > system and drawing the parallels with
      > Challenger demonstrate three things. First,
      > despite all the post-Challenger changes at
      > NASA and the agency?s notable achievements
      > since, the causes of the institutional failure
      > responsible for Challenger have not been
      > fixed.

      Again, with a lack of specifics, I can't debate this point with you.

      --
      Did you really name your son "Robert');DROP TABLE Students;--"?
    7. Re:NASA deserves more chances at failure by orac2 · · Score: 1

      You *don't* tell a person to debate with a publication that they don't think backs up your point.

      Here's the point: you don't know if the CAIB report does or not back up my point, because you haven't bothered to read it, despite it being excellently written and presented and freely available. You don't get to have an opinion, or a debate, about something you know nothing about. You're like someone commenting about long division who still hasn't got past the counting on their fingers stage.

      I'm not going to debate to entire CIAB report with you (which backs up the summary statements I quoted in great detail). If you want specifics, they're right there. I'm sorry this topic is too complex to be reduced down to what I can summarize into a /. comment box, but that's just they way the real world is sometimes.

      Now, if you seriously believe the CAIB is has made errors of fact and judgement as you claim, you have a duty to a) read the report and b) issue a rebuttal. The paper can't talk back, but the CAIB report wasn't handed down on tablets to Moses. All the board members are real walking people, and a huge community of professionals are out there who are really interested in meaningful commentary on the CAIB report, including myself.

      In the meantime, given your obvious igorance of even secondary source material, such as the CAIB, I'll take the board's recommendations and judgements over yours: I would note as well that NASA accepted these findings, as did nearly everyone else on the planet qualified to evaluate it.

      --
      "Just once, I'd like to meet an alien menace that wasn't immune to bullets." -- The Brigadier, Dr. Who
    8. Re:NASA deserves more chances at failure by Rei · · Score: 1

      Yes, I *have* read it. But I'm not debating it with itself; I'm debating it with you. That is why you need to cite what you want to claim from it.

      If you're not going to debate it with me, pick some other line of evidence to back up your claims that it's the same faults (which I already evidenced, by listing the cause of the faults, it distinctly is *not*). If you won't pick another line of evidence to debate, then kindly go post elsewhere. I'm not fond of debating with a wall.

      --
      Did you really name your son "Robert');DROP TABLE Students;--"?
    9. Re:NASA deserves more chances at failure by orac2 · · Score: 1

      Yes, I *have* read it

      Really? Then why do you keep demanding specifics, as if they don't exist, when you know they're a few pages away in the CAIB report? I've quoted directly from the report -- why haven't you?

      Debates don't occur in vacuums, otherwise they're meaningless symbolic manipulations. You must introduce some real evidence, as actual debators do all the time: it's not just about rhetorical flourishes.

      Extraordinary claims require extraordinary evidence. In this case, this means that disagreeing with the almost universally accepted findings and judgements of the CAIB requires you to bear the burden of evidence.

      So let's hear it. Let's start slow. Pick one specific finding with it's supporting evidence as presented in the CAIB report and tell me, precisely, why it's wrong. Bear in mind, I will expect you to either have examined the source material the CAIB relied on (also available online), or to have the results of an independant investigation of equal quality, otherwise you're asking me to accept a he-said she-said, and given the qualifications of the CAIB, I'll go with what they-said.

      This isn't a game. There are no debate points. Just actual lives.

      --
      "Just once, I'd like to meet an alien menace that wasn't immune to bullets." -- The Brigadier, Dr. Who
    10. Re:NASA deserves more chances at failure by Rei · · Score: 1

      > Then why do you keep demanding specifics ... because I'm debating *YOU*, not the paper. I can't debate a point if you don't state it. I want you to state something from the report that you agree with it, so that we can do point-counterpoint. How do you expect to hold a debate otherwise?

      > You must introduce some real evidence

      What do you think I did on my first post on the subject, where I discussed the O-ring failures (and management failures behind them) in comparison to the RCC damage (and management failures behind it)? You haven't addressed it at all!

      > Pick one specific finding ...

      You don't debate findings, you debate evidence. The findings are the summary of a broad range of things. You can't debate a broad, diverse range - you can only debate specifics. I presented specifics, which you have refused to debate.

      > There are no debate points. Just actual lives

      And tens of thousands of people whose work you're smearing without discussing what actually happened to cause the losses of lives (the O-ring failures/associated decisions, and the RCC impact/associated decisions).

      --
      Did you really name your son "Robert');DROP TABLE Students;--"?
    11. Re:NASA deserves more chances at failure by sexylicious · · Score: 1

      Isn't orac2's point that the management still decided to fly, even though they had evidence in both cases that would have made them shout "No go!"?

      Isn't the fact that the management over there always takes the attitude that they're right part of the problem in both cases (I have a co-worker that used to work at Dryden).

      Isn't it a fact that the safety manager for the Columbia mission gave a stamp of approval, despite her engineers' concerns that the impact was worse than thought? Also, isn't it a fact that this same person DECLINED to request or even consider DoD satellite imagery that would have confirmed the extent of the damage?

      Isn't it also true that the safety process broke down for Challenger because managers decided to NOT listen to their engineers?

      Now, isn't it true that in both cases, MANAGEMENT didn't heed their engineers' warnings? Isn't it also true that they neglected (or conveniently forgot) about mission critical information that, had they had it in front of them, would have stated that there was a huge problem that should be investigated?

      It is true as I see it that both accidents were caused by poor management decisions. NOT a faulty part (say a part that got through QA when it should have been screened out).

      I think you're mistaking the technical differences between the accidents with the managerial differences. In both cases, management pushed ahead, even when some information was stuffed away somewhere that would have stopped the go ahead decision. NASA's problem, as orac2 is trying to state, is that they don't remember their mistakes... they're not propagating the information about their mistakes properly.

      I'm an engineer myself (actually a true rocket scientist by training and trade), and I can understand the occasional calculation error, or the occasional bad call about assumptions. But when it comes to a manager, who should know something about management, who decides that he/she should press on with the mission without giving pause for previous failures, then that manager should be removed or retrained until they get it right. (It's kind of like president Bush's "we will stay the course" crap, when what's happening now clearly isn't doing the job in Iraq.)

    12. Re:NASA deserves more chances at failure by orac2 · · Score: 1

      What sexylicious said. (Damn work for interfering with /.!)

      --
      "Just once, I'd like to meet an alien menace that wasn't immune to bullets." -- The Brigadier, Dr. Who
    13. Re:NASA deserves more chances at failure by Rei · · Score: 1

      The go/no-go decision on Challenger was actually made by a board of engineers. There were dissenters, but everyone signed off. They signed off for the reasons that I stated: 1) they were missing a document that they should have had, and 2) the graph that they used plotting O-ring failures and temperatures was made from improper use of statistics. There was some political pressure to launch, of course, but most of that was just passed down from congress. If you'll remember, Rutan too launched in bad conditions (winds gusting over 40mph), and nearly lost SS1 due to it, during one of their test flights. Political pressure can do nasty things to judgement, and most of the engineers thought that the data all looked good. It wasn't.

      In the case of Columbia, *an* engineer (i.e, one - Robert Dagherty) warned that there might be a problem from the impact. He was quite adminant, too, but that doesn't change the fact that he was the only one pushing the concerns, among a huge team of engineers - concerns of a problem that they realistically couldn't have fixed (i.e., he didn't express the concerns before the impact, only after). Before the launch, the foam issue was pretty thoroughly discounted as being irrelevant.

      --
      Did you really name your son "Robert');DROP TABLE Students;--"?
    14. Re:NASA deserves more chances at failure by orac2 · · Score: 1

      No, you haven't presented evidence, merely whined about the absence of specifics in my posts, despite the fact that I have refered you to the relavant portions of a generally accepted report, something which should be a problem dealing with as you claim to have already read the report.

      Debating evidence means debating the evidence detailed in the CAIB report -- which you have failed to do, despite me asking you to start by detailing your objections to just one finding and its suppporting evidence.

      And tens of thousands of people whose work you're smearing without discussing.

      I'm smearing no-one. I haven't mentioned blame, beyond managerial failures. But there is (or at least was at time of the Columbia disaster) a dysfunctional culture in NASA, and if it causes anyone in NASA offence to hear that, tough shit. NASA's defensiveness in dealing with external criticism was also cited by the CAIB as a contributing factor to the whole ball of wax that killed Columbia and her crew. And Challenger and hers.

      Bluntly: NASA employees who won't accept the message of the Columbia report (in defiance of the general consensus amongst O'Keefe's administration, the White House, Congress, the aerospace industry and academia that it paints a fair portrait of the agency and the causes of the Columbia disaster) should be fired. Because otherwise the Roger's Report and the CAIB report will be joined by a third in time to come.

      --
      "Just once, I'd like to meet an alien menace that wasn't immune to bullets." -- The Brigadier, Dr. Who
    15. Re:NASA deserves more chances at failure by orac2 · · Score: 1

      Before the launch, the foam issue was pretty thoroughly discounted as being irrelevant.

      Sigh. You just don't get it do you? The fact that the foam issue was "thoroughly discounted" is exactly what we're talking about when say there was a management failure. Foam shedding was an occurance that was outside the safety envelope of the shuttle from Day 1. Yet, over time, it became seen as less and less of a problem simply because nothing awful had happened (yet). This is an example of what is meant by "normalization" and is an example of bad management.

      The exact same thing happened with Challenger, where O-Ring charring similary went from a serious problem to something everyone "discounted as irrelevant".

      Two vehicles, one root cause.

      --
      "Just once, I'd like to meet an alien menace that wasn't immune to bullets." -- The Brigadier, Dr. Who
    16. Re:NASA deserves more chances at failure by sexylicious · · Score: 1

      But the common thread to both of your paragraphs is that NASA had some info tucked away somewhere and didn't have it out.

      Though I do agree with the fact that NASA didn't really explore the foam impact until after Columbia.

      It's still true that NASA declined DoD imagery which would have easily spotted the shuttle's damage. Then at least they would have known. Though there was no way for the shuttle's crew to do anything about it, really... the next shuttle wasn't ready, and the next resupply vehicle was a month away. But they would have had better information about the problem, which is what NASA's management problem is.

      NASA managers seem to put too much faith in their QA process. NASA managers also have the whole attitude of "I've been here XX years, I know what's going on!" or "I have 9 engineers telling me 'GO' and one saying 'NOGO!'; 9 in favor... it's a GO!"

      Those attitudes pervade EVERY program over at NASA. Just look at the swedish engineer that found the doppler shift problem with the Hyugens probe. JPL just accepter the italian contractor's work without question. That's something that a good manager would NEVER do.


      NASA also has the whole backstabbing, take your work and call it my own, ALWAYS put a good face on me attitudes amongst most of their managerial staff. That kind of thing is counter-productive especially when they are doing primarily R&D or T&E work.

    17. Re:NASA deserves more chances at failure by sexylicious · · Score: 1

      Exactly!

      In both cases, the managers became complacent. It's the same idea as, "the sun rose yesterday, and today, it's GOING to rise tomorrow!" There is no evidence that says the sun WILL rise tomorrow; only evidence that says that it has, and that it's very likely that it will again.

    18. Re:NASA deserves more chances at failure by Rei · · Score: 1

      Quite good points - and first off, thank you for actually bringing issues up for discussion instead of assuming that I can read your mind and pointing me to something that I've already read :)

      However, don't you think it's fair to say that "9 engineers say 'GO', and one says 'NOGO'", that if the one feels willing to sign off that everything is OK (Challenger), or the others are insistant that the one is wrong (Columbia), that there's not much you can do without creating a policy that would make you do so many checks that all your work would grind to a standstill? I worry that's what's going to happen now.

      As for the issue of accepting the Italian company's work, It's pretty hard for NASA to sign an NDA. And really, NASA was only a minor partner for Cassini; you really need to address Cassini issues to the ESA.

      --
      Did you really name your son "Robert');DROP TABLE Students;--"?
    19. Re:NASA deserves more chances at failure by orac2 · · Score: 1

      Part of the problem is that frequently management now demands consensus (I'm sure you'll remember the discussion of that point in the CAIB report). Minority reports and formal dissent became a thing of the past.

      In NASA what was happening was that that the onus was placed on dissenting engineers to prove that a situation was unsafe before action was taken. While that might seem like a perfectly reasonable approach in normal life it is lethal in a high risk enterprise like spaceflight. There, once a risk is identified, the onus is those who must prove the situation is safe.

      To quote the CAIB, specifically, p 190:

      When managers in the Shuttle Program denied the team?s request for imagery, the Debris Assessment Team was put in the untenable position of having to prove that a safety-of-flight issue existed without the very images that would permit such a determination. This is precisely the opposite of how an effective safety culture would act. Organizations that deal with high-risk operations must always have a healthy fear of failure - operations must be proved safe, rather than the other way around. NASA inverted this burden of proof.

      --
      "Just once, I'd like to meet an alien menace that wasn't immune to bullets." -- The Brigadier, Dr. Who
    20. Re:NASA deserves more chances at failure by Rei · · Score: 1

      And as I already mentioned, which you (as usual) didn't respond to, there are dozens or hundreds of things with every rocket ever made that are just discounted, because without discounting them, you really can't fly.

      For example, my mention of the positioning of the SSMEs. They're not aligned with the fuel tanks (and the bulk of the mass), so the net result is a higher vibrational load. Vibrational loads are bad - every rocket has it, but they increase wear and chances of part failure. Should they ground the shuttle fleet and redesign the whole system to relocate the SSMEs? They haven't caused a problem in 100 flights; it seems that the vibrational load is "within acceptable parameters".

      The same was thought about the foam. It was wrong. Can one reasonably support grounding the shuttles to fix the foam problem before something went wrong, but not doing the same for the vibrational load problem?

      Then factor in that there are *dozens or hundreds of things* like the vibrational load issue left on the shuttle (and every rocket in the world). You'd basically ban humans from spaceflight by pursuing such a policy.

      As for O-rings, the probability of O-ring failure was known, in normal conditions, to be around 5%. There were two O-rings for each section of the SRB; There were 6 field joints. Each O-ring acted redundantly (in normal circumstances). So, each joint had a 0.25% chance of failure. That means that the probability of a total SRB failure is about 1.5%. While this may seem high, and it was a concern, such odds aren't extreme as far as rocketry goes. There was an alternative ring design with a capture lip under testing with Hercules, but it was just that - experimental. NASA had ordered some for the SRBs, but they would take several years to produce (and consequently, were too late). They now use the capture-lip design.

      --
      Did you really name your son "Robert');DROP TABLE Students;--"?
    21. Re:NASA deserves more chances at failure by Rei · · Score: 1

      (O-rings - cont'd)

      Of course, as I mentioned, the probability of O-ring failure increased at low temperatures, as did an effect that reduced the redundancy of the second O-ring; however, they didn't have the document available to them at the time they made the decision (one document among hundreds)

      --
      Did you really name your son "Robert');DROP TABLE Students;--"?
    22. Re:NASA deserves more chances at failure by sexylicious · · Score: 1

      orca2 stated well the issue with not acknowledging the 10th engineer in his reply to your post.

      The main thing that NASA's management forgot about the shuttle is that it was a EXPERIMENTAL vehicle. Even though it had a proven track record, it was still experimental. I now work in an environment that involves experimental stuff. And I'll tell you what... we make damn sure that all of the safety issues are addressed. We even have people unrelated to the program, but still proficient in areas related to the program, come in and examine how we go about test safety. Even though a lot of the stuff that we do has never been done before, we still make sure as much as we can that nothing will happen. We'd rather have our test personnel come back, and lose the experiment or lose data, than see our test personnel come back in a body bag.

      That's part of the issue here... NASA didn't want to scrub the mission for fear of losing money and face, so they lost money and life.

      As for the Italian company's work, NASA signs NDAs all the time. And JPL was where mission control was for Cassini, they had a powerful voice in the mission (and still do).
      As for NDAs... At least at my former employer, we did TONS of NASA contracts and each time NASA had no problem signing an NDA. They are a bit different when the government is involved, but the government CAN'T take your company's technology/idea and go to another company and develop it cheaper, or publish your data (it's proprietary). There are strong laws against that.



      In all seriousness, NASA has a way of pulling through. They'll figure out how to implement proper safety procedures and checks. And they'll figure out how to do those safety items without stalling their programs. I'm actually very suprised that NASA would forego the lives of its astronauts by NOT implementing good safety measures, even on experimental vehicles.

    23. Re:NASA deserves more chances at failure by orac2 · · Score: 1

      As I've stated in an another thread, when it comes to risking lives, the burden is to prove that something is safe, not that it is unsafe.

      This is the fundamental point that makes management of high-risk enteprises different from low-risk management.

      If the vibrational load problem has not been proven to be safe, or the risks quantitatively characterised, then the shuttle should be grounded until such a demonstration can made. The thinking behind "here's something we haven't checked out 100%, but nothing bad has happened so far, so it's okay"is exactly the thinking that killed the seven astronauts on board Challenger and the seven astronauts onboard Columbia.

      The point is not to eliminate all risks -- but if the risk is known and characterized, all concerned can make informed decisions about it. Risk is part of spaceflight, and no-one is saying there should be no spaceflight until the risk of failure is zero. But to allow anomolies to persist without fully investigating what was going on, without understanding the modes and consequences of failure and without establishing quantitatively whether or not the risk involved is acceptable is bad management. It killed Challenger and it killed Columbia, and if NASA can't snap out of thinking that way, it'll kill another vehicle and her crew.

      --
      "Just once, I'd like to meet an alien menace that wasn't immune to bullets." -- The Brigadier, Dr. Who
    24. Re:NASA deserves more chances at failure by Rei · · Score: 1

      > the burden is to prove that something is safe

      I guess we're not going to resolve this unless I pose it to you as a direct challenge:

      Given:
      There Are Dozens Of Things For Every Rocket That Could Potentially Be Catastrophic, That Pose As Much Or More Of A Threat Risk Than The Falling Foam Did.

      Then:
      How can you possibly justify *any* manned spaceflight without redoing every single rocket in existance (and their successors, and their successors, etc, until we finally have a rocket without any major risk factors - something that currently looks to be impossible)? Just like falling foam, we can't say whether the vibrational loads will ever cause a problem - there isn't any way to know. But they *could*.

      Should we ground all rockets? I don't think we can continue until you address that.

      --
      Did you really name your son "Robert');DROP TABLE Students;--"?
    25. Re:NASA deserves more chances at failure by sexylicious · · Score: 1

      But we can say whether vibrational loads will ever cause a problem. Vibrational testing is one of the hot "test items" that is performed for launch vehicles. Many, if not all, launch vehicles are modeled, simulated, tested, and characterized before they ever get fueled and launched.

      The vibrational problem is well understood. To give an example, active force-feedback flight control sticks are used in the Apache. One of their tests is a vibration test. As part of that vibration testing, all conceivable vibration inputs are considered. The big ones are: engine noise, sustained impacts from enemy munitions, and firing of the apache's weapons with the emphasis on the machine gun under the cockpit. A vibration on the pilot's seat caused by the pilot farting isn't really considered because it's understood that such a vibrational input has very little impact on the Apache's flight controls (unless it tickles, I guess). But it's understood that the fart won't do much because the system is almost fully characterized (testing will help to fully characterize it).

      All rockets shouldn't be grounded. I'm a bit biased on that, since I'm a rocket scientist. But it seems clearly reasonable to make sure that the rocket is understood as much as possible so suprise situations don't arise often.

      As for the falling foam being understood by NASA, ANY person with a background in fluid mechanics would have pointed out the impulse that such an impact could impart to the wing would be much greater than anticipated. That's a management issue too: not putting the right people in place, or not consulting with the right people to get the answers.
      No offense to any mechanical engineers, but I believe that the first thoughts concerning the foam were put forth by mechanical engineers with NO experience with aerodynamics. If I recall correctly, the prevailing thought was, "foam is soft, it'll just bounce off".

    26. Re:NASA deserves more chances at failure by orac2 · · Score: 1

      True unknown unknowns are part and parcel of every flght. This is the irreducible risk that must be accepted.

      But when an unknown unknown becomes a known unknown, i.e. you spot something (such as foam shedding or O-ring charring) that you either didn't expect, don't fully understand why its happening, how dangerous it is, or what the implications are then you are grounded, until you demonstrate you have at least characterized the problem.

      Understanding that distinction between risk and recklessness is to understand how to manage high-risk enterprises.

      --
      "Just once, I'd like to meet an alien menace that wasn't immune to bullets." -- The Brigadier, Dr. Who
    27. Re:NASA deserves more chances at failure by sexylicious · · Score: 1

      Which is management's fault. They should have had those documents at the forefront of their minds (if they related to safety issues).

      I understand that the shuttle is one of the most complicated things ever built. But the safety process can't stop because something is complex.

    28. Re:NASA deserves more chances at failure by orac2 · · Score: 1

      Here's another way of thinking about it. I'm the Administrator of Blobia's Space Agency:

      Case One:

      My engineers come to me and say: we'd like to launch a manned rocket. We think the chance of fatal failure is about 1 in 500, but the astronauts are okay with that. We came up with the figure by considering the following risks: the chance of the main LOX tanks bursting is XX%, the chance of a total computer failure is XX%. (and so on). I turn the list of risks over my independant safety team who go over the list to make the calculations are okay, but who also checks previous flight and ground test records to make sure nothing's been left off the risk. Nothing has and I okay the launch. The next day the rocket takes off and kills the astronaut onboard.

      Case Two:

      The engineers come to me and present their list. Unbeknowst to me, a number of things have been left off the list for varous reasons, but I don't bother getting anyone to double check the list: if they did, they would have told me the calculabe risk of failure should be pegged at 1 in 50. I give the okay, the rocket takes off and the mission is a total success.

      Case One is an example of good management and how it should go. That the astronuat happened to be killed is sheer bad luck, but I'd sleep well authorizing another dozen such missions under the same circumstances. Case Two (which is closer to the NASA described in the CAIB report) is an example of bad management. That the astronaut happened to survive is sheer good luck.Not a single additional mission should be okayed until the circumstances are changed.

      See?

      --
      "Just once, I'd like to meet an alien menace that wasn't immune to bullets." -- The Brigadier, Dr. Who
    29. Re:NASA deserves more chances at failure by Rei · · Score: 1

      Expecting them to be able to keep track of every last document at all times isn't at all realistic. And expecting an engineering board to be able to go through every document ever gathered about the shuttle during a launch prep is equally unrealistic.

      --
      Did you really name your son "Robert');DROP TABLE Students;--"?
    30. Re:NASA deserves more chances at failure by Rei · · Score: 1

      Then all rockets are grounded. There are many things fully understand about rocketry. Heck, even coming up with heating models is pretty hit or miss, because there's no way to implement a true turbulence model for Reynolds-averaged Navier-Stokes equations (there are a number of approximations out there, but they all tend to have glaring weaknesses, especially in predicting the start and length of the transition region). And without a model (or a football-field sized wind tunnel), the only way we can tell what will happen is to launch.

      Is that acceptable to you, at all - grounding all manned rockets?

      --
      Did you really name your son "Robert');DROP TABLE Students;--"?
    31. Re:NASA deserves more chances at failure by orac2 · · Score: 1

      You're pointing out problems that are known, and therefore tesst can be performed to empirically characterize the siutation and educated judgements can be made about the risks involved.

      It's in consciously or unconciously believing the problems don't exist or are insignificant without any supporting evidence (as happened with the foam and O-rings) that you end up on the Dark Side. We don't require a Theory of Everything to do rocketry, but we can't bury our heads in the sand when faced with anomalies either.

      --
      "Just once, I'd like to meet an alien menace that wasn't immune to bullets." -- The Brigadier, Dr. Who
    32. Re:NASA deserves more chances at failure by orac2 · · Score: 1

      Actually keeping track of every last document is exactly what I expect management to do. Apollo was famous for its ability to manage documentation in a time before ubiquitous computer-aided IT. And I expect an independant safety organization to keep on top of anomalies and sort the wheat from the chaff so that exactly these issues bubble to the top during launch prep.

      I'm not alone in that expectation, nor is it unreasonable.

      --
      "Just once, I'd like to meet an alien menace that wasn't immune to bullets." -- The Brigadier, Dr. Who
    33. Re:NASA deserves more chances at failure by Rei · · Score: 1

      How, exactly, do you propose to simulate the effects of the shuttle's vibrational load? At the bare minimum, I can't think of a single turbulence model for Reynolds-averaged compressible flows that takes vibration into account, despite the fact that we know that it plays a significant role in shock formation, laminar->turbulent flow transition (and consequently heating), etc. Furthermore, how do you propose that the effects on all parts of the shuttle - from the fluid in tanks and valves, to the effect of vibration on every little weld - be simulated computationally? Sure, you can test each part, but testing the integration of functioning parts is no easy task.

      > ANY person with a background in fluid mechanics would have pointed out the impulse ...

      Actually, it's not that simple. Given some basic assumptions (density of foam=100kg/m^3, and that we're talking about perhaps 1/20th of a cubic meter of foam impact per cubic meter of impact area, that's 5kg of foam impacting. Even if it gets accelerated to 300m/s or so, it still doesn't have very much energy when it hits. The only way that it could possibly do damage is to have an incredibly rapid dissipation of energy into the RCC. Even after the accident, there was still a lot of doubt - with good reason - that this could happen, until they did an actual physical test.

      --
      Did you really name your son "Robert');DROP TABLE Students;--"?
    34. Re:NASA deserves more chances at failure by orac2 · · Score: 1

      there was still a lot of doubt - with good reason - that this could happen

      Actually the CAIB was pretty convinced, on the basis of its own analysis, an analysis that should have taken place at NASA years before that the foam could cause massive damage. The test had as much to do with finally getting NASA people to accept that reality - I can't find a direct link, but this is described in William Langewiesche's award winning article in The Atlantic Monthly about the investigation. That NASA personnel believed foam was safe based on nothing more than a collective hunch is exactly the kind of problem that I, Oberg, the CAIB, sexylicious and a dozen other task groups and reports are all railing against.

      --
      "Just once, I'd like to meet an alien menace that wasn't immune to bullets." -- The Brigadier, Dr. Who
    35. Re:NASA deserves more chances at failure by sexylicious · · Score: 1

      How, exactly, do you propose to simulate the effects of the shuttle's vibrational load?

      There is a NASA owned program called STARs that can do exactly that.

      As for a reynolds-averaged turbulence model... there isn't one that takes vibration into account because CFD guys aren't concerned with aeroelastic effects. That's considered a structural dynamics problem.


      And for the foam impact, you don't look at KE when doing impact analysis. You look at maximum momentum transfered, time-averaged momentum transfered, and the structural response to a shock load. KE just gives a useful tool for measuring the energy content of your projectile. The impact analysis requires looking at the momentum imparted to the second body during the contact time. You can have something travelling with a KE of 100 Joules, but if that energy is transfered over the course of one billionth of a second, you can have a huge input of energy. It's the same idea as a flat plate with an area of 1 meter squared, not sinking into the ground much at all when loaded with 100 N of force. But when that 100 N of force is loaded onto a needle... well that needle is going to sink pretty darn easily.

  11. Circular reasoning by D3 · · Score: 2, Interesting

    The fact that human error isn't compensated for is the true human error that needs compensation.
    I think I just sprained my brain thinking up that one.

    --
    Do really dense people warp space more than others?
  12. Human error is a factor in ALL work... by ites · · Score: 1

    All moderately-complex projects have to be built around:

    1. change
    2. error

    --
    Sig for sale or rent. One previous user. Inquire within.
  13. The problem with errors by RealityProphet · · Score: 3, Interesting

    The problem with errors is that detecting all errors all the time is absolutely impossible. Think back to your intro theory cs class and to Turing Recognizability. Think halting problem. Now, reduce the problem of finding all errors to the halting problem:

    if (my_design_contains_any_errors) while(1);
    else exit;

    Feed this into a program that halts on all input and see what happens. You can't, because we know it is impossible for it to always return an answer. QED: errors are unavoidable. No need to sniff derisively in the direction of NASA's "middle management". Let's see if YOU can do a better job!

    1. Re:The problem with errors by orac2 · · Score: 1

      No need to sniff derisively in the direction of NASA's "middle management".

      We're not talking about some unknown unknowns that crop up as the inevitable residue of your halting problem analogy.

      We're talking about a class of errors that have happened before. We already know about them, they've already been detected. And yet, because of management failure, they continue to persist. The Columbia Accident Board identified this as NASA's key problem, not the weakeness of Reinforced Carbon Carbon leading wing edges.

      Unknown errors are unavoidable, but ignoring solutions to known errors is unforgivable.

      --
      "Just once, I'd like to meet an alien menace that wasn't immune to bullets." -- The Brigadier, Dr. Who
    2. Re:The problem with errors by _Sprocket_ · · Score: 1


      The problem with errors is that detecting all errors all the time is absolutely impossible.


      This is the thinking that provided an opportunity for the Japanese to become economic giants.

      Granted - it didn't start that way. Early Japanese production methods were error-prone to say the least. Then an American statistician named Dr. W. Edwards Deming taught a post-war reconstruction Japan how to improve quality. Ironically, Deming developed these ideas to improve production of military products.

      Several decades later, US industries began catching on.
    3. Re:The problem with errors by tbjw · · Score: 1

      This isn't quite correct, since the halting problem applies only to a formal system. Life is not a formal system. Consider a real computer posed a real problem. Either the computer succeeds, or the computer falls apart. The problem will not run indefinitely.

      Anyway, that waffle aside, my point is that finding errors in engineering is not a formal-system implemented task, but a physical one, so that your argument generally fails. Not that the conclusion isn't true, or anything, but you haven't proved it, you've only given an analogy.

  14. No Excuse! by webgit · · Score: 1

    Since scientists have defined Murphy's Law, no-one should have any excuses for letting anything that can, go wrong!

    1. Re:No Excuse! by Anonymous Coward · · Score: 0

      The namesake of Murphy's Law, Edward A. Murphy, Jr. was awarded an Ig Nobel award in 2003, fittingly, after he passed away.

  15. where would we be without mistakes... by woodsrunner · · Score: 5, Insightful

    If you compare the advances to Science and Knowledge due to mistakes rather than deliberate acts, it might come out that everything is a mistake.

    Recently I took a class on AI (insemination, not intelligence) and apparently the two biggest breakthroughs by Dr. Polge, in preserving semen were due to mistakes. First, his lab mislabeled glycerol as fructose and they were able to find a good medium for suspension. Secondly, he blew off finishing freezing semen to go get a few pints and didn't make it back to the lab until the next day thus discovering that it was actually better to not freeze the stuff right away.

    Mistakes are some of the best parts of science and life in general. It's best to try to make more mistakes (i.e. take risks) than it is to try and always be right. (unless you are obsessive compulsive).

    1. Re:where would we be without mistakes... by jmmcd · · Score: 2, Funny
      "he blew off finishing freezing semen to go get a few pints"

      you can't, possibly, have written that with a straight face. can you?

    2. Re:where would we be without mistakes... by khallow · · Score: 1

      We're not advocating the elimination of all mistakes but merely the elimination of easily foreseen and prevented mistakes.

    3. Re:where would we be without mistakes... by Dhalka226 · · Score: 1

      I completely agree that a lot of good has come from making mistakes and finding something new out.

      Still, I'd prefer my multi-million dollar spacecraft be programmed with the right units and not crash headlong into a planet. I'm picky that way though.

    4. Re:where would we be without mistakes... by Preferred+Customer · · Score: 1

      Evolution would stall without misteakes, would it not?

    5. Re:where would we be without mistakes... by Anonymous Coward · · Score: 0

      There you go putting money first again. So you would rather save a few pieces of paper than possibly learn that there is a flaw in your design, or be forced to learn a new and possibly safer method of landing a craft? Hopefully you never get to a position where you are funding research.

  16. we're living in an impefect world by igzat · · Score: 0

    Nasa's current difficulties arise from scattered teams that all only check their parts rather than having fully qualified teams that go over the entire vehical. The fact that the whole thing is usually designed by committee and in several pieces then assembled at the last minute probally helps facilitate error. The Saturn V rockets and other technology we used to land on the moon had hte capability of being far less relyable than today's technology but we still managed to use them for years without error.

    1. Re:we're living in an impefect world by Halo- · · Score: 3, Insightful
      Nasa's current difficulties arise from scattered teams that all only check their parts rather than having fully qualified teams that go over the entire vehical.

      I'm not sure I buy that completely. While it certainly would help to have a single SME go over the entire vehicle, I doubt such a person could exist and complete the checks in a reasonable amount of time. The guy who checks the computer code is probably not going to be an expert in metal fatigue, nor electrical engineering. Even if you could find some sort of uber-genius who had expert knowledge of every system, he or she would have to work serially. If they started at component "1" of 654224166 and went down the line in order, the checks they started with would be out of date by the time they finished.

  17. I not so sure its just Murphys law by Timesprout · · Score: 1

    Its important not to loose sight of the harshness of the enviroment these systems are designed to operate in. As a simple example striking a match is an easy task to perform yet people trapped in cold remote areas have died because they were under too much stress to light a fire even though they had the tools. Its also question of consequences when something goes wrong, and space is not very forgiving.

    --
    Do not try to read the dupe, thats impossible. Instead, only try to realize the truth
    What truth?
    There is no dupe
    1. Re:I not so sure its just Murphys law by orac2 · · Score: 1

      Except the problems Oberg describes, including the proximate cause of the Genesis crash, have absolutely nothing to do with environmental conditions in space or anywhere else, but everything to do with mistakes made in nice clean rooms here on Earth, and magnified by poor management, again here on Earth.

      --
      "Just once, I'd like to meet an alien menace that wasn't immune to bullets." -- The Brigadier, Dr. Who
    2. Re:I not so sure its just Murphys law by shotfeel · · Score: 1

      magnified by poor management, again here on Earth.

      Are you seriously trying to tell me that "management" isn't from another planet?

  18. Human Factor by xnot · · Score: 4, Insightful

    I think the biggest difficulty surrounding large organizations is the lack of communication tools linking the right engineers together. It seems unfathomable that some of these mistakes were able to propegate throughout the entire engineering process and nobody caught them.

    Unless you consider the fact that often in large organizations, the left hand typically has no clue what the right hand is doing. I work at Lockheed Martin, and typically I'm involved in situations where one group makes an improvement that then none of the other groups know about, changes/decisions are poorly documented (if at all) so nobody knows where the process is going, people making poor decisions due to lack of proper procedures from management about what to do, teams not being co-located, poor information about which people have the necessary knowledge to solve a particular problem, or any number of things that confuses the engineering process, to the detriment of the product. Most of these situations are caused by a lack of communication throughout the organization as a whole.

    This is a serious problem, and it needs to be acknowledged by the people in a position to make a difference.

    1. Re:Human Factor by OwnedByTwoCats · · Score: 1

      The problem with every large organization is that building the communication networks linking the right people is hard to do. There is a tradeoff to make when deciding which and how many people information needs to propagate to. Too few, and the right people don't get the message. Too many, and the message gets lost in the noise.

  19. Nasty Remark by mathematician · · Score: 3, Insightful

    "Either you manage and design around it or fail. NASA management still often chooses the latter."

    I find this remark very unfair. It is a really nasty snide attitude to it, like "we are perfect - why can't you be."

    Come on guys, NASA is trying to do some really difficult and ground breaking stuff here. Cut them some slack.

    1. Re:Nasty Remark by GigsVT · · Score: 1

      Ground breaking stuff... like ...?

      Collecting dust?
      A pointless low gravity lab in space?
      25 year old shuttles that are more expensive than disposable craft?

      --
      I've had enough abrasive sigs. Kittens are cute and fuzzy.
    2. Re:Nasty Remark by orac2 · · Score: 1

      As I posted elsewhere, but I belive it bears repeating:

      We're talking about a class of errors that have happened before. We already know about them, they've already been detected. And yet, because of management failure, they continue to persist. The Columbia Accident Board identified this failure as NASA's key problem, not the weakeness of Reinforced Carbon Carbon leading wing edges.

      Unknown errors are unavoidable, but ignoring solutions to known errors is unforgivable.

      During his 22-years at Mission Control, Oberg's helped design real missions. And just because you think its "nagging" doesn't mean that anyone who gives a damn about the space program, or just good engineering, should ever let up for a second in their demands for engineering and managerial excellence.

      --
      "Just once, I'd like to meet an alien menace that wasn't immune to bullets." -- The Brigadier, Dr. Who
    3. Re:Nasty Remark by Anonymous Coward · · Score: 0

      "Come on guys, NASA is trying to do some really difficult and ground breaking stuff here."
      Difficult, yes. Groundbreaking? Come on. What technological innovation has come from their space exploration antics from the last thirty years? That's OUR money they're spending. Fuck NASA.

    4. Re:Nasty Remark by Anonymous Coward · · Score: 0

      No, MY money is going to NASA, YOUR money is going toward squash farming subsidies.

      Seriously, you jerk, you have to pay taxes before YOUR money is going toward any government programs...

    5. Re:Nasty Remark by sexylicious · · Score: 1

      Ground breaking stuff... like ...?

      How about:
      autopilots, mars missions, ground penetrating radar, TDRS ( a deep space communications network), X43 (the world's only vehicle capable of Mach 7, and possibly Mach 10), the worlds first computerized inertial guidance system (apollo), Kalman filters (used in a LOT of electronics), not to mention thousands of improvements and inventions related to aviation safety (gust alleviation systems, flutter analysis methods, investigation into forward swept wings, supercritical airfoils, exploration into quieting turboprops, turbojets, and turbofans). And that doesn't even touch on the hundreds of thousands of earth sciences experiments, astronomy/cosmology experiments, planetary sciences experiments, and a whole host of other experiments.

      Without NASA, we wouldn't have reliable weather forecasting (even three or five days is better than none at all), we wouldn't have had such a huge development of consumer electronics (which stemmed from both NASA and DoD interest in electronics), a plethora of medical knowledge that directly resulted from studying human reactions to low-gravity environments and animal and plant studies done to see what development of organisms is like with low gravity, and information about the earth's ozone layer and ionosphere (did you know that you could hear radio transmissions around the world because they bounce off the ionosphere, and that the stations you recieve depend on the time of day because the ionosphere expands when hit with daylight?). NASA also does quite a bit of work with NOAA for modeling and simulation of weather systems.

      There are plenty of ways that NASA's work has influenced all of us.

    6. Re:Nasty Remark by GigsVT · · Score: 1

      Yeah, but to quote the cliche:

      What have they done for me lately?

      Most of that innovation you name is old stuff, very old stuff. And as you said, a big chunk of that stuff was military driven.

      --
      I've had enough abrasive sigs. Kittens are cute and fuzzy.
  20. armchair rocket science by onion_breath · · Score: 5, Insightful

    I love how journalists and others like to sit back and criticize these engineers' efforts. They are human, and they will do stupid things. Having been trained as a mechanical engineer (although I mostly do software engineering now), I have some idea of how many calculations have to be made to design even one aspect of a project. I couldn't imagine the complexity of such a system, trying to account for every scenario, making sure agorithms and processes work as planned for ONE mission. No second chances. That we have individuals willing to dedicate the mental efforts to this cause at all is worthy of praise. These people have pride and passion in what they do, and I'm sure they will continue to do their best.

    For anyone wanting to yack about poor performance... put your money where your mouth is. I just get sick of all the constant nagging.

    --
    this is my sig, be amazed.
    1. Re:armchair rocket science by andrewbaldwin · · Score: 1

      Well said!!

      Oh how I wish I had mod points right now!

    2. Re:armchair rocket science by BenjyD · · Score: 1

      Exactly - people should listen to themselves sometimes.

      "Can't you even fling a 2 tonne piece of incredibly delicate scientific apparatus a billion miles across space without one thing going wrong? Call yourself a scientist?"

    3. Re:armchair rocket science by orac2 · · Score: 1

      Except that Oberg is a 22-year veteran of Mission Control.

      Except that everything he's saying here is an echo of what the Columbia Accident Investigation Board said about NASA's manned space program.

      Except that, if you'd bother to read the article, you'd see that the criticism is not of "engineer's efforts", but of management.

      Except that "are human, and they will do stupid things" is the whole point of Oberg's article, and he's talking about the failure of NASA to provide oversight to catch these inevitable screw-ups.

      For anyone wanting to yack about poor performance... put your money where your mouth is. I just get sick of all the constant nagging.

      Oberg's helped design real missions. And just because you think its "nagging" doesn't mean that anyone who gives a damn about the space program, or just good engineering, should ever let up for a second in their demands for engineering and managerial excellence.

      --
      "Just once, I'd like to meet an alien menace that wasn't immune to bullets." -- The Brigadier, Dr. Who
    4. Re:armchair rocket science by mks180 · · Score: 1

      It's more than just calculations and trying to account for every scenario. What most people don't realize who don't do R&D is that all designs are based on approximations and assumptions. You can never be completely certain of their consequences; that's why there are factors of safety. On top of that, you can never be certain of the exact material strength and composition. We don't understand all the non-linear effects that arise in even simple systems, let alone something extremely complicated as most aerospace systems.

      I agree, human error is always a major contributor to these disasters. Management short sightedness and narrow-mindedness as well as penny-pinching had a role in all of them too. But at the same time you have to consider the logistics of these operations and and making do with often less than adequate resources. Try coordinating hundreds, if not thousands, of people across the country, probably the world, in the design and manufacture of a device that has thousands of parts that all have to work together. It's not as simple as the reporters would like everyone to believe. They like to sensationalize things, since that makes people read or listen, and gets them paid. I don't see that changing anytime soon

    5. Re:armchair rocket science by klipsch_gmx · · Score: 1
      Having been trained as a mechanical engineer (although I mostly do software engineering now),

      <runs for life>
    6. Re:armchair rocket science by Anonymous Coward · · Score: 0

      > For anyone wanting to yack about poor performance... put your money where your mouth
      > is. I just get sick of all the constant nagging.

      I think it's mostly taxpayer's money that pays for NASA's mistakes...

    7. Re:armchair rocket science by Anonymous Coward · · Score: 0

      And WHEN HE WAS there, they didn't make any of these mistakes, no siree...

      In my day sonny, etc., etc.

      I'm sure he's a smart guy, but just because he worked in mission control doesn't make him an expert in every aspect of engineering or spacecraft design (or the oversight process therein).

      How many decisions are made in the design and implementation of a major space mission? Not saying there isn't room for improvement, but an oversight process that will guarentee not one failure in any of these decisions seems a difficult problem to solve.

    8. Re:armchair rocket science by orac2 · · Score: 1

      just because he worked in mission control doesn't make him an expert in every aspect of engineering or spacecraft design

      Nobody is an expert on "every aspect of engineering or spacecraft design." If we took your position to its logical conclusion there could be no oversight or criticism of any problem more complex than what a single engineer could reasonably tackle.

      To say good oversight is hard to do is true, but it doesn't invalidate someone reporting demonstrably bad oversight.

      --
      "Just once, I'd like to meet an alien menace that wasn't immune to bullets." -- The Brigadier, Dr. Who
    9. Re:armchair rocket science by rbmyers · · Score: 1

      Finger-pointing by bystanders is not only unattractive, it's also useless. Once we get past the finger-pointing, is there anything to be done other than handwringing? The basic subject here is decision-making in the face of uncertainty, and it's something that we, as human beings, know a great deal about. You could spend the entire GDP and _still_ miss something that seems obvious in retrospect, or you could scrape everything to the bone, get lucky, and brag about your management technique. In place of the ad hoc opinionating and after the fact handwringing, there has to be a systematic way of managing risk in situations where there is almost no experience. NASA thought it had such a systematic approach with fault-tree analysis, which turned out to have fundamental logical flaws. NASA is still crashing and burning--probably no way around that--but its explanations suggest the absence even of movement toward a better systematic approach.

  21. John Galls Systemantics by Anonymous Coward · · Score: 4, Interesting
    Systems display antics. John Gall has written a great book which vastly expands on Murphys law which is called Systemantics - The Underground Text of Systems Lore. I cannot recommend this book enough. It contains some truths about the world around us that's blindingly obvious once you see it, but until then you're part of the problem. Systemantics applied to political systems is very enlightening. Too bad that the only people who think like this in politics are the selfish and egomanical Libertarians (yeah, yeah.. I know. Libertarianism is the new cool for the self styled nerd political wannabe).

    Here are some of the highlights:
    • 1. If anything can go wrong, it will. (see Murphy's law)
    • 2. Systems in general work poorly or not at all.
    • 3. Complicated systems seldom exceed five percent efficiency.
    • 4. In complex systems, malfunction and even total non-function may not be detectable for long periods (if ever).
    • 5. A system can fail in an infinite number of ways.
    • 6. Systems tend to grow, and as they grow, they encroach.
    • 7. As systems grow in complexity, they tend to oppose their stated function.
    • 8. As systems grow in size, they tend to lose basic functions.
    • 9. The larger the system, the less the variety in the product.
    • 10. The larger the system, the narrower and more specialized the interfaces between individual elements.
    • 11. Control of a system is exercised by the element with the greatest variety of behavioral responses.
    • 12. Loose systems last longer and work better.
    • 13. Complex systems exhibit complex and unexpected behaviors.
    • 14. Colossal systems foster colossal errors.
    1. Re:John Galls Systemantics by fontkick · · Score: 1

      Also known as "Taxpayers Guide to the U.S. Government"

    2. Re:John Galls Systemantics by Anonymous Coward · · Score: 0

      Wow, you can copy and paste. Now to chapter 2 of "AOL for dummies".

    3. Re:John Galls Systemantics by Anonymous Coward · · Score: 0

      Your point being?

    4. Re:John Galls Systemantics by Anonymous Coward · · Score: 0
      5. A system can fail in an infinite number of ways.

      Unless it's Windows, in which case it only fails in one way.

    5. Re:John Galls Systemantics by Anonymous Coward · · Score: 0

      Hmm... sounds just like Windows to me.

  22. In Science it is $Serendipity ... by foobsr · · Score: 1

    ... not mistakes. Have a look

    CC.

    --
    TaijiQuan (Huang, 5 loosenings)
  23. Comforting quote from the article. by wiredog · · Score: 3, Informative
    "these switches were reportedly developed as a nuclear warhead safety device"

    Very comforting to know how easy it is to wire the safeties on nuclear weapons up backwards.

    1. Re:Comforting quote from the article. by Anonymous Coward · · Score: 0

      don't worry, it would be hopefully discovered after the first explosion.

    2. Re:Comforting quote from the article. by advocate_one · · Score: 1

      yeah, but they were meant to work the other way round in that instance... you had to experience several seconds of hi-G during launch followed by no G before the devices were supposed to arm. Wonder if the original devices had been Murphy'd to be installed only the right way round and their subsequent use in the opposite sense wasn't intended...

      --
      Donald 'Duck' Dunn: We had a band powerful enough to turn goat piss into gasoline.
  24. kneel before the overlords by Anonymous Coward · · Score: 0

    ...I for one, welcome the computer overlords...

  25. If Murphy knew what was going on today by Prince+Vegeta+SSJ4 · · Score: 1
    he would roll over in his grave and say:

    After all this post has GENESIS and outer space in it.

  26. Wisdom by Anonymous Coward · · Score: 0

    What humans do well is use wisdom. It is the process of making decisions when you don't have enough information. Then you use judgement based on experience and training.

    The idea that, if we give a technician a detailed set of instructions, we will get the right result is the opposite of wisdom. It is bureaucracy. It will always produce errors in unfamiliar situations.

    The management system that I like best is that practiced by Admiral Nelson. He spent much time with his officers discussing strategy and tactics. When the time for battle came, it didn't matter what happened to the command structure, somebody would be able to take the right decision. That's why the British navy was so good at that time.

    The opposite was when the British formed a square to fight the Zulus. The Zulu general decided to attack one side of the square only. That side ran out of bullets. They sent a runner for more ammunition and the quartermaster said something like: "Sorry, you've used your allocation. No bullets for you." The Zulus won of course.

    The bottom line is that if you want someone to do something for you, make sure they UNDERSTAND what they are doing and why. If someone installs a fitting backwards for you, it's your fault because the installer clearly didn't understand what was required.

    1. Re:Wisdom by Ignignot · · Score: 1

      The bottom line is that if you want someone to do something for you, make sure they UNDERSTAND what they are doing and why. If someone installs a fitting backwards for you, it's your fault because the installer clearly didn't understand what was required.

      This is bullshit. Sometimes people can't be made to understand something, and they are in a position above you so you can't do anything about it. For example, I once found a big problem with some inhouse software I was helping develop. I couldn't fix that part because the code wasn't even accessable to me, and I didn't know how to program in that language. So I told the manager - "hey, this is a big thing and when X happens, we're gunna get screwed". But he ignored me. So I sent out a memo detailing the problems. I got ignored. I told the guy who wrote the thing to fix it, got ignored. Finally they told me to shut up or be fired. After awhile, everything got fucked up because of the problem. Who gets the blame? Me - and they said it was because I didn't make them understand. There is no cure for willful ignorance.

      --
      I submitted this story last night, and it didn't get posted.
    2. Re:Wisdom by Anonymous Coward · · Score: 0

      You sound a bit like you have a chip on your shoulder. Applying such a general rule to a subject which has so many dynamic aspects is futile.

      This discussion is proof positive that almost any discussion can be dragged into a pissing contest of pride, & wit........

  27. Anyone else find this sobering quote? by argStyopa · · Score: 1

    from the article:
    "After all, these switches were reportedly developed as a nuclear warhead safety device, so one could just assume that they were properly wired."

    Nice to know those safety devices are foolproof.

    --
    -Styopa
  28. Time for grass-roots action? by mwood · · Score: 1

    Maybe we should pass the hat and send every NASA manager a copy of _Systemantics_, for their enlightenment. (Likely the scientists and engineers already have their own copies.)

  29. Re: Straight face by woodsrunner · · Score: 1

    I guess I have passed the threshold where I even think of these jokes. That actually was funny and I didn't even realize it. er, my mistake...

    It's not too difficult to become immune to it talking about semen all day. We even have post it notes and pens with pictures of semen on them.

    I guess that is the fate of working in AI. On the upside, it must have an effect on overall fertility. Most of the people in this company seem to have 3 kids minimum!

  30. Self-referential Murphy's Law by quoob · · Score: 1

    As the article quotes Murphy's Law: "Every component than can be installed backward, eventually will be."

  31. KISS... by Kong99 · · Score: 3, Interesting
    A great engineer demonstrates his/her skill not by designing something terribly complex, but by designing the object that meets required specifications as SIMPLY as possible with as FEW unique parts as possible. That is GREAT engineering.

    However, with that being said I really do not believe Engineers are the problem at NASA. Bureaucracy is the enemy at NASA. NASA needs a complete top to bottom overhaul.

    1. Re:KISS... by Detritus · · Score: 1

      The floggings will continue until morale improves.

      --
      Mea navis aericumbens anguillis abundat
  32. Author of TFA... by Anonymous Coward · · Score: 0

    Looking at the author's webpage about himself, one sees that he's has a long history with NASA, stretching back to the 70's.

    Then I noted how he represents his name. WTF is with the "O"?? Suddenly, I'm reminded of the Demotivational poster for Consulting...

    TFOAE

  33. We need Errors! Right? by mu_shadow · · Score: 0

    When you make an error you eliminate a possibility out of the equation thus bringing you closer to a solution. You can't learn without making mistakes. Lots of AI is based on computers making mistakes.

    http://www.darwinmag.com/learn/curve/column.html?A rticleID=44 Explains: "A computer using AI to play chess, for example, has the ability to "learn" millions of possible moves, process them and make a choice. Witness Gary Kasparov's stunning defeat by IBM Corp.'s Deep Blue. The computer did not just mimic a memorized set of moves; it altered its strategy based on the results of the mistakes it made in previous games."

    Of course you could argue that humans wrote the code, that caused the computer to intentionally make a mistake therefore it is the human's fault, Wait? Do you call an intestinal mistake an error? Now I'm confused!

    Wait? Errors can be used to make a funny too! Did I make an error typing above or was it intentialy funny? Because farts are funny!

    --
    Thanks, because I don't know what I'm talking about and never claimed I did...
  34. You'd think so. by Bill,+Shooter+of+Bul · · Score: 2, Insightful

    Thats a very popular cliche. The fact is with NASA's shrinking budgets, they don't have the resources to design around potential failures. There's old school NASA that desinged the Cassini probe that has redundant systems and is properly designed and tested, and there's new school NASA that makes the cheap Mars probes. Just looking at the Mars probes you'll see why they have moved to this method. If you can make five fault intolerant probes for the same cost as one fault tolerant probe, and odd are that only two of the five work, then its a better idea to build the five crappy probes as you'll probely get twice the science benifit. The problem comes once you start throwing in human lives. Its okay if a manless probe crashes, its only things, but if you apply the same logic to manned missions.. you get the Columbia accident. Its not that NASA intentionally overlooked the problems because they expect people to die, its just that the methodology from the non manned flights have crept into their minds. At least thats my non-expert opinion.

    But I guess it sort of applies to your software analogy as well. There have been a few companies who have discovered that its cheaper to have paying customers find the flaws in their software, rather than do any kind of formalized testing before release.

    --
    Well.. maybe. Or Maybe not. But Definitely not sort of.
    1. Re:You'd think so. by RAMMS+EIN · · Score: 2, Insightful

      ``There have been a few companies who have discovered that its cheaper to have paying customers find the flaws in their software, rather than do any kind of formalized testing before release.''

      Not only that, but it's actually beneficial to produce and ship buggy software. Bugs have to be fixed, and who can fix them better than the people who wrote the code? So, it makes sense for programmers to leave flaws in their programs. Companies that ship flawed products can make customers pay for upgrades that also fix bugs, or get good karma by providing bugfixes for free. In the process they get publicity, and the world can see they're not sitting still and their products have not been abandoned.

      --
      Please correct me if I got my facts wrong.
  35. The REAL REAL Reason for Errors! by Ced_Ex · · Score: 5, Funny

    Here's the real reason for NASA and their errors, as quoted by Gordon Cooper a former astronaut.

    "Well, you're sitting on top of this rocket, about to be flung into the most hostile environement know to man, and you keep thinking, 'Everything here was supplied by the lowest bidder.'"

    --
    Live forever, or die trying.
    1. Re:The REAL REAL Reason for Errors! by Daagar · · Score: 1

      Liar! That's a mangled quote from the movie Armageddon! Everyone know it was Bruce Willis that said it. Geez.

  36. Plan for Software Project Failure by StCredZero · · Score: 2, Interesting
    I'm just waiting for management to get a clue about this. Most software projects fail on some level. Most software in big corporations sucks on some level, in at least one important component. Management should get a clue and PLAN for this. They should:
    • Have redundant competing projects
    • Have standards that mandate how components/systems fit together
    • NOT mandate that thou shalt use software X all across the enterprise

    What large corporations have been doing is Soviet style central planning. What happens is that they get stuck with mediocre or sucky software that they cannot replace. Eventually, a few smaller companies start up that manage to have good software (out of many that fail in part because of sucky software) which gives them a competitive advantage. These either get bought up by or grow into ossified bureaucratic behemoths with no internal competition.

    Sometime a corporation is going to become the Bazaar within, instead of the Cathedral (Cathedral & the Bazaar) and they'll maintain a long term competitive advantage by having internal competition.

    I'm not holding my breath, however.
  37. In testing, you WANT it to fail! by ishmalius · · Score: 2, Insightful
    During design and testing, Murphy is your best friend. Before the baby chick leaves the nest, you want everything that can possibly go wrong, to do so. You can address each of the failures encountered, and then move on to new opportunities for error. This is a mysterious process called "learning," which definitely has its good points.

    NASA does test everything. He didn't mention in the article, but I would be almost certain that the accelerometers were tested, and passed the tests; but that the tests themselves were improper.

    1. Re:In testing, you WANT it to fail! by orac2 · · Score: 1

      What Oberg is objecting to (and its one of the the things the Columbia Accident investigation board objected too as well) is that NASA has a long history of forgetting lessons learned.

      That is why he is holding managements feet to the fire here, as did the CAIB.

      And NASA doesn't test everything. In fact, NASA's relience on simulation and extrapolation and just plain guess work was harshly criticized in CAIB report, which is free for anyone to read.

      --
      "Just once, I'd like to meet an alien menace that wasn't immune to bullets." -- The Brigadier, Dr. Who
    2. Re:In testing, you WANT it to fail! by Anonymous Coward · · Score: 0

      Correct. This system was tested and it passed. The test was inadequate to detect that the G-switches were installed backwards.
      Everything that can possible be tested, is tested, and in a "test like you fly" environment to the greatest extent possible. I've worked on Genesis since nearly day one and still work on the program (we have a s/c bus still flying around out there that we need to do something with - anyone remember that?) and believe me, every program here pays very close attention to the TLYF motto.
      The problems come when the limited funding allocated to Discovery-class missions means that you can't build the perfect test environment. With budget and schedule constraints there is only so much you can do.
      The problem here is that NASA never fully communicated to the public that faster, better, cheaper means there will be more failures. It turns out that NASA has built up a "Failure Is Not An Option" expectation with the US public and has did nothing to change that as they moved away from once-a-decade, multi-billion dollar missions to less-expensive, science-oriented missions that could be thrown up every few years and where the cost of failure or mission degradation isn't so catastrophic.

    3. Re:In testing, you WANT it to fail! by Feanturi · · Score: 1

      He didn't mention in the article, but I would be almost certain that the accelerometers were tested, and passed the tests; but that the tests themselves were improper.

      The article did mention the accelerometers appearing to work properly in testing, but as it turned out, the testing equipment was also wired backwards. How's that for bad luck?

  38. It's not number of errors caught but importance by tentimestwenty · · Score: 3, Insightful

    There might always be errors which you can reduce with many checks. The key is to have the checks done by someone who has an eye for potential problems. There is a particular skill set/personality that can forsee unknown problems better than say an engineer who is single minded and focussed. You can get a hundred experts to check the same work but often it's the one guy who says "why is that wheel upsidedown" that reveals a completely unanticipated problem.

    1. Re:It's not number of errors caught but importance by feargal · · Score: 1

      ...often it's the one guy who says "why is that wheel upsidedown" that reveals a completely unanticipated problem.

      Indeed.

      --
      "A goldfish was his muse, eternally amused"
  39. Most people choose the latter. by Anonymous Coward · · Score: 2, Insightful

    It is a difficult thing to design something to face failures. It requires a mind set vastly different than that of most "builders of things." Those folks tend to think in the positive: my creation does this, and this, and this, and ... This is true whether the thing being built is a program, a car, or a team of people.

    If you want to see this in action, find your favorite developer and ask the following: "What does your program do, and how doe it do that?" Prepare for a long response :-)

    Then ask: "How does it break?" You will most likely get a blank look. You may get a list of things the program doesn't do (missing or removed features), or possibly a list of known bugs, but you will almost never get an answer detailing the failure modes of the program itself. That is, they will not be able to tell you what happens when various assumptions are wildly wrong.

    Answering those sorts of questions requires thinking in the negative (not necessarily negative thinking), which is an entirely different mode of thought. It's also much less pleasant. After all, considering the destruction of the beautiful thing you've built is not a psychologically easy task.

  40. no news by mec_cool · · Score: 0

    errare humanum est.

    It's two thousands years old, some latin guy said that;

  41. A History of Murphy's Law... by VValdo · · Score: 1

    And the subsequent controversies...

    is here.

    (This paper won a prestigious 2003 Ig Nobel award for engineering.

    W

    --
    -------------------
    This is my SIG. There are many like it, but this one is mine.
  42. The Murphy myth: what really happened? by argent · · Score: 2, Funny

    Summary of The fastest man on Earth:

    George Nichols: "The Law's namesake, was Capt. Ed Murphy Jr., a development engineer... Frustrated with a strap transducer which was malfunctioning due to an error in wiring the strain gauge bridges caused him to remark-- 'if there is any way to do it wrong, he will'-- referring to the technician who had wired the bridges. I assigned Murphy's Law to the statement and the associated variations..."

    David Hill: "Murphy was kind of miffed off. And that gave rise to his observation: 'If there's any way they can do it wrong, they will.' I kind of chuckled and said, that's the way it goes. Nothing more could be done really."

    John Paul Stapp: "we do all of our work in consideration of Murphy's Law. [defined as] the idea that you had to think through all possibilities before doing a test."

    Dr. Dana Kilanowski: "at the time I believe Stapp said something like, 'If anything can go wrong he'll do it.' A couple days later there was a press conference in Los Angeles and Stapp said something like, 'it was Murphy's Law -- if anything can go wrong, it will go wrong.' [...] I have heard that Murphy claimed he invented Murphy's Law, but Stapp is the one noted for his witticisms, his haikus, and his plays on words."

    Ed Murphy: "I didn't tell them that they had positively to orient them in only one direction. So I guess about that time I said, 'Well, I really have made a terrible mistake here, I didn't cover every possibility.' And about that time, Major Stapp says, 'Well, that's a good candidate for Murphy's Law'. I thought he was going to court martial me, but that's all he said. [Stapp reeled off a host of other Laws, and said] 'from now on we're going to have things done according to Murphy's Law'."

    Chuck Yaeger: "Look, what you're getting into here is like a Pandora's Box. Goddamn it, that's the same kind of crap...you get out of guys who were not involved and came in many years after."

    And in the end it wasn't as extreme a failure as Genesis:

    According to Nichols the failure was only a momentary setback --"the strap information wasn't that important anyway," he says -- and regardless good data had been collected from other instruments. The Northrop team rewired the gauges, calibrated them, and did another test. This time Murphy's transducers worked perfectly, producing useable data. And from that point forward, Nichols notes, "we used them straight on" because they were a good addition to the telemetry package. But Murphy wasn't around to witness his devices' success. He'd returned to Wright Field and never visited the Gee Whiz track ever again.

  43. NASA chooses the later!?! by Anonymous Coward · · Score: 0

    As a NASA contractor that works works on the Space Shuttle, I can assure you that NASA has a very robust system for dealing with Murphy's law.

    When I first started working here, I was taken aback by the extreme amount of peer review required to accomplish something as simple as a screw replacement on the Space Shuttle.

    But having put a lot of thought into it, I now appreciate all this extra "bureacracy". Some very smart people have put a lot of time into circumventing Murphy's Law in this space program.

    Still not convinced?

    "Go Linux!"
    "Microsoft suxx0r!"
    "George Bush is stupid!"
    "(insert another instant /. credibility establisher here)"

    P.S. Did you ever see that episode of South Park where Stan becomes a Goth?

  44. Easy to answer... by zogger · · Score: 2, Insightful

    ...it's because humans have this weird deal with society. We wind up with the greediest, lamest most megalomaniacal people for governmental/corporate "leaders". 999 out of a thousand are this way, it just happens, we notice it.

    These people are quite *insane*. they may be brilliant, but still bonkers. They have the most power and money of everyone on the planet. They hire the smartest people they can find, and reseaqrch advanced weaponry. All governments spend a huge amount of time and money and resopurces on this. they hire the smartest scientists and engineers they can find for this task. then the hire the people who psychologically and intellectually are the most prone to use these devices that the scientists and engineewrs create. these people are given more power than "ordinary" citizens, they are tasked with killing people and breaking peoples things, using these advanced machines. This weaponry, consisting of mechanical machines augmented with electrical and chemical advances, are *exactly* designed to "harm humans" and they DO harm humans with this machinery. Happens every day around the globe, by the thousands. Literally thousands of humans a day are killed, and many more horribly mutilated and injured. And the way the system has evolved, it is rigged to always have the megalomaniacs wind up "in charge" and all populations have a certain percentageof "ask no questions" order followers.

    So, stuff happens,evil wicked nasty horrible screaming stuff. This leads to this "fear" which isn't in the least bit an irrational fear for anyone sane to have. It's because it's reality.

    Lately, we can read that they want to automate and robotosize this even further, and to take these machines as far as they can push it with near unlimited budgets and millions of man hours of advanced research. It is not a "tin foil hat" phenomenon for folks to notice that. We also have a veifiable past track record to show that yes indeed, these megalomaniacs tame scientists and engineers and order followers screw up, we get what is called "unintended consequences" and "collateral damage", as if the intended consequences and planned-for damages aren't bad enough. So we as ordinary humans all around the globe who really do not have a beef with joe over there all get to have these "benefits", and we notice that we don't want those sorts of benefits, but there's not a thing we can do about it, because this advancing technology system is rigged in favor of those who like and enjoy and profit-from doing harm.

    You see, we DO have a lot of at least technically "competent people programming the machines", the problem is, they ARE designed to harm. And it's set up to be self perpetuating/advancing and is based from the git-go on forced wealth transference, ie, "theft" and it goes down hill from there into every worse things..

  45. Human Nature by z3r0w8 · · Score: 1

    Couldn't Murph just be an extrapolation of the fact human's tend to remember the bad, especially the really bad, over success?

    --
    -----
  46. Re:The Gimli Glider by dtmos · · Score: 3, Interesting

    The jet liner to which you refer, I think, is the Gimli glider which, through a forehead-slapping number of independent goofs, ambiguities, and misunderstandings made by a frighteningly large number of people, ran out of fuel over Cananda in 1983.

  47. I Vote Moore's Law by phobos13013 · · Score: 1

    Forget Murphy's Law, why cant Moore's Law win out at NASA?! I want to see the time it takes to get to Mars HALVE every ten years... Hmmm, i guess then Murphy's Law would be second in this case tho so it might be extremely dangerous for the crew.

    --
    ...and it should be known by now
  48. Mark my words by nounderscores · · Score: 1

    genetic algorithms

  49. Shouldn't the title be "Nasa 0wn3d by Murphy?" by waferhead · · Score: 2, Funny

    `nuff said...

    Note the "law" doesn't just torture NASA exclusively, it just rears its head very visibly in their case.

  50. Expect Failure by Anonymous Coward · · Score: 1, Interesting
    Human error is an inevitable input to any complex endeavor. Either you manage and design around it or fail.

    Strike the word complex from the above quotation.

    As a software developer that's responsible for developing protocols for various tasks, I've learned that any system needs to be robust against failure and should also fail safe. All too many times I've seen people come up with systems that function well when every part works exactly as it should, but blow up in terrible ways when a single mistake is made. For example, consider using the bronze-gold-silver way of doing revision control versus a real revision control system like CVS et al. The former system works only so long as people copy the proper file from one area to another every time, for as long as the system's in use. I've witnessed developers completely trash a production environment by accidentally copying old files into the gold area.

    Mistakes are going to happen and processes won't be followed 100% all of the time. The key is to design systems that expect this to occur and provide ways of dealing with the failure.

  51. Quick bit on murpheys law.. by anethema · · Score: 1

    Murpheys law isnt actually "If anything can go wrong, it will go wrong", This is Finagle's Law of Dynamic Negatives.

    Murphey orginally said "If there's more than one way to do a job, and one of those ways will result in disaster, then somebody will do it that way."

    --


    It's easier to fight for one's principles than to live up to them.
  52. That's a good argument for avoiding C/C++ by GCP · · Score: 1

    Human error is an inevitable input to any complex endeavor. Either you manage and design around it or fail.

    There are times when the only choice is to use C because of certain very tight constraints, but C and C++ are Murphy magnets. You know there's something wrong when you have not just an entire book but whole SERIES of books devoted to hidden traps that are entirely due to the design of the language itself.

    Using them when you don't have to on the grounds that you're so smart that you never fall into those traps is the sort of fundamental foolishness that makes the Murphy gods chuckle with glee.

    --
    "Those who have never entered upon scientific pursuits know not a tithe of the poetry by which they are surrounded."
    1. Re:That's a good argument for avoiding C/C++ by faragon · · Score: 1

      There are tons of C code on NASA apparatus, usually to critical ones -say ships, aircrafts or satellits-. C grammar is quite simple, C code can be extremely readable, and often, relliable, and most times 100% validable when not using dynamic memory. The C++ approach is deprecapted too often due to that C++ code is/was harder to validate because involves two entropy fonts: 1) application inherent critical bug ocurrence; 2) compiler hidden critical bug.

      In the offline side, C++, Fortran, Ada and Smalltalk are used despite C, because of flexibility and speedy design, due to miscelaneous causes: i) not everybody is C++ expert, ii) mathematicians were heavily trained on Fortran, iii) other surrealistic reasons

      For embedded systems, C, and from around 5 to 10 years ago, C++, with custom dynamic memory management to avoid memory fragmentation and deterministic resource allocation, there are no other common sense alternatives but assembly.

      Regard, have a good flight.

  53. System Level Issues for Engineering by danila · · Score: 1

    Check out The Columbia Tragedy: System Level Issues for Engineering, a free video from MIT World. In this video Sheila Widnall, MIT Institute Professor (Engineering Systems Division, Aeronautics) talks about her work on the investigation board. She talks about NASAs "culture of invincibility" where well-intentioned people became "desensitized to deviations from the norm". That is not only something often goes wrong as the Murphy's Law says, but people learn to ignore this until the disaster strikes.

    As Sheila noted, if an engine fell off from the shuttle, people tended to notice, act and do something about it, but when it was something small, like foam, they ignored the issue, even though it was clear that this anomaly warranted investgation, testing, etc.

    Interesting presentation that shows not only NASA sometimes ignores the potential for human error, but ignores the actual errors when they happen.

    --
    Future Wiki -- If you don't think about the future, you cannot have one.
  54. HOW IT HAPPENS by LaCosaNostradamus · · Score: 3, Insightful
    1. Manager issues a stupid order.
    2. Subordinates obey order out of fear.
    3. Manager gains confidence that stupidity is a valid method.
    4. Stupidity gains an increasing foothold until a catastrophe occurs.
    OR
    1. Manager cuts another corner or cost.
    2. Nothing immediately bad happens as a result.
    3. Manager gains confidence that cutting is a valid method.
    4. The cuttings increase until a catastrophe occurs.
    Managers are among the most moronic of the "educated" Western class ... because, after all, they don't understand the trends I outlined above.
    --
    [You have a stable society when some nut guns down a schoolyard and the law doesn't change.]
    1. Re:HOW IT HAPPENS by dvdeug · · Score: 1

      And engineers have no concept of how much it costs to do all the tests, and that sometimes you have to cut corners to have a job. Engineers and managers are looking at different parts of the big picture, and managers probably have the same attitude about engineers, because engineers worry about different things.

      Sometimes, if the choices is between doing it right and doing it cheap, doing it right means it's never going to get done. But engineers often don't see the bottome line.

    2. Re:HOW IT HAPPENS by Brew+Bird · · Score: 1

      Heh, You have a stagnent society when some nut guns down a school yard, and instead of punishing someone with existing law, the populous ignores the law, and clamers for a 'new' law so they can feel they have 'done' something about it..

    3. Re:HOW IT HAPPENS by LaCosaNostradamus · · Score: 1

      What's that saying?:

      "GOOD, QUICK, CHEAP ... choose any two."

      ... and it's TRUE.

      If managers would just admit things like "we're destroying the company with fatal levels of cost cutting", then I'd have nothing to criticize. But their dishonesty or ignorance are the real problems here.

      Furthermore, managers are paid to manage. Hence, I can only hold them responsible for the consequences of (1) cutting too many corners and (2) issuing stupid policies. They should have known. After all, don't they want to be considered to be the "management class" of Humanity? If so, they'll have to earn it.

      --
      [You have a stable society when some nut guns down a schoolyard and the law doesn't change.]
    4. Re:HOW IT HAPPENS by LaCosaNostradamus · · Score: 1

      A cogent observation. I feel strongly that we have more than enough law to apply to situations where people hurt other people. But we get failures of cultural will, hence leading to stuff like you pointed out. We also get other things like the concept of "hate crime" (since for some reason, the laws against murder, assault, stalking and vandalism weren't enough). Of course, in short, we're doomed.

      --
      [You have a stable society when some nut guns down a schoolyard and the law doesn't change.]
  55. This was not a troll by Anonymous Coward · · Score: 0

    I'm serious about this. Try getting a state job and see who works there. They'll look the part, but they can't perform the part. The hiring practices are no longer elitist.

  56. Just Blame GOD by lashi · · Score: 1
    >When a computer program crashes it's usually down to the human(s) who programmed it, and in the rare occasions it's a hardware glitch and it was humans who designed the hardware, so we're still to blame either directly or indirectly.

    And supposedly God created humans so we should all blame him. Your windows xp crashed? Act of God.

    Actually that's why I don't believe in the Christian God. God is infalleble but God created people. hmm...

  57. The real Bob Abooey by Anonymous Coward · · Score: 0
    Been there, done that!

    Back in 1963, our team developed the mathematical model. We then programmed it on the (then) state of the art GE 165 digital processing computer. Then we ran the model against then current U.S. census data. It took us 196 hours just to read it all in on paper tape. Then, in June 1964, the batch job completed. The result set was printed out on the teleprinter:

  58. Not specifically related, but you tend to see the same problems in medicine.

    You have several specialized departments working with each other, each with distinct and massive protocols that don't mesh well.

    Within certain regards, it almost seems the system is designed to fail by not designing for failure. Each department wants to give the appearance of toeing the company line even as internal systems fail. This reduces error reporting and functional problem solving because "it isn't SOP". Everyone is more concerned with placing blame than solving the problem.

    And when the problems become so massive you can't ignore them, a new policy is issued, adding to the multi-volume sets of policies; which are internally conflicting, and effectively enforces mediocrity from all departments.

    Add to this a shortage of people, people working under stressed conditions when lives are at stake, and you understand how a multitude of problems go undiagnosed simply because they are unknown. How do you know you are spreading nosicomial infections unless you check for them? And from what department are you going to pull the people to check, and how can you isolate the problem when each department is invested in covering their ass?

    I haven't read "The Underground Text of Systems Lore", but I imagine a lot of the same observations would apply (I actually looked at it as applying chaos theory to system management in the ER, but the same ideas are expressed in "The Cluetrain Manifesto", and other publications).

    Highly organized systems ultimately catabolism themselves unless they can restructure easily and effectively. NASA has not, medicine has not, and most political systems have not.

    Something to think about when you in the OR at 2AM.

  59. Re:interesting AND really true ... by quarkscat · · Score: 1

    One of the biggest problems that NASA faces is its control by politically appointed bean counters, instead of real engineers. While I was subcontracting to NASA (in another lifetime), I saw tens of millions of dollars spent upon teleconferencing equipment, while engineering emails that raised questions (prior to launch) about insulating tiles on the SST went unanswered. The rest is, unfortunately, history.

  60. Re:The Gimli Glider by Anonymous Coward · · Score: 0

    Thanks for the wonderful link. Don't ask what phrases got emitted verbally (loudly) (it's c.a.
    2.a.m. on saturday here in Athens GR as I write this). (2000-2500 fpm descent rate : Ouch. and
    I thought that sweating it out in MS flight sim
    with the engines turned off in the Lear was hard)

    I wish NASA had *read* this article. They wouldn't have made the same stupid mistake of reading metric and "thinking" imperial (ok, I Brit (grin) but YKWIM). Forgotten which Mars probe got dusted,
    but who cares - they'll do it again even if they
    promise not to. Perhaps the best argument for a
    manned approach is that they *can't* get away with
    screwing up so badly....

    Anyone interested in the "true" story of Murphy's law (spoiler: is there one?) should check

    http://www.improbable.com/airchives/paperair/vol um e9/v9i5/murphy/murphy0.html

    Hint: You'll need a good hour or more to read this carefully, so plan ahead (otherwise you know who will strike...). Or do you?

    (Sorry, but in keeping with the spirit, this *is*
    sort of a re-posted URL).

    I think the moderators didn't mark this highly enough - it ought to get a 5... (hint to CowboyNeal , I mean the *parent* to *this* reply).

    Anyone who wants to see the potential for the eponymous "Captain" to strike should consider
    checking out the (approx) weekly USENET "comp.risks" posting (almost zero noise, great laugh). My favourite from a few years back involved a spider (but not for reasons which are
    at all obvious).

    Finally, a plea to Peter Neumann if he lurks here - there are enough of these stories to make a web
    site a la Rob Carroll's skeptics dictionary or
    MadSci... Any takers? Plenty of slashdotters and
    assorted brethren would perhaps contribute..

  61. I thought they were just clumsy fuckers... by mat+catastrophe · · Score: 1

    After seeing this, wouldn't you say the same?

    --
    sig not found