Slashdot Mirror


NASA Finds Critical Assembly Fault in Shuttle

mzs writes "During corrosion inspection on Discovery, technicians noticed that one of the gears in a rudder actuator had been installed backwards. This particular actuator was the top-most of four that control the air brakes on the tail. As luck turns out, if it had been the bottom-most actuator, loss of the shuttle and crew would have been nearly inevitable. Plans are in place to have four spares by the time Shuttle missions resume next year."

33 of 383 comments (clear)

  1. Simple solution, really. by grub · · Score: 5, Funny


    Could they not stamp "THIS SIDE UP" or whatever on the components?

    --
    Trolling is a art,
    1. Re:Simple solution, really. by Lattitude · · Score: 5, Insightful

      Even better, they are going to machine them so they mechanically impossible to put in backwards.

    2. Re:Simple solution, really. by fbform · · Score: 5, Funny


      Could they not stamp "THIS SIDE UP" or whatever on the components?

      They did something similar with the modified 747 that carries the shuttle orbiter back after landing. See this picture. This is supposed to be a sign on top of the 747, where the orbiter links to the top of the 747's fuselage. It reads "Place Orbiter Here...Black Side Down".

      If this is real, they have one hell of a sense of humor.

      --
      Time flies like an arrow. Fruit flies like a banana.
    3. Re:Simple solution, really. by Kenja · · Score: 5, Funny
      "Could they not stamp "THIS SIDE UP" or whatever on the components?"

      There is no "up" in space....

      --

      "Have you ever thought about just turning off the TV, sitting down with your kids, and hitting them?"
    4. Re:Simple solution, really. by Anonymous Coward · · Score: 5, Funny

      Impossible? Nah, they'll just need a bigger hammer.

    5. Re:Simple solution, really. by Imperator · · Score: 4, Funny

      And I bet someone will manage to force it in backwards, breaking three other parts in the process.

      --

      Gates' Law: Every 18 months, the speed of software halves.
    6. Re:Simple solution, really. by s20451 · · Score: 5, Interesting

      This is the original definition of Murphy's Law: If it is possible to do something in more than one way, and one of those ways leads to catastrophe, someone will inevitably do it the wrong way.

      In fact the original inspiration for Murphy's Law was a G-force meter that was installed backwards, thereby taking meaningless readings. (It probably didn't go below zero.)

      Another example is the 1969 gearbox fire on the Canadian navy ship, HMCS Kootenay. A gearbox bearing was installed backwards, which restricted its flow of lubrication oil (on a naval vessel, the gearbox is the size of a car and absorbs tremendous loads). Apparently it did say which way to install it, but the installation was made in a foreign shipyard where the workers could not read English. The poorly lubricated bearing overheated and caused an explosion during a full-power trial; nine sailors were killed and dozens injured.

      The moral of all these stories is: if it's important which way something is installed, make it asymmetric so that it's physically impossible to install it the wrong way. Labels are not enough.

      --
      Toronto-area transit rider? Rate your ride.
    7. Re:Simple solution, really. by Ummagumma · · Score: 4, Interesting

      Dunno about the mortars, but current claymore mines are stamped as such. On the front curved side, its imprinted:

      FRONT TOWARDS ENEMY.

      Here is a pic

      --
      "The natural progress of things is for liberty to yield and government to gain ground." - Thomas Jefferson
    8. Re:Simple solution, really. by chadjg · · Score: 4, Insightful

      In some cases stamping lettering into a stressed rotating part can lead to sudden failure. Stamping can put odd stresses into the part, and makes a slightly weaker space in the part.

      I believe that the NTSB found a stamping on a rail car wheel caused a failure and derailment, but i'll be danged if I can find the incident. Maybe tonight...

      Moisture could collect in the little crevices of the letters leading to corrosion. i suspect that this is not really a concern.

      Silk screening or some creative powdercoating could totally avoid these issues. But what do I know. I managed to smoke my e-machine 500 last night. Nobody would hire me to be a shuttle engineer!

      --
      Why do I have this? I don't smoke.
    9. Re:Simple solution, really. by Stitch_626 · · Score: 5, Funny

      They could stamp "If you can read this, turn gear around and install correctly" on one side and "There you go Einstein...you finally got it right" on the other.

      --
      Ohana means family. Family means nobody gets left behind or forgotten.
    10. Re:Simple solution, really. by bjohnson · · Score: 4, Funny

      Damn straight!

      Quick...it's total darkness, the bad guys are on your trail and you have 15 seconds to place and rig the claymore, or you're all gonna get captured or killed.

      Yes you want it to be as completely foolproof as possible.

  2. How can a fault go unnoticed for so long? by amigoro · · Score: 5, Interesting
    Discovery flew safely 30 times with the defective actuator since its first launch August 30, 1984, and no one suspected a problem until the actuator was taken apart to check for corrosion.

    I thought they even checked Airplanes more thoroughly

    --


    Nothing to see here
    1. Re:How can a fault go unnoticed for so long? by Ryan+C. · · Score: 4, Insightful

      They found the problem before anything went wrong while checking for defects. I don't know what your definition of QA is, but that's pretty much the accepted definition.

      As for horribly flawed: Compared to whom? Spaceflight is dangerous. Minor oversights that in most industries would cause a misprint in a news article tend to blow up and kill people.

      Basically, until you land something off-planet, you have no room to talk. If you want to point to someone who is doing a better job of it, you pretty much have the Russians for comparison, and they have had even more problems.

      -Ryan C.

      --
      -Ryan C.
  3. Instead of slamming NASA by 0x0d0a · · Score: 4, Interesting

    ...and I'm sure there will be lots of negative posts about NASA here...

    It'd be nice to give some credit for the people that have put in layer upon layer upon layer of safeguards to check for exactly this sort of thing and the dilligent people that find this stuff. And caught it.

    The awful thing is that this is going to be just another reason for Congress to loot the NASA money bag.

    1. Re:Instead of slamming NASA by molo · · Score: 4, Informative

      It'd be nice to give some credit for the people that have put in layer upon layer upon layer of safeguards to check for exactly this sort of thing and the dilligent people that find this stuff. And caught it.

      Maybe you missed the details, but this has been in place on the Discovery for over 20 years.

      -molo

      --
      Using your sig line to advertise for friends is lame.
    2. Re:Instead of slamming NASA by GreyPoopon · · Score: 4, Insightful
      ...but please spare me the pity party because Congress may want to save me a few BILLION dollars.

      This might make sense if Congress (or any group of politicians) was actually interested in SAVING money anywhere. No tinfoil hat is needed to know that anytime a politician takes money away from one line item in the budget, they are merely diverting it to another that involves their own interests.

      --

      GreyPoopon
      --
      Why is it I can write insightful comments but can't come up with a clever signature?

  4. Transparency. by commo1 · · Score: 5, Interesting

    I'm quite surprised they're being quite so upfront about this. Kudos to them... On the other hand, I believe it to be a part of the healing process to convince the general public that they are, in fact taking the Columbia disaster extremely seriously, and want to show progress in the inspection and faliure-cathing procedures that obviously did not work for Columbia.

    It was, however, just a matter of time before a Columbia-type disaster occured. The suttle program has a remarkable safety record, Challenger and Columbia no matter.

  5. Geez... by Anonymous Coward · · Score: 4, Funny

    Come on NASA, it's not rocket science! Oh wait...

    1. Re:Geez... by Hays · · Score: 4, Interesting

      Quoth John Carmack

      "I'm kind of at the top of my field [in gaming]," he said. "When I started reading about aerospace, I realized there was an incredible level of things to learn. ... There's this mytholigization of aerospace that it's the hardest thing you can do. That's just not true. In terms of actual difficulty, it's not that hard. Aerospace is plumbing with the volume turned up."

    2. Re:Geez... by Unknown+Kadath · · Score: 4, Informative

      Like all high-tech endeavors, "rocket science" is a blend of many different fields. I happen to think that electrical engineering is far more difficult than aerospace engineering ever could be, but I have helped EE friends with their required mechanical engineering classes, and they got stuck on things I thought were simple and obvious. I'm sure they felt the same way when the time came to help me with my EE requirement. Ditto me and my CS friends. A lot of it is a matter of training and experience.

      I'm qualified to work on things like airframes and engines, and I can calculate a pretty mean orbit, if I do say so myself. But I'm lost when it comes to things like avionics or heat shield design. So "rocket science" is indeed complex and tricky, and a successful rocket design will require experts from many fields. But things like compressible flow, which seems to be what Carmack's talking about, aren't really outside the grasp of a dedicated student at all. And of course, all of this sounds like black magic to the nontechnical layman.

      Of course, we don't go around telling people this, or we wouldn't be able to look down our noses at everyone else. "I design jet engines, and I've done some work on the Mars program. Oh, you write computer games? Aw, that's cute." ;)

      -Carolyn

      --
      Like Daddy always said: if you can't dazzle 'em with brilliance, baffle 'em with bullshit.
  6. I'm betting... by Anonymous Coward · · Score: 5, Funny

    ...I'll be running Duke Nukem: Forever on Microsoft Longhorn before the next shuttle launches.

  7. Should have used Java by iamacat · · Score: 4, Funny

    NASA Finds Critical Assembly Fault in Shuttle

    I know NASA is conservative with technology, but using assembly in this day and age is way backwards!

  8. Just goes to show.. by iswm · · Score: 4, Interesting

    They should really do some double checking on this stuff. It's hard to imagine mistakes like this happen when dealing with something that holds the fate of a handfull of people's lives; not to mention all the millions of dollars put into these projects that would go down the drain. When dealing with people's lives and huge sums of money it's worth it to go over _everything_, and put in for better training so these thing don't happen again. They caught it this time, but if they don't take enough precaution, they might not be so lucky in the future.

    --
    Buckethead
  9. Twice?! by 2marcus · · Score: 5, Interesting

    And if you read the article, you realize that NASA installed defective actuators not once, but twice! The first being the one that was successfully flown 30 times, and the second in the spare actuators.

    Given the complexity of a system like the shuttle, it is not surprising that out of 1000s of components there could be a mistake in one of them (and given some redundancy and robustness, it is not surprising that the shuttle could fly 30 times with one or more poorly installed components, though one would not normally want to bet on that...).

    However, two errors out of 8 actuators checked implies some serious quality control issues.

    -Marcus

  10. When is a problem a problem? by stecoop · · Score: 4, Insightful

    Discovery flew safely 30 times with the defective actuator

    When does a defect become a problem? I wonder if this was really a Critical problem because shouldn't some indication have already been seen by now?

    I mean since they have fixed this problem will two other problems surface that are more critical and maybe they should have left it alone?

  11. Engineering practices by n3xup · · Score: 4, Insightful

    In engineering, it's usually good practice to design somthing that only assembles one way. That way, whoever is assembling it (no matter how intelligent they are) can only install the component the way it was meant to be.

    It's strange and somewhat disconcerting that this was not the case for this shuttle component, but I haven't seen the part in question.

  12. Re:Murphy's Law by AlterTick · · Score: 4, Insightful
    "If there are two or more ways to do something, and one of those ways can result in a catastrophe, then someone will do it."

    The important lesson here is about the design of critical parts. Nothing important should be made asymetrical and reversable. Even labeling it "THIS SIDE UP" on one side and "WRONG! DANGER! WRONG!" on the other isn't good enough. The part should either be symetrical, so it doesn't matter which way it goes, or non-reversable, so it wont fit upside down/backwards. Important thing to remember in mechanical engineering (gears|levers|*) or even eletrical parts specifications (connectors should be keyed to mate only one way: the right way).

    --
    Conclusion: the Empire squashes the Federation like a bug. Accept it.
  13. This happens: See the F-111 program by dougermouse · · Score: 4, Interesting

    The same basic thing happened with the F-111 program in the 60s. The drawing had a piece that was installed upside down, but the technician installing them said, this ALWAYS goes right side up and installed it that way. A couple crashes and the grounding of the whole F-111 fleet later, and the trouble was found. I don't know what happened to the installer, but I can't imagine it was any good. Check twice, install once :)

  14. Stuff happens; learn from it. by Kozar_The_Malignant · · Score: 5, Interesting

    In this case nobody died and several lessons were learned, including something about fault-tolerance in actuators. I think two of the most valuable space flights from this point of view were Apollo 13 and the Mir mission that caught fire.

    Things will go wrong. Learning how to cope when the evil wind blows is critical. In this case, we now know that the thing can be flown with one actuator in upside down. If the bottom one malfs, swap it out in orbit with the top one, and you still might get home. People are going to get killed doing this. People got killed learning to sail the Mediterranean. It's still worth doing.

    --
    Some mornings it's hardly worth chewing through the restraints to get out of bed.
  15. manual by Johnny+Mnemonic · · Score: 4, Interesting


    I can imagine the guy that noticed this first. Probably went something like: looks at actuators. looks at diagram of how they're to be installed. looks at diagram again. looks at actuators. turns diagram around; notices that the legend is now upside down, so concludes that can't be it. checks other pages of diagram to see if this page is unusual--different view, maybe. finds that it isn't. checks back for errata. finds none.

    Looks around. "Hey Bob, what do you make of this?" Thinks about all the work that day that isn't going to get done, because now management and, if he's lucky, congressional inspectors are going to crawl up his ass. At least he knows that he didn't *install* the things.

    --

    --
    $tar -xvf .sig.tar
  16. This stuff happens all the time by Unknown+Kadath · · Score: 4, Interesting

    I work in New England, contracting for a jet engine manufacturer (and you can get it in two if you know the aero industry). Things like this happen frequently in manufacturing, especially with development hardware, before the kinks have been worked out of the assembly process and parts are ready to go to production. Assembly mistakes range from things that are easy to do but also easy to fix, like cut or cracked O-rings and tool knicks on non-critical parts, to things that are real screw-ups and result in major headaches, like parts left out entirely or vital parts being installed incorrectly and badly damaged because of it. You could consider the entire shuttle program to still be development-phase engineering, since only a few shuttles were ever built.

    An example: a while back, we had a test engine spewing fuel all over the test cell for no readily apparent reason, prompting a panic that an entire compartment of the engine would have to be redesigned from scratch--until one of the test engineers found a fuel line seal that had not been reinstalled in the engine after the last teardown and reassembly. How do you miss something like this when there's a careful set of instructions to follow for every step of the assembly? I don't know, but I do know that humans are fallible, so we are constantly dealing with a stream of lost, damaged, and defective parts. Anyway, they put the seal back in, and the engine worked fine. (I have an NDA, so this is not what actually happened, but it is analogous.)

    When I was in school, the more I learned about the environment the shuttle operates in, the more I was impressed by the fact that it worked at all, and now that I'm learning more about manufacturing engineering (not what I studied for; stupid job market), I'm surprised that the shuttles have as few problems as they do.

    -Carolyn

    --
    Like Daddy always said: if you can't dazzle 'em with brilliance, baffle 'em with bullshit.
  17. Nasa Haters... by bobej1977 · · Score: 4, Interesting
    I think it's worth saying that very few of us would be capable of the high standards that go into the design, fabrication and assembly of these crafts. Quoth Nasa:
    "...[Discovery is] assembled from more than 2.5 million parts, 230 miles of wire, 1,060 valves, and 1,440 circuit breakers..."
    I for one stand amazed at what Nasa accomplishes every single day. Could they do better? Certainly. Would we do better? Almost certainly not.

    On a side-note, the reason Nasa is stuck in the proverbial hard-place between multi-billion dollar budgets and missions that nobody cares about is that we've all started over-valuing human-life. It's ridiculous that space exploration all but stopped because of the 2 shuttle disasters. Certainly, the loss of those crews was tragic, but the best way to honor those crews is to relentlessly pursue the dream that they died for, not hamstring ourselves being overly cautious.

    Call me old-fashioned, but I still believe there are things more important than one or a dozen human lives. IMO, exploring the universe is one of them.

    --
    The meek shall inherit the earth, in 3 by 6 plots. - Lazerus Long
  18. Re:Why are heads not rolling? by pz · · Score: 4, Insightful

    When the Mars Climate Orbiter was lost due to a mixup between NASA and Lockheed Martin concerning imperial vs. metric units, I recall Dan Goldin, the then-head of NASA, being asked by a reporter if heads would roll. He replied something like, "sure we could do that, but then we'd have to replace those engineers. Now we already hired the best ones we could find in the world, so where are we to get better ones?"

    The Shuttle is one of the (if not *the*) most complex bits of engineering we, as a world-wide society, have achieved. I would never expect any shuttle to be manufactured/assembled/flown/controlled/maintained perfectly without flaw. I would, however, expect it to be designed to work well in the face of such problems, as does seem to be the case here. Discovery had 30 successful flights despite a part on a major control surface being installed backwards? Everyone landed safely? Tons of good science, too? Doesn't mean the problem shouldn't be fixed, but kudos to NASA for what sounds to me like a job well done.

    --

    Put my fist through my alarm clock with its ding-dong death inside my ear. - The Blackjacks.