Slashdot Mirror


Upside-Down Sensors Caused Proton-M Rocket Crash

Michi writes "According to Anatoly Zak, the crash of the Russion Proton rocket on 1 July was apparently caused by several angular velocity sensors having been installed upside down. From the source: 'Each of those sensors had an arrow that was supposed to point toward the top of the vehicle, however multiple sensors on the failed rocket were pointing downward instead.' It seems amazing that something as fundamental as this was not caught during quality control. Even more amazing is that the design of the sensors permits them to be installed in the wrong orientation in the first place. Even the simplest of mechanical interlocks (such as a notch at one end that must be matched with a corresponding projection) could have prevented the accident." A review of the quality control procedures used by the contractors responsible is underway.

20 of 323 comments (clear)

  1. vodka and work don't mix by alen · · Score: 4, Interesting

    being from there i bet half the people working on this came to work drunk and/or hung over most days

  2. Re:QA is not the problem by jeffmeden · · Score: 5, Insightful

    What seems more amazing is that a simple software check pre-launch (i.e. "do all the sensors think they are pointed up?") was not part of the SOP. Given that their exact function is orientation detection, skipping the opportunity for self-test via that function is somewhat baffling.

    Obligatory: It's not rocket science!

  3. Wrong hemisphere by jovius · · Score: 4, Funny

    Should have launched from Australia.

  4. Shades of the US "Genesis" sample return probe... by Ellis+D.+Tripp · · Score: 4, Interesting

    which plowed into the desert floor without deploying any parachutes because a G-switch was installed backwards...

    http://www.universetoday.com/73/genesis-accident-report-released/

    --
    Remember "News for Nerds, Stuff that Matters"? Help make it a reality again! http://soylentnews.org
  5. Re:In Soviet Russia by RenderSeven · · Score: 5, Funny

    Perhaps the thinking is, as long as the arrow isnt pointed at you it's probably safe.

  6. Re:QA is not the problem by jrumney · · Score: 4, Insightful

    The greatest pleasure my toddler ever got from his shape sorter was when he discovered that the 3 could be forced through the hole for the C. Never underestimate the satisfaction a disgruntled office worker gets from jamming the ink block into the printer the wrong way around.

  7. Re:(The Real) Murphy's Law strikes again! by MickLinux · · Score: 5, Interesting

    Yes, the real, original Murphy's law apparently came from Col. Stapp, who was testing rocket sleds for the rocket program.

    I should note that the putative original Murphy's Law reads, "If there are two or more ways to do something, and one of those ways can result in a catastrophe, then someone will do it." . The website goes on to say "This is a principle of defensive design, cited here because it is usually given in mutant forms less descriptive of the challenges of design for lusers. For example, you don't make a two-pin plug symmetrical and then label it `THIS WAY UP'; if it matters which way it is plugged in, then you make the design asymmetrical."

    Highly appropriate to the topic, I might say. If only they had labeled, with the arrow, the words "up", and put another arrow down, with the letters "dn" for "down", then none of this would have happened.

    For those who wish to nit-pick my attention to detail and editing, also, I will for further irony include the wikipedia link, as well: http://wikipedia.org/wiki/Muphry%27s_law

    --
    Correct Horse Battery Staple: 72 bits of entropy. Enter "Correct H" into google. When it generates the phrase, that's
  8. Re:QA is not the problem by TheCarp · · Score: 5, Informative

    Amusingly, when someone actually attempted to track down who murphy was, and where the law came from.... http://en.wikipedia.org/wiki/Murphys_law

    Edward Murphy proposed using electronic strain gauges attached to the restraining clamps of Stapp's harness to measure the force exerted on them by his rapid deceleration. Murphy was engaged in supporting similar research using high speed centrifuges to generate g-forces. Murphy's assistant wired the harness, and a trial was run using a chimpanzee.

    The sensors provided a zero reading; however, it became apparent that they had been installed incorrectly, with each sensor wired backwards. It was at this point that a disgusted Murphy made his pronouncement

    So this is potentially, very much related to the original usage.

    --
    "I opened my eyes, and everything went dark again"
  9. Re:The quality conrol problems... by PetiePooo · · Score: 5, Informative

    ...aren't so amazing when you look at the track record of Russian manufacturing.

    Before we Americans point too many fingers, let's not forget NASA is not immune to similar mistakes.

  10. Re:(The Real) Murphy's Law strikes again! by PetiePooo · · Score: 5, Informative

    My favorite is Cole's Law...

  11. Re:QA is not the problem by Athanasius · · Score: 4, Insightful

    My reading of 'angular velocity sensor' is that they're meant to sense rotation. If you're sat stationary on the pad there is no such rotation and thus you'll get a 'correct' zero reading. You'd have to perform such a test during some known movements of the rocket (part).

  12. Re:QA is not the problem by Baloroth · · Score: 5, Informative

    What seems more amazing is that a simple software check pre-launch (i.e. "do all the sensors think they are pointed up?") was not part of the SOP. Given that their exact function is orientation detection, skipping the opportunity for self-test via that function is somewhat baffling.

    Obligatory: It's not rocket science!

    The sensors in question were for angular velocity. Given that pre-launch the craft doesn't have any (peculiar) angular velocity, the sensors would return the correct results (zero) no matter how they were installed.

    --
    "None can love freedom heartily, but good men; the rest love not freedom, but license." --John Milton
  13. Re:QA is not the problem by KiloByte · · Score: 5, Interesting

    An old joke:

    A militia (communist police) station has been ordered to conduct an intelligence test. It consisted of a board with three holes: a circle, a triangle and a square, and three corresponding blocks. The next days, the commandant announces: I'm very proud of our station: all of you passed the test! 5% have shown exceptional intelligence, 95% exceptional strength!

    --
    The creatures outside looked from Alt-Right to Antifa; but already it was impossible to say which was which.
  14. Mars orbital failure by Okian+Warrior · · Score: 5, Insightful

    The US once sent a probe all the way to mars, only to have it fail because the ground computer was in imperial units while the orbiter was in SI units.

    Getting everything correct is hard... really hard. For most projects you have elaborate "fail gracefully" modes which rely on external agents to notice the problem and take action. A doctor or pilot can take appropriate action, but it's hard to do with rockets.

    For comparison, I wrote the software for the altimeter that goes into some 747 aircraft. Total of about 21,000 lines of C, about 40% comments so figure 12,000 lines of code. The testers (and I) worked really hard to find all bugs in the system, knowing that a mistake could knock a plane out of the sky. There were elaborate internal checks both in software and process, and Boeing did their own testing on top of ours. Everything passed, all requirements were met, things looked good.

    The device had 1 bug, found after installation. A software typo which wasn't caught by QA even though it had a specific testing requirement. No one was negligent, it just slipped by despite best efforts.

    Multiply this by all the devices in an aircraft, and add in the other engineering disciplines like electronics and mechanical. It's really hard to get everything right all at once, and on the first try.

  15. heh by wbr1 · · Score: 4, Funny

    In Soviet Russia, snesors installed correctly, rocket installed upside down.

    --
    Silence is a state of mime.
  16. Re:QA is not the problem by LordLimecat · · Score: 4, Funny

    Ive seen RAM modules installed backwards. "Wait!", you say, "Isnt there a notch which prevents that?" Well, yes, there WAS a notch...

  17. Re:(The Real) Murphy's Law strikes again! by akh · · Score: 5, Funny

    If only they had labeled, with the arrow, the words "up", and put another arrow down, with the letters "dn" for "down", then none of this would have happened.

    Except that "dn" upside-down is indistinguishable from "up". Murphy strikes again?

    --
    Accept Eris as your Fnord and personally sate her
  18. Re:The quality conrol problems... by 0123456 · · Score: 4, Insightful

    Communism (n) - an unattainable standard that is constantly held up as a model of perfection despite having no functional real world example past or present.

    Sorry, but that's nonsense. All you need to do to create perfect communism is kill everyone else so no-one can disagree with you (you can't just kill the ones who disagree, because the others might only be pretending to agree).

    Stalin made a pretty good attempt, but didn't quite succeed.

  19. Re:The quality conrol problems... by Immerman · · Score: 4, Insightful

    >That would be like saying that disaster relief volunteers are an example of communism
    Actually that sounds about right to me. "From each according to their ability, to each according to their need" and all that.

    Most family households also run on at least partially communist principles. Luxuries may need to be at least partially earned (or not, plenty of douches with entitlement issues out there), but it's a sad family where everyone's *needs* aren't taken care of first.

    The problem with communism seems to be that it doesn't seem to scale well beyond the tribe/monastery/commune level. Once the population gets too large to allow for effective communal decision making, communal ownership tends to become de-facto ownership by the decision makers, massively exacerbating the problem of corruption.

    --
    --- Most topics have many sides worth arguing, allow me to take one opposite you.
  20. Re:The quality conrol problems... by wagnerrp · · Score: 4, Interesting

    Actually, that's wrong. Capitalism is designed with corruption and greed in mind. Greed motivates entities to perform better in the market and get more stuff. Corruption is dealt with by entities shifting to competitors who are screwing them less. Capitalism's failure is in assuming all involved entities are sufficiently intelligent to be aware of when they're being screwed, and principled enough to forgo what they want to avoid being screwed.