Slashdot Mirror

← Back to Stories (view on slashdot.org)

ESA: European Mars Lander Crash Caused By 1-Second Glitch (space.com)

Posted by BeauHD on from the what-went-wrong dept.

An anonymous reader quotes a report from Space.com: The European Space Agency (ESA) on Nov. 23 said its Schiaparelli lander's crash landing on Mars on Oct. 19 followed an unexplained saturation of its inertial measurement unit (IMU), which delivered bad data to the lander's computer and forced a premature release of its parachute. Polluted by the IMU data, the lander's computer apparently thought it had either already landed or was just about to land. The parachute system was released, the braking thrusters were fired only briefly and the on-ground systems were activated. Instead of being on the ground, Schiaparelli was still 2.3 miles (3.7 kilometers) above the Mars surface. It crashed, but not before delivering what ESA officials say is a wealth of data on entry into the Mars atmosphere, the functioning and release of the heat shield and the deployment of the parachute -- all of which went according to plan. In its Nov. 23 statement, ESA said the saturation reading from Schiaparelli's inertial measurement unit lasted only a second but was enough to play havoc with the navigation system. ESA said the sequence of events "has been clearly reproduced in computer simulations of the control system's response to the erroneous information." ESA's director of human spaceflight and robotic exploration, David Parker, said in a statement that ExoMars teams are still sifting through the voluminous data harvest from the Schiaparelli mission, and that an external, independent board of inquiry, now being created, would release a final report in early 2017.

24 of 110 comments (clear)

  1. This never happened to me before... by hyades1 · · Score: 5, Funny

    Man, if I had a nickel for every time some kind of sensory saturation forced a premature release...

    --
    I've calculated my velocity with such exquisite precision that I have no idea where I am.
    1. Re:This never happened to me before... by rsmith-mac · · Score: 5, Funny

      Man, if I had a nickel for every time some kind of sensory saturation forced a premature release...

      Then you'd still be broke. This is Slashdot; you're not fooling anyone.

    2. Re:This never happened to me before... by Freischutz · · Score: 2

      Man, if I had a nickel for every time some kind of sensory saturation forced a premature release...

      Then you'd still be broke. This is Slashdot; you're not fooling anyone.

      Visit the lair of any Slashdot poster, buried deep in the basement of his parent's house, and you will find that the height and majesty of the tissue mountain on the nightstand next to his bed thoroughly discredits your hypothesis. If you need further confirmation, shine a black light at his laptop and prepare yourself to be blinded by the glow.

    3. Re:This never happened to me before... by silentcoder · · Score: 3, Insightful

      He never said there was another person involved. He's just complaining about never managing to make it to the end of the pornhub clip.

      --
      Unicode killed the ASCII-art *
  2. Cheater? by Anonymous Coward · · Score: 2, Funny

    They're blaiming lag?

  3. Kalman filter by little1973 · · Score: 4, Insightful

    https://en.wikipedia.org/wiki/...

    How in hell did they test their Kalman filter to allow such bad data to reach the decision logic? (I assume they used one.)

    --
    Government cannot make man richer, but it can make him poorer. - Ludwig von Mises
    1. Re:Kalman filter by gTsiros · · Score: 3, Insightful

      I find it more weird that *one* sensor misbehaving lead to the entire mission failing.

      I have more robustness in my thrust measuring rig made of wood beams and zipties :|

      --
      Looking for people to chat about multicopters, coding, music. skype: gtsiros
    2. Re:Kalman filter by 0100010001010011 · · Score: 2

      "Check for integer overflow" is a checkbox in Simulink.

      How was this not caught on the Hardware in the Loop test benches?

      Jesus people, is this amateur hour.

    3. Re:Kalman filter by vtcodger · · Score: 2

      "How in hell did they test their Kalman filter to allow such bad data to reach the decision logic? (I assume they used one.)"

      1) A Kalman Filter probably is not really appropriate here because the parachute has just been deployed and you wouldn't have state statistics available to filter the input data. Doesn't mean they didn't use one with ad hoc statistics. That's not as uncommon as perhaps it should be.

      2) Presumably the IMU is expected to tell you the probe has run into the planet (i.e. landed) and it's time to get rid of the 'chute before it lands on your probe and also time to shut down the thrusters lest they bounce the probe across the landscape or flip it upside down. Depending on how often the IMU is read out during landing, a full second of bad data may be pretty convincingly NOT noise.

      Not that I know anything about landing Mars probes.

      --
      You can't see ANYTHING from a car, You've got to get out of the goddamned contraption and walk...Edward Abbey
  4. They didn't learn the lesson by LordHighExecutioner · · Score: 4, Informative

    Overflows and bad data problems happened to ESA before.

    1. Re:They didn't learn the lesson by thegarbz · · Score: 5, Insightful

      To be fair, few people did. Multiple cases of overflows and bad data problems have occurred and still continue to occur not just in space programs around the world but in other industries too.

  5. Teach it Phenomenology! by Bongo · · Score: 2

    "Obligatory" Dark Star reference.

  6. Filter or not by evanh · · Score: 4, Informative

    When the altitude stops changing for a whole second the filter is going to have to be a long one! And that ain't desirable for responsive control.

    The real question is how could the sensory processor have overloaded in the first place? My money is on simple [b]code bloat[/b]. Ie: They used a bunch of generic libraries that use further libraries that use further libraries that use further libraries that use further libraries that use further libraries ...

    1. Re:Filter or not by Solandri · · Score: 4, Insightful

      Dynamic range. Sensors which can measure from 0 - 100 g's are not as sensitive in the 0-1 g range you may be more concerned about. So you instead opt for accelerometers which max out at 10 g's and try to deal with the periods of max acceleration in software.

      A more elegant solution is to use both the sensitive accelerometer and an accelerometer with a greater max threshold. That way you keep the higher max limit without giving up low-gain sensitivity. But spacecraft tend to be both weight- and budget-constrained...

      More troubling to me was that there wasn't some basic sanity checking going on. Like a calculation that says "3 seconds ago I was at 4 km high. Now I think I'm on the ground. Does it make sense that I could've traveled that far in that little time? No? Then the instruments saying I'm on the ground are probably wonky, and I should give other instruments a higher priority in calculating my altitude for a bit." Same way I write my code (and spreadsheets) to calculate important numbers two, three, or sometimes even four different ways to make sure they all agree before proceeding to act on it.

    2. Re:Filter or not by Kjella · · Score: 2

      More troubling to me was that there wasn't some basic sanity checking going on. (...) Same way I write my code (and spreadsheets) to calculate important numbers two, three, or sometimes even four different ways to make sure they all agree before proceeding to act on it.

      Well it's not exactly like the lander can abort, it's do or die. So you got inconsistent or unlikely data, but what's good and what's bad? It is a glitch, is it defective, did a misfire flip us around or put us in a spin or block the sensor? Can we salvage it or is the mission fucked no matter what? That's really the million dollar question, is there a contingency plan that could work and if so what should trigger it.

      I'm guessing that with combinatorics you'll have potentially very many possible failure modes and it's hard to find the realistic and salvageable ones while not screwing up non-sensor issues because you don't trust the data. I imagine you can spend a lot of time and resources on a few paths only to find it fails in a completely different way you didn't expect. It's a lot easier in hindsight...

      --
      Live today, because you never know what tomorrow brings
  7. What the? by NewtonsLaw · · Score: 5, Informative

    So they didn't correlate the IMU data with ranging radar or even barometric altitude information so as to avoid this?

    I know weight and volume are at a premium on such craft but a barometric sensor (even one capable of operating in Mars's rarefied atmosphere, is the size of a thumbnail and weighs just a fraction of a gram.

    Sigh!

    1. Re:What the? by thegarbz · · Score: 3, Insightful

      even barometric altitude information

      I'm interested to know how you calibrate your barometric altitude information, and even more so what vacuum followed by a sudden atmospheric entry will do to such a sensor.

      If I'm going to take a guess I'd so no, an instrument capable in operating that range of pressures, temperatures, vibration, etc is not the size of a thumbnail weighing a gram.

    2. Re:What the? by Ihlosi · · Score: 2
      I know weight and volume are at a premium on such craft but a barometric sensor (even one capable of operating in Mars's rarefied atmosphere, is the size of a thumbnail and weighs just a fraction of a gram.

      Even one that works at the velocity encountered during atmospheric entries?

      Sounds like you're suggesting putting a Pitot tube on a space probe ...

  8. Oops by wonkey_monkey · · Score: 4, Funny

    Should've used metric seconds.

    --
    systemd is Roko's Basilisk.
  9. Re:Ariane 5 by Anonymous Coward · · Score: 3, Informative

    > ...control software spat an Ada stack trace over a line...

    Eh, no. The failure of the INS's control software caused the INS to send diagnostic data (rather than sensor data) to the control systems, which then did what they _thought_ they were being commanded to do.

    None of the code in the system was modified in flight.

  10. Calculations were prescient by tomhath · · Score: 2
    FTFA:

    "[T]he erroneous information generated an estimated altitude that was negative," ESA said.

    Which resulted in an actual altitude that was negative.

  11. Open Source? by Midnight+Thunder · · Score: 2

    I wonder whether making the source code of these probes available to the public, for vetting would help spot bugs like these? I am also curious whether releasing the code would be problematic for any reason?

    --
    Jumpstart the tartan drive.
  12. Re: In retrospect by turbidostato · · Score: 2

    And then, what? Should it take an "instantenous" decision about what to do next?

  13. Sanity checking code by presidenteloco · · Score: 2

    So say they're doing some kind of weighted average of an altitude computation from the inertial navigation unit and an altitude computation from the doppler radar altimeter.

    They should have some code in there saying: If these two values that we're averaging are wildly off from each other, let's not take the average. Instead, let's go into some exception handling code which uses some kind of heuristic (and a little time perhaps) to determine which of the two instruments should become the solely trusted source of the altitude value.

    Sounds like a lack of hazard analysis / fault tree analysis and or fault-tolerant design in the design process.

    --

    Where are we going and why are we in a handbasket?