Slashdot Mirror


Missing Files Blamed For Deadly A400M Crash

An anonymous reader writes: Think you had a bad day when your software drivers go missing? Rejoice, you get to live! A fatal A400M crash was linked to data-wipe mistake during an engine software update. A military plane crash in Spain was probably caused by computer files being accidentally wiped from three of its engines, according to investigators. Plane-maker Airbus discovered anomalies in the A400M's data logs after the crash, suggesting a software fault. And it has now emerged that Spanish investigators suspect files needed to interpret its engine readings had been deleted by mistake.This would have caused the affected propellers to spin too slowly causing loss of power and eventually, a crash.

9 of 253 comments (clear)

  1. Good god. by Anonymous Coward · · Score: 5, Insightful

    Is it so hard to have a integrity check and diagnostic set run as part of the preflight checks? If you can place hundreds of miles of wire and know what's what, surely they have computer engineers competent enough to make something like this to catch such glaring errors.

    1. Re:Good god. by Anonymous Coward · · Score: 5, Insightful

      We've lost that kind of 'slow down and make sure it's right' attitude that engineers really need to have. In this fast-paced road of cutting costs and letting the marketing group run the show, the pressure to get product out the door as quickly as possible no matter what is unstoppable for software in particular, but really almost anything that is able to be 'patched' later. Making consumers into your beta testers is douche-y enough, but doing it when lives are at stake should be punished as criminal and in an extremely harsh and public way.

    2. Re:Good god. by Anonymous Coward · · Score: 5, Insightful

      Read the article... the warning was not designed to kick in until the aircraft was at an altitude of 400ft (120m).

      Not only do you not know you have a problem until are in the air. You don't know you have a catastrophic problem until you are at an unsurvivable altitude. Too low to effectively use a parachute. Too high to just 'jump out' or belly-land it.

      The worst thing is... a committee signed off that this was an 'acceptable risk'. Members of that committee should be brought of up on criminal negligence and manslaughter charges.

      Not a Luddite, but give me my bicycle back...

    3. Re:Good god. by TWX · · Score: 5, Insightful

      If I remember right, Toyota got into trouble in court when the firmware provided to the investigators did not match the firmware in the vehicles. Toyota never did provide the real code if memory serves.

      --
      Do not look into laser with remaining eye.
    4. Re:Good god. by tlhIngan · · Score: 5, Insightful

      They screwed up, yes, but if they would be "punished as criminal and in an extremely harsh and public way" nobody would ever do anything useful anymore. The problems leading to this crash have to be analyzed and understood and then they have to make sure that the same thing can't happen again.

      You know, when an accident happens, the safety board (NTSB, TSB, BEA, etc) interviews are actually privileged information. As in, if you're being interviewed by the safety board, anything you say cannot be used as evidence against you.

      It's a privilege that the safety boards all fight for.

      The reason for this is the safety board's goal is to not find fault, but to find solutions to preventing it from happening again. Doesn't matter if someone hit a button that said "Crash this plane" and pushed it on purpose. They know that if the interviews were not privileged communications, no one would speak to them for fear of self-incrimination. And when that happens, everyone clams up, and you can't figure out why an accident happened or make recommendations to prevent the issue the next time it happens.

      This is especially more so when most complex accidents are a chain of events - this happened, then that happened, then this next thing, plus X, Y and Z and if any of them didn't occur, the accident wouldn't have happened. Almost never is it the result of one definitive action.

  2. Re:This is what happens when you use Luddite softw by fuzzyfuzzyfungus · · Score: 5, Interesting

    Depressingly, that might actually be true.

    Not because of 'apps' of course; but because no self-respecting consumer OS would fail to cryptographically verify the execution environment(lest some precious 'premium content' be absconded with by pirates) and an entire missing file probably would have caused the aircraft to refuse to move until taken back to Airbus HQ for re-blessing by the vender.

    They don't succeed against motivated pirates, of course; but this is one area where consumer software vendors do actually give a fuck. If people believed that a sabotaged voting machine or a defective ECU could pirate Blu-rays, we'd live in a safer world.

  3. Big fail from the software engineering standpoint. by Frosty+Piss · · Score: 5, Interesting

    Just my take as a software engineer and current DoD employee that works with C17...

    There should have been some process on firing up the jet / avionics / computers that ran checks to see that even if software was not latest, was it CONSISTENT?

    Big fail from the software engineering standpoint.

    --
    If you want news from today, you have to come back tomorrow.
  4. Return codes? by cfalcon · · Score: 5, Insightful

    This is a tragedy, but since we're on a tech site, lets talk tech.

    Return values are handled oddly in pretty much every major language. Many API calls want to return something simple- int or bool- and if anything is more complex than that, generally require an actual data structure to be returned, often as a reference. This means that the "I didn't do this" action has a variety of ways to be be passed back- none of them even close to standard.

    If something returns a distance, magnitude, or size, "0" normally means "Error, nothing happened" which is often the same as "Sure, I wrote 0 bytes. Really."
    If something needs to distinguish between success ("I did the thing 0 times as requested" and failure "I couldn't do the thing because of an error condition"), then sometimes a -1 is returned, or an exception thrown, or something else.

    In this plane, something was, at some point, responsible for getting data about the engines. Likely, this happened in layers, each one having access to the results of the lower pieces. One of those pieces had the task of parsing those files.

    So EITHER someone (process, program, whatever) meant to say "This is a problem" and instead said "Here's some default data", OR someone ELSE in that chain of commands (process, program, whatever) has a default for a "This is a problem" result to use as a failsafe, and it was never tested or never communicated up.

    We probably won't get the technical details that go from "files missing" to "engines don't work". Certainly, several level of software or hardware could allow for any number of workarounds in this case, and I'm sure they have a complex system and this was some eventuality that was hard to test for.

    Still, interesting to think about the error return methodology, and how it's so different everywhere in CS.

  5. Re:So, how did ... by Anonymous Coward · · Score: 5, Informative

    The story seems to massively simplify how the ECUs work. Each engine needs to be calibrated after production so that the sensor data it hands to each ECU is actually meaningful due to the way it's actually acquired in the engine. The parameter set isn't stored in the engine, but in the associated ECU. To prevent them from getting out of sync, the engine itself contains a little register with the checksum of the parameter set. If that checksum doesn't match, the ECU shouldn't power up the engine. However, the register and the ECU are initially loaded with a default parameter set used in testing scenarios. Looks like that one might have been untouched for the engines on that flight. Now, this is bad because the ECU now misreads the true engine status in various ways and can even think that an engine which is otherwise running fine is seemingly in some critical condition - e.g. power output too high, which causes an immediate shutdown to prevent engine damage. A jet engine that fails by disintegration has a high chance of slicing other airplane parts with ripped off fan blades. This is why hard engine shutdowns do make sense. But when putting the pieces of this puzzle together, this is starting to look similar to how Murphy's law came to be: an exceptionally unlikely chain of human errors ruining everyone's day.