Slashdot Mirror


Software Error Caused Soyuz/Galileo Failure

schwit1 writes An investigation into the recent failed Soyuz launch of the EU's Galileo satellites has found that the Russian Fregat upper stage fired correctly, but its software was programmed for the wrong orbit. From the article: "The failure of the European Union’s Galileo satellites to reach their intended orbital position was likely caused by software errors in the Fregat-MT rocket’s upper-stage, Russian newspaper Izvestia reported Thursday. 'The nonstandard operation of the integrated management system was likely caused by an error in the embedded software. As a result, the upper stage received an incorrect flight assignment, and, operating in full accordance with the embedded software, it has delivered the units to the wrong destination,' an unnamed source from Russian space Agency Roscosmos was quoted as saying by the newspaper."

24 of 157 comments (clear)

  1. In other news... by msauve · · Score: 5, Funny

    A software error in Russian GLONASS receivers has resulted in thousands of Russian troops innocently crossing the border into Ukraine.

    --
    "National Security is the chief cause of national insecurity." - Celine's First Law
    1. Re:In other news... by rubycodez · · Score: 5, Funny

      two months ago software error in a 9M317 missile controlled by a BUK missile system rendered it unable to avoid being struck in midair by the careless pilot of Malaysian Airlines Fligh 17MH. Sadly, the missile was a total loss.

  2. Misleading title... once again by x0ra · · Score: 2

    From the linked article (emphasis mine): "Galileo Satellites Incident Likely Result of Software Errors", there is still an uncertainty. Though, I guess I should not be surprised, this is /. afterall....

  3. Pfffft by Tablizer · · Score: 5, Funny

    It's not like it's rocket science to get it right

  4. Given current tensions, ... by theycallmeB · · Score: 4, Interesting

    the strategic value of satellite navigation and general asshole-erly at the top of the Russian government, I am guessing that Europe's very expensive satellites ended up exactly where Russia wants them.

    1. Re:Given current tensions, ... by Guppy06 · · Score: 4, Interesting

      The Russian GLONASS has its own problems, and the whole point of Galileo is a GNSS that is independent from the US. Do you think the Russians like falling back on US technology? Or do you think they're planning to rely on Beidou?

  5. Re:Russian Programmer's are Brilliant! by viperidaenz · · Score: 2

    Ukraine is Russia now

  6. Not A SW error! by Anonymous Coward · · Score: 5, Insightful

    This is not a SW error! The software put them right where they were told to. The orbital parameters were wrong! This is a data error not a SW error!

    1. Re:Not A SW error! by Kittenman · · Score: 2

      More like a failure to double check settings or something.

      - "Are you really sure you want to trash those two satellites?" <click> - "Did you get your boss's approval?" <click>

      Or... the Russian version of Clippy,..

      "Hi - it looks like you're trying to trash two satellites. Do you want a hand with that?"

      <click>

      --
      "The greatest lesson in life is to know that even fools are right sometimes" - Winston Churchill
  7. "Programmers" shouldn't write critical software by Brannon · · Score: 5, Insightful

    There's almost no overlap between the skills & techniques necessary to write & verify critical software (e.g. when lives or huge amounts of money are on the line) vs. what is considered to be "programming". Modern software engineering's approach to reliable system design is about where hardware engineering was fifty years ago, and about where civil engineering was 100 years ago.

    SQA is a joke. Reliable systems are made using way more robust techniques, including: (a) a severely restricted state space, (b) redundancy, (c) formal proofs, (d) fully (and formally) specified interfaces, (e) random simulation, (f) several different types of coverage, (g) physics-based analysis, etc.

    The failure of the software community to understand this distinction is why I'm scared to death about the coming world of driver-less cars and robots performing surgery. How many people are going to be killed by C++ in the next decade?

    1. Re:"Programmers" shouldn't write critical software by ShanghaiBill · · Score: 4, Insightful

      I'm scared to death about the coming world of driver-less cars and robots performing surgery.

      Your fears are not rational. Self driving cars and robotic surgeons are tested for thousands of hours, under live conditions. SDCs are not perfect, but they already have a far better safety record than the average human driver. I had LASIK eye surgery done by a robot. I trusted it far more than I would a human surgeon. Getting rocket software right is difficult precisely because there is no way to do a live test. It has to work perfectly on the very first attempt. Very few other applications have such a severe constraint.

      How many people are going to be killed by C++ in the next decade?

      A lot fewer than would have died without it.

    2. Re:"Programmers" shouldn't write critical software by gl4ss · · Score: 5, Insightful

      it seems to me that in this case the programmers job was done 100% perfect.

      but the program was given wrong place to take the satellites to.

      --
      world was created 5 seconds before this post as it is.
    3. Re:"Programmers" shouldn't write critical software by Type44Q · · Score: 4, Funny

      Your fears are not rational.

      Just because he's paranoid doesn't mean C++ isn't out to get him...

    4. Re:"Programmers" shouldn't write critical software by jythie · · Score: 2

      I am always surprised when I go to interview at a company and their test team is actually smaller then their development one.

    5. Re:"Programmers" shouldn't write critical software by Giant+Electronic+Bra · · Score: 2

      Ah, I just love these sorts of pronouncements.

      When was the last time you heard of a 747 crashing because of a software glitch? My first job was to verify the design and implementation of a major part of the flight software for that aircraft, so I'm kind of an expert on this subject. You have no idea how multi-faceted and sophisticated the verification and SQA processes are on these projects. First of all formal logical methods are used to design and validate all the control algorithms. Then the actual system is designed, with different subsystems being individually broken out and decomposed to the component (software and hardware) level so that a complete description is created, including every single state, all modes of operation, all possible conditions under which the aircraft could operate, etc. Then the various components are designed. During that design process a complete set of failure mode analyses are performed. For every single combination of components in the system it is determined what the individual effects of failure of each one in all possible modes of failure would be, then a fault tolerance matrix is constructed which allows the analysis of all possible combinations of failures and their effects.

      Then I come in. I construct a complete simulation of the actual aircraft electrical and mechanical systems and its flight control system. Now I can literally put each card, box, subsystem, etc into a virtual aircraft and test it to determine that it ACTUALLY performs as predicted under at least the vast majority of these conditions and failure modes. This is all IN PARALLEL with the formal SQA process for the flight software in which each module is tested with all possible inputs, formal static code analysis is performed, etc.

      We didn't HAVE errors. In all the millions of lines of code that was ever under my jurisdiction we never passed a single piece of code that had any error in it that could ever effect the safety of flight of any of the 7+ aircraft that I worked on. I'm not saying everything was always perfect. There were times when we found that flight software had issues, that there were system level issues that weren't discovered in design/test/review, but they were never things that went into a production aircraft and caused a problem that could have resulted in the aircraft being lost or even not flying that day.

      The upshot of all this is I know something about quality of software. Russia's aerospace industry has a very serious issue, this is only like the 4th lost mission in the last couple years that I can count without even trying. They have to be cutting some serious corners and its BAD.

      --
      "Malo periculosam, libertatem quam quietam servitutem." -- Jefferson
    6. Re:"Programmers" shouldn't write critical software by Giant+Electronic+Bra · · Score: 2

      Well, it sounded pretty clear that the actual flight CONTROL software worked fine and executed a program perfectly, it just wasn't the program that was intended due to SOME sort of issue with another piece of 'management' software. It sounded like that was also in the spacecraft, but its just as likely it was something running at ground control (makes more sense to me, generally you only run the least amount of software onboard that you can, why waste money on CPU and etc that could be cheaper ground stuff?).

      --
      "Malo periculosam, libertatem quam quietam servitutem." -- Jefferson
  8. Testing is not verification. by Brannon · · Score: 3, Insightful

    This is probably something that is well understood by the engineers who are building robot surgeons (and maybe even by those building driverless cars), but it certainly isn't well understood by the overwhelming majority of software engineers and it's just a matter of time until the unwashed hordes of C++ monkeys are unleashed unto critical systems.

    Bridges aren't designed and tested by "trial & error"--if they were then half of them would fall down within a few weeks. Neither are buildings or pacemakers or computer chips.

    There are some scary problems with how [many if not most] software engineers see the world which don't bode well for a world where software can kill:
    (a) by and large they've had essentially no exposure to any method of verification other than "trial & error"
    (b) they have insufficient reverence for cause and effect because most of their bugs have really low cost (as in, nobody dies)--therefore they aren't mentally trained to make disciplined decisions.
    (c) arrogance: unlike every other kind of engineer, software engineers rarely encounter the boundaries of their knowledge. A civil engineer knows when to call a materials engineer, a mechanical engineer knows when to talk to an industrial or chemical engineer, but a software engineer spends their entire lives inside a carefully constructed virtual world where they can't really do that much damage.

    1. Re:Testing is not verification. by ShanghaiBill · · Score: 4, Informative

      it's just a matter of time until the unwashed hordes of C++ monkeys are unleashed unto critical systems.

      No way. The corporate lawyers will never let that happen. Neither will the regulators. It is very hard to certify a SDC for public roads. Reams of test data are required. It is even more difficult to get a medical device approved by the FDA. Therac-25 happened almost 30 years ago, a lot of lessons were learned, and it hasn't happened again.

      Bridges aren't designed and tested by "trial & error" ... Neither are buildings or pacemakers or computer chips.

      I have never designed a bridge or pacemaker, but I have designed computer chips. I sit at a workstation, and I type Verilog code into Emacs. It is the same process as writing software, which is mostly trial and error. I write unit tests, do regression testing, etc. I watch it fail, I fix the bugs, and I iterate. Once I get all the bugs fixed, I load it into an FPGA, and watch it fail with some signal skew that I didn't think of. So I write more tests, and repeat. When it runs flawlessly on the FPGA, I ask a co-worker to test it some more, and review my code. Eventually we go to silicon, where a bug costs a million bucks. Usually everything is fine, but that isn't because it is "different" than doing software. It is basically the same process. It is more reliable because most ICs are far less complicated than even a typical iPhone app. They tend to have lots of the same cells repeat over and over. So an IC with a million gates isn't like a million lines of code. It is more like a few dozen 50 line subroutines, that are called a million times.

    2. Re:Testing is not verification. by Anonymous Coward · · Score: 2, Insightful

      The requirements for functional safety in programming industrial safety critical systems are well known, and are very different from the requirements for programming. Boiler flame safety systems are commonly microprocessor based now, and rarely if ever fail. Here are some links explaining some of the requirements.

      PLC® vs. Safety PLC – Fundamental and Significant Differences
      FM Global Class 7605 Approval Standard for Programmable Logic Controller Based burner Management Systems
      IEC 61508 Functional Safety

      This level of care is mandated by insurance companies and legislation, due to the history of boiler explosions early in the 20th century. Searching "Boiler Explosion", and "Functional Safety" will lead to many references on this subject.

  9. Re:Russian Programmer's are Brilliant! by R3d+M3rcury · · Score: 2

    Yeah, but Moscow girls made them sing and shout.

  10. How many people... by R3d+M3rcury · · Score: 3, Funny

    How many people are going to be killed by C++ in the next decade?

    4.

    I always find the "how many people will be killed" / "how many people have to die before" statements can be answered with this number.

  11. Pun Error on Line 472 by Tablizer · · Score: 3, Funny

    They had trouble putin it in the right orbit

    1. Re:Pun Error on Line 472 by Megane · · Score: 2

      But in Soviet Russia, glorious President would have launched satellite with his own bear hands!

      --
      #naabhaprzrag, #sverubfr-000, #agi-fcbafberq, negvpyr[pynff*=' negvpyr-ary-'] { qvfcynl: abar !vzcbegnag; }
    2. Re:Pun Error on Line 472 by ostrich2 · · Score: 2

      Russians have bear hands? I hope if we ever go to war, it's not hand-to-hand combat!