Slashdot Mirror
Software Error Caused Soyuz/Galileo Failure
schwit1 writes An investigation into the recent failed Soyuz launch of the EU's Galileo satellites has found that the Russian Fregat upper stage fired correctly, but its software was programmed for the wrong orbit. From the article: "The failure of the European Union’s Galileo satellites to reach their intended orbital position was likely caused by software errors in the Fregat-MT rocket’s upper-stage, Russian newspaper Izvestia reported Thursday. 'The nonstandard operation of the integrated management system was likely caused by an error in the embedded software. As a result, the upper stage received an incorrect flight assignment, and, operating in full accordance with the embedded software, it has delivered the units to the wrong destination,' an unnamed source from Russian space Agency Roscosmos was quoted as saying by the newspaper."
I've been hearing all this about the much vaunted chops of these Russian coders, but frankly I don't ever see it. They obviously haven't even heard of SQA. What gives?
"Malo periculosam, libertatem quam quietam servitutem." -- Jefferson
A software error in Russian GLONASS receivers has resulted in thousands of Russian troops innocently crossing the border into Ukraine.
"National Security is the chief cause of national insecurity." - Celine's First Law
From the linked article (emphasis mine): "Galileo Satellites Incident Likely Result of Software Errors", there is still an uncertainty. Though, I guess I should not be surprised, this is /. afterall....
It's not like it's rocket science to get it right
Table-ized A.I.
the strategic value of satellite navigation and general asshole-erly at the top of the Russian government, I am guessing that Europe's very expensive satellites ended up exactly where Russia wants them.
This is not a SW error! The software put them right where they were told to. The orbital parameters were wrong! This is a data error not a SW error!
There's almost no overlap between the skills & techniques necessary to write & verify critical software (e.g. when lives or huge amounts of money are on the line) vs. what is considered to be "programming". Modern software engineering's approach to reliable system design is about where hardware engineering was fifty years ago, and about where civil engineering was 100 years ago.
SQA is a joke. Reliable systems are made using way more robust techniques, including: (a) a severely restricted state space, (b) redundancy, (c) formal proofs, (d) fully (and formally) specified interfaces, (e) random simulation, (f) several different types of coverage, (g) physics-based analysis, etc.
The failure of the software community to understand this distinction is why I'm scared to death about the coming world of driver-less cars and robots performing surgery. How many people are going to be killed by C++ in the next decade?
This is probably something that is well understood by the engineers who are building robot surgeons (and maybe even by those building driverless cars), but it certainly isn't well understood by the overwhelming majority of software engineers and it's just a matter of time until the unwashed hordes of C++ monkeys are unleashed unto critical systems.
Bridges aren't designed and tested by "trial & error"--if they were then half of them would fall down within a few weeks. Neither are buildings or pacemakers or computer chips.
There are some scary problems with how [many if not most] software engineers see the world which don't bode well for a world where software can kill:
(a) by and large they've had essentially no exposure to any method of verification other than "trial & error"
(b) they have insufficient reverence for cause and effect because most of their bugs have really low cost (as in, nobody dies)--therefore they aren't mentally trained to make disciplined decisions.
(c) arrogance: unlike every other kind of engineer, software engineers rarely encounter the boundaries of their knowledge. A civil engineer knows when to call a materials engineer, a mechanical engineer knows when to talk to an industrial or chemical engineer, but a software engineer spends their entire lives inside a carefully constructed virtual world where they can't really do that much damage.
How many people are going to be killed by C++ in the next decade?
4.
I always find the "how many people will be killed" / "how many people have to die before" statements can be answered with this number.
What's a "Programmer"? Also precisely who should we get to write critical software? A maths teacher? The after hours cleaner? Maybe some random MBA from middle management? Programmers most definitely SHOULD be the ones writing critical software. It's when it is written by non-programmers or hobby programmers with full time other careers (physicists, engineers, etc) that you end up with some of the most basic mistakes and unexpected behaviour.
Your big mistake is to assume that all programmers are the same, and that all hardware designers are the same, and that all civil engineers are the same. A civil engineer who's speciality is designing sewers and town water systems is unlikely to be the one you want designing a skyscraper. Just like in my world I have a VERY experienced instrument engineer sitting next to me, but we wouldn't ever let him work on safety shutdown systems.
QA for software is exactly as much of a joke as people make it. At a small software house, it may be almost non-existent. At a company designing safety shutdown systems it is a whole world of hurt. Unfortunately it's management which are the biggest risks. There's never enough time to do it right, but there's always enough time to do it again.
You mean using the EU satellites for GLONASS? Unlikely. However, they have intentionally crippled Galileo.
Excuse me, but please get off my Pennisetum Clandestinum, eh!
They had trouble putin it in the right orbit
Table-ized A.I.
The software worked perfectly. This is a case of misprogrammed destination ("What do you mean this is Auckland? I wanted OAKLAND!")
Probably.
Your fears are not rational. Self driving cars and robotic surgeons are tested for thousands of hours, under live conditions.
Among the other parts of my job I run a Quality Assurance department for my company and I've worked in QA for several years. It doesn't matter how much you test something if the process for designing and building the product was inadequate. QA testing is like the goalie on a hockey team - necessary but even the best goalie is going to fail if the team in front of him can't play defense. Good quality comes from good designs which are rationally and systematically well executed. Testing is a part of the equation but the correlation between hours of testing and the ultimate quality of a product is a weak one.
I had LASIK eye surgery done by a robot. I trusted it far more than I would a human surgeon.
And you looked up the hard evidence to back up this assumption for that specific procedure? (you may have - not trying to be rude) I've had LASIK as well and while I agree that it is absolutely possible for a robot to help a surgeon do a better job, I wouldn't trust it more simply because it was a robot. Furthermore there is a difference between an autonomous robot and a robotic assistive device. Most "robotic surgery" is with devices that assist and (hopefully) improve the capability of the surgeon doing the work but it is still a surgeon operating on you at the end of the day. He's just using a fancy tool to help him be a bit steadier.
Getting rocket software right is difficult precisely because there is no way to do a live test.
Umm, yes there is. It's called "doing a live test". They're often expensive but they very often are possible. We did lots of them in the early days of the space program. Companies still do them to this day. They might choose not to for economic reasons but that doesn't mean they cannot be done.
Most programmers and software engineers have the limitations you mention because consumers don't want to pay for the high quality software we want to build for them.
I would lend more credence to this statement if most software engineers actually had any actual experience designing and implementing high reliability software. Most unfortunately have very little clue what that actually means or how to do it for real. They like the idea (it's a good idea!) but have zero experience or training in the implementation techniques required. You are correct that there is an economic component to the problem but that doesn't appear to be the core problem. Even when we take money out of the equation altogether with open source software, we STILL don't see software developers using the formal engineering techniques that would result in the most reliable software. I think this is in large part because most of them have no idea whatsoever how actually develop like this. Most software is designed by trial and error because that is the only way most developers know how to do it. They still code like they did when they were a teenager in mom's basement because no one showed them a better way.
For what it's worth, this problem isn't unique to software engineers. I'm not a programmer and I see similar problems with electrical and mechanical engineers on a daily basis. Trial and error is easy to understand and quick and generally works whereas formal engineering is much harder.
When software is used in places where it has to work the first time, we'll be more than happy to adapt to the new set of circumstances.
Maybe but I doubt it. I really don't see developers genuinely pushing for more reliable development techniques in the real world. They talk about them in a "wouldn't that be nice" sort of way but they don't really try to make them happen.
unlike every other kind of engineer, software engineers rarely encounter the boundaries of their knowledge
I'm not so sure about this. I agree about the arrogance but I think a more accurate statement might be that software engineers too often fail to recognize when they encounter the boundaries of their knowledge. I think they bump into those limits all the time and go merrily on their way past them. We've all seen software that was clearly developed by someone who clearly never actually had to use it to do the job it was designed for. I was staring at a piece of accounting software today that clearly was designed by someone who has never actually worked as an accountant. Either work flow was not a primary consideration or the programmer badly misunderstood how accountants go about their daily business. As you say, the consequences of their decisions are so far removed from their incentives and feedback that they have no real appreciation of how their work affects others.
cracking US banks.
so their day jobs, launching rockets and calculating how far Russia could go in subjugating Ukraine, suffered.
we all know how that works...
if this is supposed to be a new economy, how come they still want my old fashioned money?
What's the old saying? There is no izvestia in Pravda, and no pravda in Izvestia.
"Win treats sysadmins better than users. Mac treats users better than sysadmins. Linux treats everyone like sysadmins."
Revenge is a dish best served in the wrong orbit?
Funny, isn't it, in the midst of all these sanctions and general brou-ha-ha over the Ukraine, with Russia taking all kinds of tit-for-tat punitive measure in response by EU attempts use economic fines in order to restrain their bad behavior, that, âoeThe nonstandard operation of the integrated management system was likely caused by an error in the embedded software," which manages to cost the EU the full use of a multi-million dollar satellite whose purpose was to provide competition with Russia's GLONASS system (in addition to American GPS)?
They didn't even have to do anything fancy, just twiddle a few lines of code to send it off course, then blame it on random "unforeseeable" coding error that they'll refuse to accept responsibility for.
"Inveniemus Viam Aut Faciemus" 'We will find a way... Or we will make one!' --Hannibal of Carthage