Slashdot Mirror
Software Bug Caused Qantas Airbus A330 To Nose-Dive
pdcull writes "According to Stuff.co.nz, the Australian Transport Safety Board found that a software bug was responsible for a Qantas Airbus A330 nose-diving twice while at cruising altitude, injuring 12 people seriously and causing 39 to be taken to the hospital. The event, which happened three years ago, was found to be caused by an airspeed sensor malfunction, linked to a bug in an algorithm which 'translated the sensors' data into actions, where the flight control computer could put the plane into a nosedive using bad data from just one sensor.' A software update was installed in November 2009, and the ATSB concluded that 'as a result of this redesign, passengers, crew and operators can be confident that the same type of accident will not reoccur.' I can't help wondering just how a piece of code, which presumably didn't test its input data for validity before acting on it, could become part of a modern jet's onboard software suite?"
The worst part is that Google wants to build a driverless car. Flight pilots have been trained to react to emergencies in a calm manner and they have time to do so while in air. Neither is true for cars. People will panic when something goes wrong, and there won't be any time to react to them. Your life (and others life) will be completely dependent on the AI, and lets face it, there will be bugs.. Google isn't exactly known for bug free products. Hell, even NASA has bugs and they use billions so that there wouldn't be any. I just think it's a really bad idea and Google is being irresponsible and malicious with such project. Of course they will also hide some "we are not responsible for accidents in any way" under some clause. Let me just say that somewhere in the future we will be hearing how Google killed some innocent people and children.
same way presumable don't proof read.
Presumably, in the same way a story with the phrase "software suit" got posted to the front page of Slashdot without being about some sort of matrix-like cyberworld.
This, from the same company, while building the A380 megajet decided to upgrade half of their facilities to plant software version 5, while the other half decided to stick with version 3/4. And did not make the file formats compatible between the two versions, resulting in multi-month delays of production as a result.
Point being, in huge projects, simple things get overlooked (with catastrophic results). My favorite is when we slammed a $20 million NASA/ESA probe in to the surface of mars at high speed because some engineer forgot to convert mph in to kph (or vice-versa).
moox. for a new generation.
It is clearly the fault of the programmer. They should be held liable for incidents like this. Management tries their best, but ultimately it always comes down to the coders. The company can be protected by the ToS, but not the lazy programmers.
"I can't help wondering just how could a piece of code, which presumable didn't test its' input data for validity before acting on it, become part of a modern jet's onboard software suit?""
How about reading the darned final report, conveniently linked in your own blurb? There was lots of validity checking. In fact, some of it was relatively recently changed, and that accidentally introduced this failure mode (the 1.2-second data spike holdover). (Also, how about someone spell-checking submissions?)
we're going to see a huge change in programming methods coming pretty soon. Today, A.I. is still math and computer based. The problem is that data, input, and all of the algorithms you're going to write can result in a plane nose-diving -- even though no human being has ever chosen to nose-dive under any scenario in a commercial flight.
Why was an algorithm written that could do something that no one has ever wanted to do?
The shift is going to be when psychology takes over A.I. from the math geeks. It'll be the first time that math becomes entirely useless because the scenarios will be 90% exceptions. It'll also be the first time that psychology becomes truly beneficial -- and it'll be the direct result of centuries of black-box science.
That's when the programming changes to "should we take a nose-dive? has anyone ever solved anything with a nose-dive? are we a fighter jet in a dog fight like they were?" Instead of what is it now: "what are the odds that we should be in a nose-dive? well, nothing else seems better."
The cheap IT Guys Qantas may have hired are irrelevant. the Fight control software is all Airbus.
Qantas doesn't build the aircraft or the avionics. This is an airbus issue, not a qantas issue.
Wait, you mean cell phones aren't being blamed?
I can't help wondering just how could a piece of code, which presumable didn't test its' input data for validity before acting on it, become part of a modern jet's onboard software suit?"
---
I'm surprised there are people who think that we have the technology to program computers to make decisions about how to control things like airplanes better then a human being.
Computers excel at solving mathematical problems with definitive inputs and outputs, but our attempts to translate the problem of controlling an airplane, or an organism, into a simple circuit...will necessarily be limiting.
They can only test that the computer program will behave as expected, but there is no test to prove that the behavior we attempted to implement is actually a "good" way to behave under all circumstances.
I prefer my suits to be made of gabardine, or maybe some modern synthetic. They're easier to care for than a suit made of an airplane.
What are you? some kind of person that doesn't read the actual articles or documents? Oh wait.. this is slashdot. Here let me copy paste some text for you
So there you go, there actually really was validity checking performed. Multiple times per second in fact, by three separate, redundant systems. Unfortunately all 3 systems had the bug. Here is the concise summary for you:
I don't need to test my programs.. I have an error correcting modem.
Airbus poached engineers from Toyota!
If Qantas did all that outsourcing etc., than Qantas was safe .. purely due to the very limited flights it has, and Australia's foreign policy.. where it only bullies immigrants in boats.
Wow, you've got a really fucked up idea about aircraft if you think the operator (QANTAS) is in any way linked to flight control software, LOL.
But I'll throw you a bone so you can continue trolling Aus media style; this airspeed sensor fault has occurred three times across the worldwide A330 fleet, all on QANTAS aircraft.
in the millions of lines of code in these modern flying death traps.
I really do not think Qantas is responsible for the software on the Airbus 330. Please stop blaming every Qantas incident on outsourcing.
the idea that a bunch of automatically piloted vehicles is somehow a better solution to city transport than mass-transit, it boggles my mind.
real people do not have money to maintain their cars properly. things are going to break. there are not going to be 'system administrators' to fix all the glitches that come up when cars start breaking down after a few years.
there will be problems. do i know which problems? no, but i know the main problem.
arrogance amongst revolutionaries. it is historically a pattern of the human species. declaring that nothing could go wrong is usually a precursor to a lot of things going wrong. not because the situation was unpredictable, but because human beings in an arrogant mindset tend to make a lot of mistakes, be reckless, and try to cover their asses when things go wrong.
but successful engineering is the anti-thesis of arrogance. nobody worth his salt is going to say 'what could go wrong'? they are going to have a list of 500 things that could go wrong, and all the ways they have tried to counter-act those wrong things happening.
After all, buying planes that someone else made is outsourcing. However I am not sure they'd fair better building their own.
Boffoonery - downloadable Comedy Benefit for Bletchley Park
just like those runaway Toyotas. We all know that no bug of that severity could make it into the finished product.
uhm, a few days in a row last year.
if you were being logical, you would say "i trust trains and subways more than the automobile-highway system. we should get rid of car subsidies and start building trains and bicycle paths everywhere"
we already know that big passenger cabins, when situated on rail systems, do not veer off into playgrounds or farmers markets and kill people.
we already made the decision to abandon those systems in favor of the deadly automobile, which kills 30,000 people a year.
now, you want to convince me that Google's "driver less car" is so wonderful because, "think of the children". I did think of the children. big industry, big oil, big auto, and corrupt governments decided to say "fuck the children", abandoned public transit, and went with the mass-car culture we have, on purpose, deliberately, to make money.
so you want me to trust google, another huge, faceless corporation, whose only duty is to its shareholders, to make them a profit. and you expect me to believe that they are doing this 'for the children'? if we cared about the children, we have solutions already, and we simply chose not to spend money on them, because it wasnt profitable enough for hedge funds and investment bankers.
i will believe google cares about 'the children' when it does something about e-waste farms in china, child laborers in the mines in africa, etc etc etc.
not when it makes 'driverless cars' to appease some people who spent too much time watching "beyond 2000" when they were kids.
Why was an algorithm written that could do something that no one has ever wanted to do?
Two or three times no less ... at least that's what I've been told repeatedly, that three independent airplane computer systems are written from spec by different teams, and then given the input they all produce output, and the best 2-out-of-3 results win and cause physical action to happen.
So either at least two teams messed this specific item up the same crazy way, or that airline computer safety story they've been telling is a crock.
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
This sounds very much like the failure of the Pitot tubes (used to measure airspeed) on the A330 that chrashed in the Atlantic on 1 june 2009. Does anyone know if that might be the case?
is this article about driverless cars or software for an airplane?
Never say never. Ah!! I did it again!
other Airbuss had issues with auto pilot over riding the pilots and having no way to force it off. Now a sensor malfunction can make the auto pilot do stuff that a real pilot will never do and does the software have any kind of work around for broken sensors or way to find out that a sensor is out of range and to stop reading it?
So are good software engineers still cost centers rather than assets, where hiring the cheapest programmers in 3rd world countries for something like this makes sense?
http://saveie6.com/
Shame so little is displayed in the Australian media and the fear-of-the-other crowd.
Even a rudimentary comprehension of the report shows the event has nothing to with Qantas in particular. The problem lies in the Northrup Grumman ADIRU equipment fitted by Airbus and the Airbus software response to unusual outputs from that equipment. This is backed up by the prompt issuing of interim procedures and software fixes by the aircraft manufacturer (two years ago). If anything, the decision by Qantas pilots to fly the aircraft above all else, and put it down in a remote location rather than continue to Perth, is what made sure that the injured did not become mortalities.
Things fail on aircraft all the time. Aircraft are hostile environments to electronics and other systems. This is not unique to Qantas or any other operator regardless of the part of the world their maintenance is done in. The unique thing with Qantas is the incessant media hype around every little thing that goes wrong.
Patent litigation: A doctrine of Mutually Assured Destruction... in which everyone seems willing to push the button
Being responsible in the criminal ways and the auto insurance has there own legal teams that can fight that as well.
Now what if someone get's killed or hurt out side of the auto car I don't think "we are not responsible for accidents in any way" will work with some in there own car or on the street why sue the owner of the car (who may not be the driver) any ways when you can go after the deep pockets of Google.
And someone get't killed may be a accidental death investigation and the people who made the software can end up facing the court under criminal negligence.
Have 3 versions of the software vote so then the 1 bug can not show up in all 3 versions! (kidding)
Democracy Now! - uncensored, anti-establishment news
Although the FCPC algorithm for processing AOA data was generally very effective, it could not manage a scenario where there were multiple spikes in AOA from one ADIRU that were 1.2 seconds apart. The occurrence was the only known example where this design limitation led to a pitch-down command in over 28 million flight hours on A330/A340 aircraft, and the aircraft manufacturer subsequently redesigned the AOA algorithm to prevent the same type of accident from occurring again.
ADIRU = Air Data Inertial Reference Unit
Incidents like this are why I've always considered 'software proof' to be worthless drivel. I do data acquisition and control/command software, but if your sensors give you garbage, how can you expect the software to behave properly ? Garbage in, garbage out.
Non-Linux Penguins ?
I would not blame Quantas. They didn't build the plane.
Airbus, with its fly-entirely-by-wire system is known for such failures. Boeing on the other hand had MECHANICAL backup systems built into all their planes, so even if the computer totally crapped out, there was still a workable (if difficult) manual system the pilots could use to fly the plane. (The very newest Boeing jet, just now starting to roll off the production line, is their first entirely fly-by-wire plane.)
Time after time after time, Airbus planes crashed because of "computer" (maybe software, maybe not) errors, and the pilots could do nothing but sit there and watch.
I am not saying that it was that particular situation that caused their worse safety record, but I do believe it contributed significantly. Even that famous "missing" flight off of South America that crashed a few years ago now appears to have done so due to a computer error that the pilots could not correct.
Is fly-by-wire a bad idea? I don't necessarily think so. But it certainly WAS a bad idea, given the state of the art.
But the best part is that once you fix a bug in an automated system, it's fixed forever, whereas a fresh new crop of novices hits the roads/skies every day.
Is the bug fix forever --- or simply until the next version of the hardware and software?
The problem I have with the driverless car is "Fire and Forget."
The temptation for the layman will be to let the thing loose without really understanding the limits of the technology.
In wind, and snow and ice, for example, a pedestrian or cyclist will find it difficult to keep his balance, he may take an unexpected fall, and since the sidewalks and bike paths haven't been cleared,he may be forced much farther out into road than he is comfortable with, and so bundled up he can't see or hear very well.
An experienced driver will see the possibilities.
The question is whether the robotic vehicle's sensors and AI will be anywhere near as alert and responsive.
Excellent piece of information, I had come to know about your website from my friend kishore, pune,i have read atleast 8 posts of yours by now, and let me tell you, your site gives the best and the most interesting information. This is just the kind of information that i had been looking for, i'm already your rss reader now and i would regularly watch out for the new posts, once again hats off to you! Thanx a lot once again,
Regards,
people01
DISCLAIMER: I hate air travel, but do it most weeks.
I have worked in and around the safety critical software industry for over 20 years. The level of testing and certification that the flight control software for a commercial aircraft is subjected to far exceeds any other industry I'm familiar with. (I'm willing to be educated on nuclear power control software however.)
The actual problem on the Qantas jet was a latent defect that was exposed by a software upgrade to another system. So the bug was there for a long time and I'm sure there are still others waiting to be found. But this doesn't stop me getting on a jet at least twice a week.
As a software professional and nervous flyer, do problems with the aircraft software scare me? No not really. What scares me is the airline outsourcing maintenance to the lowest bidder in China, the pilots not getting enough break time, the idiotic military pilot who ignores airspace protocol, and the lack of english language skills in air traffic controllers and cockpit crew across the region where I fly (English is the international standard for Air Traffic Control).
A good friend is a senior training captain on A330's, and in all the stories he tells software is barely mentioned. What get's priority in the war-stories is the human factors and general equipment issues - dead nav aids, dodgy radios, stupid military pilots. One software story was an Airbus A320 losing 2 1/2 out of 3 screens immediately after takeoff from the old Hong Kong airport. The instructions on how to clear the alarm condition and perform a reset were on the "dead" bottom half of one of the screens.
A great example of software doing it's job is the TCAS system - Traffic Collision Avoidance System (http://en.wikipedia.org/wiki/Traffic_collision_avoidance_system). To quote my friend "If it had lips, he'd kiss it". It's saved his life, and the lives of 100's of passengers, at least twice. Both times through basic human error on the part of the pilot of the other aircraft.
One final thought - on average about 1000 people die in commercial aviation incidents each year world wide (source: aviation-safety.net) . In the USA, over 30,000 people die in vehicle accidents every year.
Wasn't there a Michael Crichton novel about this?
There are two philosophies at work here.
On Boeing planes, the pilot can always override the computer.
On Airbus planes, the pilot cannot override the computer, no matter how wrong the computer can be.
It's been this way ever since Airbus and Boeing computerized their flight controls.
There are advantages and disadvantages to each: The Airbus depends less on pilot skill. But when the shit hits the turbofan, you're proper fucked. The Boeing requires skilled pilots, and inexperienced pilots have been known to rip off tail rudders by stomping their feet all over the pedals.
Pick your poison.
--
BMO
The developer should be shot.
Basing A.I. on psychology will be only a stop gap measure, on the way to the true solution to this sort of problems: basing A.I. on evolutionary anthropology. You see, both the crew and the passengers can be modeled as as tribe, trying to adapt their stable trajectory based culture to changing conditions, namely a nose dive. As more and more air tribes experience such disruptions to their familiar environment, you will find that some develop better coping strategies than others. After a number of generations, all air passengers will be descendants of the air tribes with the more successful coping strategies, and you will find that nose dive causing bugs no longer matter. The people on board will have learned to deal with it. They will probably even have developed tools, either to survive the crash, or to patch firmware on board.
I must have, I must have put a decimal point in the wrong place or something. Shit. I always do that. I always mess up some mundane detail.
Posting anon because I moderated.
I had a very similar problem once with firmware on a TI DSP. The symptom was that a peltier element for controling laser temperature would sometimes freak out and start burning so hot that the solder melted. After some debugging, it turned out that somewhere between the EEPROM holding the setpoint, and the AD converter, the setpoint value got corrupted.
The cause turned out to be a 32 variable that was un-initialized, but always set to 0 by the stack initialization code.
Only the first 16 bits were filled in because that was the value stored in the EEPROM. The programming bug was that the other 16 bits were left as is. In >99% of the time, this was not a problem. But if a specific interrupt happened at exactly the wrong moment during initialization of the stack variable, that variable was filled with garbage from an interrupt register value. Since the calculations for the setpoint used the entire 32 bits (it was integer math) it came out with a ridiculously high setpoint.
Having had to debug that, I know how hard it can be if your bug depends on what is going on inside the CPU or related to interrupts.
There may only be a window of less a micro second for this bug to happen, so reproduction could be nigh on impossible.
I've always been amused by people who want computers to have complex mental lives.
If you really mean what you say, to have computer psychology, we won't have an auto-pilot system tested and deployed to all planes, we'll have to teach each one individually, in an auto-pilot school, and then try to weed out the psychologically unstable ones that are thrill-seekers, or that adopt strange ideologies and decide to become martyrs for an inscrutable cause only understood by other unstable auto-pilots (and the occasional sympathetic toaster oven).
We'll have therapists asking the computer, "tell me about your compiler." Try too hard to model machine vision after humans, and we'll start to have machines seeing faces in clouds, demons in dark shadows, and Alan Turing in the pattern on a slice of toast.
Yes. Human pilots can fuck up as easily as anyone else.
But in an emergency, I'd rather have a human pilot making the decisions and being in control.
On Airbus vehicles, if the avionics computers crash, the airplane crashes. There's exactly ZERO way to pilot the computer manually in such a failure.
Moreover, the avionics system can and does overrule pilot input. So if you get sensor malfunctions like this, even if the pilot is trying desperately to save the plane, the computer can still crash you.
On Boeing, if the avionics computer fails, the pilot at least has a chance of saving the aircraft.
You can come up with all the sleepy, crazy, stupid, drug-addled, locked in a bathroom with a stewardess horror scenarios you want.
What would you rather have in a failure scenario? A slim chance or no chance?
Chas - The one, the only.
THANK GOD!!!
The software did not cut the accelerator when the brake was depressed if a rug was stuck on the accelerator. The problem was caused by floor mats but it could have been prevented by software.
It looked like half of one 32-bit word was combined with half of another 32-bit word during queue assembly on at least some occasions. But there are errors not explained by that.
This is why I like fuzzing. Sending random and/or corrupted data to software to evaluate the software's robustness and sensitivity to corrupted inputs. For a project like this I would like to send simulated inputs from regression tests and recorded data from actual flights to the software while fuzzing each playback, repeat. Let a system sit in the corner running such tests 24/7.
In theory some permutation of the data should eventually resemble what you describe.
Does no one else see the typo? That's what leads to these issues. "Updated" vs. "Update" should be noticed. Creativity should be important enough to be checked by the creator.
My father grew up in a fairly major AU city where horses were still a primary means of commercial transport at the time - although on the way out. He told some interesting stories about the practicalities of such an arrangement. Excrement, and dramas of dealing with it being the obvious one. But how about the complications of a horse dieing in the milk delivery yards - somewhat more of a drama than a truck breaking in the warehouse. I remember tales about feed, care and stabling which'd startle you - the time it takes to do enough of such things would be uneconomic in today's world.
Rose coloured views of history are fine when you didn't live thru it.
Yes, I'm older than the lot of you put together. Now get off my paper tape.
The lecturer's story seems consistent with recollections of being on dive boats (scuba) as dive computers were entering the market. The subgroup using mechanical/analog devices for depth, pressure and time and carrying a plastic card with no-decompression time limits was where nearly all the computer programmers and electrical engineers could be found.
I think nuclear power is safe. It will reduce the flow of cash to the middle east and solve all our energy problems.
I'm not going if it is not a Boeing.
"I can't help wondering just how a piece of code, which presumably didn't test its input data for validity before acting on it, could become part of a modern jet's onboard software suite?"
While I agree that it was a poor design, you should not make the assumption that the data from the sensor was invalid. 0 knots is valid input. (I'm presuming that the software is designed such that low airspeed causes it to sacrifice altitude to gain speed, and perhaps an extremely low airspeed, one that is possible but nonsensical, would cause an extreme reaction such as a nosedive.) Maybe it failed and froze with a SLIGHTLY low value - say 300 knots instead of 310 knots - the PID controller would continue to tip the nose down until it sees an increase of speed approaching the desired speed. If the value is frozen, the nose keeps going down until you get a nosedive.
I don't know what they implemented in software to mitigate this kind of failure, but as someone who writes software for industrial controls, I certainly hope they have some kind of supervisory software that watches the rate of change of their sensors and turns off any kind of automatic control (returning to manual) if a maximum rate of change is exceeded OR if a minimum rate of change is not met. Sensor failures are a bitch because you never know how it's going to fail... is it going to fail high? low? slowly? quickly? If you have two sensors reading the same data, how do you know which to trust? We will not in my lifetime have fully unmanned passenger travel (road, air, etc) until there's a revolutionary step in industrial sensors to combat these issues... one which at this point is not on the horizon, regardless what the manufacturers claim.
injuring 12 people seriously and causing 39 to be taken to the hospital.
That is why you keep your safty belt shut.
If you don't like the feeling, losen it a bit, but keep it closed.
I really wonder why people keep taking such nonsense risks and open the seat belt directly after launch.
Cost free eBook I read (by iBook/Kobo/Amazon/ObookO/Gutenberg etc.): "The Green Odyssey" by Philip Jose Farmer.
I agree that humans can be superior solution to automated cars right now, given two conditions: * Humans are well trained. * Humans are paying attention and actually trying to do the work properly. Given that often enough one of those conditions won't be satisfied, automated cars will be feasible and will be better than humans on average. Humans can be great with challenging tasks requiring creativity, or solving unexpected problems. Machines are great at doing boring repetitive stuff. Maybe driving isn't boring (depends on how long is your commute), but it is certainly repetitive. Hence it can be automated. Or are you going to argue that humans beat machines doing repetitive stuff, let's say working at assembly line? I consider myself a keen driver. I do enjoy driving- for hours sometimes. However, now I moved to a big city that has underground/buses/trains and I commute using public transport. This allows me to do other stuff like reading on my way to work. I can let my mind wander and not be concentrating on the road. And I'd rather spend my commuting time doing other things than driving. --Coder
Reading about the A330 Air France Flight 447 disaster [*], I was surprised to find that the software also took the two steering sticks and AVERAGED THEIR INPUTS. On old aircraft carriers, the sticks/wheels used to be mechanically linked together, so each pilot would feel each other's steering.
To me, this sounds like an amazingly stupid thing to do. In fact, it sounds like it actually had a role to play in the Air France 447 disaster.
WHO wants to AVERAGE two control inputs? It sounds like amazingly stupid software engineering, if you ask me, done by guys who should know better.
[*] http://www.popularmechanics.com/technology/aviation/crashes/what-really-happened-aboard-air-france-447-6611877
What a load of uninformed bullshit - Airbus has several levels of computer control, called laws, one of which is Direct Law which passes all inputs directly to the control surfaces. And if that isn't enough, they have mechanical backup controls for all surfaces on the flight deck, so even with a completely dead computer the aircraft is still flyable.
You sir, are talking complete shit, but that seems to be normal when someone wants to put Boeing on a pedestal over Airbus.
Let's go over some of your "mistakes"...
The 787 isn't Boeings first FBW aircraft, they have had one flying since the mid 1990s with the 777. The 787s system is an evolution of the 777s.
AF447 didn't crash because of a computer problem, it crashed because of poor crew relationships in the cockpit - three pilots in that cockpit and not one was interested in what the others were doing. They didn't run basic check lists, they ignored other information, and the pilot flying did completely the wrong thing - the situation was completely survivable if they had carried out the correct procedures, except they didn't. The crash wasn't caused by the computer, it was caused by the pilot taking a stable aircraft and stalling it badly when nothing about the computer error forced him to do that.
Why was this modded "Troll"? For crying out loud, the Troll option is not equivalent to "Disagree". Even if you don't agree with him, that's no reason to call him a troll. If I had mod points, I would mod him back up immediately. There's absolutely nothing trollish about his post.
When I realized after 10 minutes of reading and tried to reply to a post, that I'm reading the 3 year old story linked in the summary and not the current one...
this is, put simply, the difference in quality between airbus and boeing. nothing more.
Wealth is the gift that keeps on giving.
I wonder if there is any connection between this software and the problems faced by copilots on Air France flight 447 which crashed killing everbody on board.
That too suffered pitot problems, but the report concluded the copilots handled it all wrong by not reacting correctly to the stall problems that followed.
I really wonder why people keep taking such nonsense risks and open the seat belt directly after lunch.
FTFY
Have gnu, will travel.
Nothing to fix. Lunch in a plane is not that big that you should need that. ... which is simply stupid.
And I indeed ment: launch.
As soon as the "fasten seat belts" sign gets off, 90% of the passengers open them
Cost free eBook I read (by iBook/Kobo/Amazon/ObookO/Gutenberg etc.): "The Green Odyssey" by Philip Jose Farmer.
Pilots are becoming nothing more than subway drivers in these planes. It's one thing to have a computerized navigation system, but when you have computer software controlling every facet of airplane operation, you are just asking for trouble. You have to have a backup where you can hit a switch and control the stick and throttle manually, without worrying that the thrust vectors are going to push your A330 with the thrust for a Cessna.
"how can you expect the software to behave properly"
Simple - it goes into failsafe mode or shuts down. It doesn't just accept whatever values it sees on the input as always valid. I do hope you only work on mickey mouse systems and arn't involved in aircraft or - god forbid - nuclear power plant control systems! With your approach to software there's no way I'd ever hire you for the role you say you're now in.
Time after time after time, Airbus planes crashed because of "computer" (maybe software, maybe not) errors, and the pilots could do nothing but sit there and watch.
What a load of pish!
Exactly, I'm always baffled when I'm flying as a passenger and as soon as the Fasten Seatbelts sign goes off, I hear "click click click click" everywhere around me from people who have no intention of getting up. What the hell is so bad about having a seatbelt fastened on your lap while you're seated that you absolutely have to get rid of it as soon as you possibly can? While sitting in a tube suspended by wings in an airflow that might become turbulent unexpectedly at any time?
"After this latest software update, operators can be confident that the same type of security fault will not reoccur."
What cuts down trees at 200 knots? http://www.youtube.com/watch?v=2eQpUgHkBcg
Conservative, mod down for violating
The only thing I say is, why did it take Airbus 2 years to find and fix that major bug?
Probably, the lack of a plane-wide (I mean used by every piece of software running ona given plane) standarized practice for distinguishing raw input/doctored data/commands, that should have been forced onto every signle sub- and sub-sub- contractors writing every firmware.
Think of it as something similar to not correctly processed tainted input leading to cross-site-scripting.
Except that, instead of concerning 1 service running on 1 company website, it concerns ~1000s of small electonic equipment each running its own firmware all done by a large collection of subcontractors.
And except that instead of crashing the database and exposing credit card number, this could have crashed the plane, had the pilot not taken back control.
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
I can't help wondering just how a piece of code, which presumably didn't test its input data for validity before acting on it, could become part of a modern jet's onboard software suite
Here's how: high reliability software engineering may be slowly turning into a cargo cult, where certain artifacts become part of the ritual without much understanding what they are for. There's a certain process that has to be followed to get the software past certification bodies, and there are expensive (think $150k per seat and up) and inexpensive (bound notebook with numbered pages + a pencil) tools that can be used to help you follow the process in various ways (by offering mathematical proofs that various pre/post conditions are fulfilled, by tracking specifications, requirements and test coverage, etc). Those artifacts are not a substitute for thinking and understanding how what you're coding fits into the system, and that the requirements and specs may need to be amended. Either the spec/reqs were bad, or the implementation didn't quite follow them, and somehow it slipped under the radar. If it were on the space shuttle software team, they'd dig deep to understand how their process had failed, and would fix the process first. Whoever maintains that software probably simply issued a bugfix, filed requisite paperwork somewhere, and promptly forgot the whole thing.
A successful API design takes a mixture of software design and pedagogy.
My best bet at this point is that it was a known issue with airspeed sensors, and the pilots were not properly trained how to deal with it. That particular plane can be flown quite well without any airspeed data -- you pretty much set the throttles by looking it up in the table, based on air pressure and desired airspeed. What most likely killed them was that the pilots were unaware that the airspeed data was wrong, so they were flying the plane on wrong (most likely low) airspeed indication. There was nothing particularly computer-related in the whole incident so far. Good old wetware issue.
A successful API design takes a mixture of software design and pedagogy.
Well said. And I own no aerospace stocks and have nothing to gain by siding with you.
A successful API design takes a mixture of software design and pedagogy.
Insert lowest bidder comment here
I am Bennett Haselton! I am Bennett Haselton!
If you saw the procedures required to get airworthiness certification from the FAA for a critical piece of software, you would shake your head in disbelief. It is almost all about ensuring that every line of code is traceable to (and tested against) a formal requirement somewhere. In spite of greatly increasing software development costs (due to the additional documentation and audit trails required), the procedures do amazingly little to ensure that the requirements or code are actually any good, or that sound software engineering principles are employed. It does not surprise me that GIGO situations occasionally arise -- it is perfectly plausible that a system could meet the certification criteria but shit's still busted because the formal requirements didn't completely capture what needed to happen.
The cost of compliance can also warp the process. A co-worker once told me a story about an incident that happened years ago at a former employer of his. A software system with several significant bugs was allowed to continue flying because the broken version had already received its FAA airworthiness certification. A new version which corrected the bugs had been developed, but getting the new version through the airworthiness certification process again would've been too costly; so the broken version was allowed to continue flying.
Look up "DO-178B" sometime if you're curious...
Did this story make anybody else think about Russinovich's novel "Zero Day"?
Luggage falling from over head bins. Drink cars rolling toward the front of the plane at 40 MPH. Hit coffee splashing in your eyes. Stewardess landing in your lap.
OK, I could go for that last one.
While your point is valid, not all injuries are from the pax coming out of their seats.
I am very small, utmostly microscopic.
Maybe if you actually read what I wrote, you could come up with some real objections, rather than your own uninformed bullshit.
The 777 has a mechanical backup system, exactly as I stated. Look it up, bozo. All Boeings to date have.
I did not state that Boeings were not "fly by wire", I said they had MECHANICAL BACKUPS. I meant as opposed to entirely (or nearly entirely) fly-by-wire only. It's right there in my comment, in plain English. Further, despite what I stated earlier, even the 787 has electrical/mechanical manual backup in case of computer failure. Of course, if you suffer a complete loss of electrical power (including even the turbine generator), you are SOL either way.
And if you dispute that Airbus has been prone to crash due to computer failure, I invite you to look that up, too.
If you look up the Airbus laws yourself, you will see that the only manual control for pitch, for example, is the trim wheel. This is going to do you very little good unless your plane is already in stable horizontal flight... seldom the case when catastrophic failure occurs.
But the main thing -- one I didn't see the point of going into until you decided to butt in -- is that in order to go into "Direct Law" in an Airbus, which is what you might consider your main "manual backup", you have to jump through a lot of hoops, which (as it turns out) pilots just do not have time to do in an emergency.
It still amounts to the pilots not being able to control their aircraft when the computer fails.
I worked in the avionics field and had to perform testing under RTCA DO178A guidelines, which stipulate 7 Sigma or about 14 billion years between failures. For every dollar spent on avionics software, 95 cents is for testing.
The computers have too much control over flight operations. The flight crew doesn't have enough. When something like this happens, the flight crew can only look at each other and say "WTF was that?" We're losing the human interface between the flight computers and the airframe itself. If I die in a crash, I'd rather know it was because of pilot error, and not because someone in the SQA department suffered from a lack of imagination, or glossed over some obscure test scenario that would have failed.
Further, for your information, a number of Airbus catastrophes have been caused by malfunctioning Air Data Inertial Reference Units, leading FAA to issue an "unsafe condition" warning for Airbus 319, 320, and 321 models. (If the "mechanical backups" were adequate, why did these disasters occur?) And "coincidentally", before it went down Air France 447 had been transmitting that its ADIRU was malfunctioning.
The problem with Airbus, as you can find on several online pilots' forums, is that when there is a catastrophic computer error it takes too long to disable the computers and put the plane in "direct law" mode. By the time they accomplish that, as often as not they're toast.
Regardless of whether it would normally be possible to fly without computer airspeed data, a number of Airbuses have suffered catastrophes related to malfunctioning air speed (ADIRU, in this case) units, and the Air France plane had already signaled that its unit was malfunctioning, making it just be one incident among many. The only thing unusual about it was that it was a 330, not a 320.
Aha, yet more uninformed bullshit.
Maybe if you actually read what I wrote, you could come up with some real objections, rather than your own uninformed bullshit.
The 777 has a mechanical backup system, exactly as I stated. Look it up, bozo. All Boeings to date have.
I did not state that Boeings were not "fly by wire", I said they had MECHANICAL BACKUPS. I meant as opposed to entirely (or nearly entirely) fly-by-wire only. It's right there in my comment, in plain English. Further, despite what I stated earlier, even the 787 has electrical/mechanical manual backup in case of computer failure. Of course, if you suffer a complete loss of electrical power (including even the turbine generator), you are SOL either way.
I'm afraid you are spouting pure shite - no aircraft would be certified by either the FAA or the EASA if your stance was correct...
There is a reason that there are so many independent power generation systems onboard a modern (1970s onwards) aircraft, and there is a reason that those systems are checked and certified - there has been zero incidents where an aircraft has completely and utterly lost power during any stage of flight.
And if you dispute that Airbus has been prone to crash due to computer failure, I invite you to look that up, too.
If you look up the Airbus laws yourself, you will see that the only manual control for pitch, for example, is the trim wheel. This is going to do you very little good unless your plane is already in stable horizontal flight... seldom the case when catastrophic failure occurs.
I am *very* familiar with the Airbus control systems, and your assertion that the trim wheel is useless unless in stable horizontal flight just screams that you know fuck all about what you are talking about. You can use the trim wheel to fully control the airframe in all circumstances where you would have to do so with the elevators, because the forces you have to exert during direct mechanical linkages to the elevators would break your arms on a large civil airplane and involve leverage forces that you could not exert from the tiny control columns on modern aircraft.
I'd love you to show us any cases of an Airbus crash that was attributed to computer failure - you won't find one. And I dare you to trot out the Habsheim A320 crash...
On the other hand, we have perfect examples such as the Air Transat flight 236, which lost both engines due to a fuel leak which exhausted the entire fuel supply (and once again missed by the pilots - they completely ignored all the warning signs), and had to glide for 65 miles without engines and only running on the Ram Air Turbine electrical generator. That aircraft landed perfectly safely.
But the main thing -- one I didn't see the point of going into until you decided to butt in -- is that in order to go into "Direct Law" in an Airbus, which is what you might consider your main "manual backup", you have to jump through a lot of hoops, which (as it turns out) pilots just do not have time to do in an emergency.
To put an Airbus aircraft into Direct Law takes precisely two button presses, both within reach of either pilot - but if you aren't in Direct Law, there's really no reason for you to manually put yourself into it, but its there if you want to.
It still amounts to the pilots not being able to control their aircraft when the computer fails.
Again, the same utter shite from yourself.
There's a lot of bullshit touted on the internet about Airbus aircraft - I suggest you actually educate yourself. And I suggest staying away from Wikipedia - there is so much wrong information on there, and so many trolls willing to spend their lives pushing their own warped view through articles such as the ones linked to this, that I no longer consider Wikipedia worth time or effort at all.
Simple, it was written by Canonical.
Interesting. I see lots of lawsuits from the families that lost loved one in the Airfrance Crash.
Yeah, and you're still ignoring the arrogance that went into Airbus's FBW design. THERE WAS NO TACTILE FEEDBACK IN THE STICK. So why the one pilot was freaking out and pulling up on the stick, the other had no clue that he was pulling up and had the nose up at 30 degrees IN THE MIDDLE OF A STALL. As a designer of aircraft controls, you should give the pilots every single thing that you CAN give them in order to ensure safety. Even if you think that they won't need the controls to feel any different from a video game joystick, maybe, just MAYBE it could help out if your copilot can feel where the controls are?
Imagine driving your car at 50 kph on the highway and having immense fog overcome the vehicle. Now imagine even being able to try to keep the vehicle straight with a turn potentiometer. Remember that you ordinarily know where your steering wheel is because the caster of your wheels causes a mechanical feedback which centers the wheel above a certain speed. But you don't have this feedback anymore. You have an index mark on a turn pot. But even if you can't see, wouldn't it be nice to have a tactile feeling of where the controls are? Now add to the mix that you have a copilot going nuts on the controls, you don't have a clue that he's going nuts because the system just averages the inputs of the two of you, and you have a problem. Might work well in clear skies when the two of you have visual feedback and can accordingly coordinate with each other or have sensor data which you can rely on (believe me, these guys were confused when the stall warning came on when they nosed down), but you're otherwise cruisin for bruisin.
Give these pilots every tool that you can realize in order to maximize safety margins. It is purely arrogant to think that you cleverly designed your system so that you wouldn't have to put tactile feedback like stick/yoke and throttle position. And who designed these...
The flight control computers implement all control laws, so you're not disabling any computers at all. Let's talk about, say A330. There is a set of three primary control computers (PRIM) that produce data for the control surface actuators, and a set of two secondary computers (SEC). SEC only implements the direct law, PRIM implements all control laws (normal, alternate 1/2, direct, with mode modifications for flare and ground).
The control law downgrading can only be handled automatically. You pretty much have to pull circuit breakers to change it manually. If there's a "catastrophic computer error", say all PRIM computers down due to overheating (has happened at least once due to air conditioning failure) the downgrading or selection of SEC computers will happen automatically, the pilot isn't expected to have to handle that.
A successful API design takes a mixture of software design and pedagogy.
Example: with both radar altimeters dead (breakers pulled), the computer selects direct law automatically when you deploy the gears. If you're brave and would kill the primary ADIRU, then you'd probably also immediately get the direct law activated, although don't quote me on that.
A successful API design takes a mixture of software design and pedagogy.
When profit is the goal of your company, you do whatever it takes to make as much profit as possible.
Could they of tested it better? Yes, very much so. Did they? Doesn't seem like it, does it?
Is someone getting in trouble for it?
No.
Be seeing you...
Sorry, that would be flare law, close enough to a direct law :)
A successful API design takes a mixture of software design and pedagogy.
List those "catastrophes" please.
And it's been well established by black box data that the air speed disagree errors were not the ultimate reason 447 crashed - the only people still claiming it is are anti-Airbus people such as yourself. The final minutes of 447 are very well documented.
Something in the water is making everyone stupid.
Just thought I would throw it out there.
In fact the Qantas safety briefing tells you to keep your belt fastened at all times. It seems like good advice to me, although often when I fly I notice people on the flight that don't have their belt on.
Qantas is one airline that is now actively encouraging its business-class (and higher) passengers to be up and about during their flights. Not only do they tell them all about the need to exercise, on larger planes (like this one) they provide a bar, as an alternative to waiting for the flight attendants to come round.
One of the axioms of computer science is: "Robots should not come close to people." Robots have moving parts that hurt people, and they posses no judgement.
In the case of Airbus aircraft - the people are flying INSIDE the robot-- where the flight crew are only "voting members" -- with the computers having a majority vote. The crew are relegated to the position of redundant components and systems mangers, where the comment most often heard by the crew is: "why did it just do THAT?!"
Boeing, you got it right by allowing the pilots to always have the ability to override the computers.
And the author failed to protect his code from a failed (out of range) sensor signal. I would fire the bxxxxxd.
Leslie Satenstein Montreal Quebec Canada
I'm afraid you are spouting pure shite - no aircraft would be certified by either the FAA or the EASA if your stance was correct
That's speculation, not a factual argument. Do you set the rules for the FAA? Is the FAA infallible? (I'll stick with them since I'm more familiar with them than the EASA.)
"There is a reason that there are so many independent power generation systems onboard a modern (1970s onwards) aircraft, and there is a reason that those systems are checked and certified - there has been zero incidents where an aircraft has completely and utterly lost power during any stage of flight."
No shit, Sherlock. Where did I state otherwise? I mentioned myself that you are SOL either way, if your power goes completely out, including the ram turbo. I guess you missed that part?
I "am *very* familiar with the Airbus control systems, and your assertion that the trim wheel is useless unless in stable horizontal flight just screams that you know fuck all about what you are talking about. You can use the trim wheel to fully control the airframe in all circumstances where you would have to do so with the elevators, because the forces you have to exert during direct mechanical linkages to the elevators would break your arms on a large civil airplane and involve leverage forces that you could not exert from the tiny control columns on modern aircraft."
Really? I won't argue with you about it, but if so, then it's unlike the trimwheel in just about every other aircraft I have ever heard about, in all of which the trimwheel controls either separate small trim surfaces, or the entire surface but only in a very small range. (Hint: that's where the name "trim" came from.) The situations I am talking about were like the one that happened some years back, in which failure of an ADIRU caused the control systems to put the plane in a 30-degree nosedive. Good luck disabling the computers and pulling out of that in time, if you aren't at altitude. You'd better be at altitude if you lost all power including the generator anyway, otherwise you'd never get the turbo deployed, and that trimwheel is all you'd have for pitch.
"I'd love you to show us any cases of an Airbus crash that was attributed to computer failure - you won't find one. And I dare you to trot out the Habsheim A320 crash..."
Glad you said "attributed to" rather than proven, since the causes are often not proven.
Let's start off with Air France flight 447, which as I stated previously, had transmitted earlier that its ADIRU was malfunctioning. The only unusual thing here was that it was a 330.
http://www.bloomberg.com/news/2011-05-26/air-france-crash-pits-pilot-brains-against-computers-taking-over-cockpits.html
The autopilot was not functioning at the time of the crash of 447 (probably due to that failed ADIRU). Guess what? That's a computer failure. Although I admit that does not prove that it was the proximate cause of the crash. It certainly DOES mean that the crash has "been attributed" to computer failure.
http://www.computerweekly.com/news/1280090068/Air-France-Airbus-crash-system-pitot-sensors-a-factor
But ADIRU failures, while they can't be said to be "common", are not exactly what you would call extremely rare on the Airbus, either. See the FAA ruling further down. There was one failure in a Quantas flight a few years ago, in which 51 people were injured due to the computer malfunction caused by a failed ADIRU. Not a crash, though... not that time. But don't take my word for it:
Please see my other replies. I'm simply quoting other people who are supposed to know much more about this stuff than I do.
"The final minutes of 447 are very well documented."
Yes, they are. And if you follow those links I posted in my other replies (I am not going to post them again) you will see that the plane had been transmitting that its ADIRU was malfunctioning, and it is also known that the autopilot was not functional at the time of the crash. Whether the failing ADIRU caused that condition is a matter of speculation, but it has happened on other Airbuses, sometimes with tragic consequences. Mostly, though, on the A320, not the 330s.
I am not saying that the computer malfunction was the "proximate cause" of the crash. But the computer did malfunction. That much is also, as you say, very well-documented.
damn. not the post you want to read 2 days before flying on an A330.
"Everyone knows that vi vi vi is the number of the beast" -- Richard Stallman
Launch?!?!? What, are we on the space shuttle now?
I'll go with "take off" for $200, Alex.
From my experience it comes down to bad testers. Or if you wish to go further, bad management decisions in doing generic testing, by professional software testers, using testing software, rather than real business area experts.
It is entirely different. It is way cheaper to farm out the testing to a bunch of computer guys running automated testing software than it is to pull experts off projects to have them actually look at things. Automated testing for example would say, "Yes this sensor is reporting a value, and that value is being fed into an algorithm, and it is calculating an event", and the testing monkey would take is testing document, and put a big check mark next to that section, and continue on. The business area expert would be the one to notice, "Hey that number is unreasonable, and it shouldn't be reporting that, and hey when that number is fed into the algorithms it produces an invalid result!"
So don't blame the coders. I blame this on management for not ensuring the code was tested properly. That said it is easy to miss bugs, but if you are looking for blame make sure it goes to those responsible, not just a scape goat on the guy that did the actual work.
On corporate applications I don't do the actual coding but I do take a look at testing. I have had software versions sent to me to verify, and had EVERY single feature fail. Presumably some sort of testing went on at several stages, but if you have absolutely NO idea what you are looking at or ZERO understanding of the business, which describes most outsourced vendor code monkeys (which is how it is done everywhere), I don't think this is at all surprising.
"I can't help wondering just how a piece of code, which presumably didn't test its input data for validity before acting on it, could become part of a modern jet's onboard software suite?"
The author is clearly not a software engineer.
Agile programming strikes again!
Thanks! Made me laugh!
-kgj