Stack Overflow Could Explain Toyota Vehicles' Unintended Acceleration

New submitter robertchin writes "Michael Barr recently testified in the Bookout v. Toyota Motor Corp lawsuit that the likely cause of unintentional acceleration in the Toyota Camry may have been caused by a stack overflow. Due to recursion overwriting critical data past the end of the stack and into the real time operating system memory area, the throttle was left in an open state and the process that controlled the throttle was terminated. How can users protect themselves from sometimes life endangering software bugs?"

Comments

Wow

[rsilvergun]

Is there anything Stackoverflow can't do?

Re:Wow

[wickerprints]

Gives a new meaning to "race condition," doesn't it?

Did anyone else read that title and think...

[RevWaldo]

..."We'll I'm sure somebody on there could!"

.

Go Amish?

[dbc]

"How can users protect themselves from sometimes life endangering software bugs?"

Amish buggies typically don't have software throttle failures. Run-away horses can be an issue.... and actually having to share the road with dipshit drivers who don't understand the number of slow moving vehicles (not just buggies) that there are out in farm country are a real danger.

Seriously, software has bugs. Mecanical throttle linkages can stick, too. Life has risks.

Re:Go Amish?

[CodeArtisan]

Coming from the aerospace industry, you cannot have software that has bugs. And if there was the possibility of a software bug, you have to prove that you can mitigate the effect in hardware. So just to say "software has bugs...life has risks" isn't an acceptable answer (in my opinion). We have to remember this is not an apples to apples comparison. Just because traditional consumer software always has bugs in it (which are acceptable) doesn't mean they are acceptable in other industries. Considering that the failure puts someone's life at risk, I would think it should be considered unacceptable in automotive industry as well.

If you want your cars to be as expensive as a 747, then you can attain that goal. I used to work in the automotive industry designing embedded software for engine management systems. At that time, no automotive company would pay more than $100 for the Engine Control Unit. Probably 60% of the code was written to manage failures (both software and hardware), and there were other electronic fail safe mechanisms. But you can't mitigate every possible failure event without introducing costs that would have made the unit orders of magnitude more expensive.

Re:Go Amish?

[arkhan_jg]

Even in the aerospace industry, there are software bugs. Very few, yes, because a lot more time and money is spent to track them down. There are mechanical failures too, despite the best engineering efforts. But anything we build has the potential to be flawed somewhere in the process. That's why we still put highly trained pilots in the things, for when something goes wrong - and even then, human error causes unintended faults, sometimes catastrophically.

If a car cost as much as a jet, and drivers went through as much training as a passenger pilot - and had to have a co-driver at all times - we'd be far safer on the roads.
After all, the vast majority of car crashes are driver error. And failure modes when you're at 30mph on wheels are a lot better on the whole than when at 30,000 feet. But if we built cars to the same standard, and held drivers to the same standard as aerospace engineering, only the rich could afford to.

There's a trade off between acceptable risk, and cost. Even though the designs get safer every year, maybe we allow too much risk in drivers and their cars. But absolute flaw free cars? Impossible.

Mental stack overflow of the driver is more likely

[Anonymous Coward]

Idiot drivers hit the gas pedal instead of the brake and instead of owning up to their incompetence as a drivers, they blame the car instead. The Toyota sudden acceleration problem disproportionately affects the elderly and inexperienced drivers. It also a uniquely an American problem and it occurred during a deep recession where GM and Chrysler were going bankrupt and Americans needed some FUD against Toyota because supporting American car companies was the jingoism of the day. The toyota sudden acceleration is more of a case study of an American moral panic and mass hysteria perpetrated by the media than it was an engineering problem.

coding standards

[lkcl]

... you know... i worked for pi technology in milton, cambridge, in 1993. they're a good company. they write automotive control systems, engine control management software, vehicle monitoring software and diagnosis systems and so on. one of the things i learned was that coding standards for mission-critical systems such as engine control have to be very very specific and very very strict. the core rules were simple:

1) absolutely no recursion. it could lead to stack overflows.
2) absolutely no local variables. it could lead to stack overflows.
3) absolutely no use of of malloc or free. it could lead to stack overflows.

now you're telling me that there are actually car manufacturers that cannot be bothered to follow even the simplest and most obvious of safety rules for development of mission-critical software, on which peoples' lives depend? that's so basic that to not adhere to those blindingly-obvious rules sounds very much like criminal negligence.

Stack overflow vs. buffer overflow (difference)

[Trax3001BBS]

For anyone else that's curious; at first I thought it was double speak, so not to sound as bad.

Stack overflow refers specifically to the case when the execution stack grows beyond the memory that is reserved for it. For example, if you call a function which recursively calls itself without termination, you will cause a stack overflow as each function call creates a new stack frame and the stack will eventually consume more memory than is reserved for it.

Buffer overflow refers to any case in which a program writes beyond the end of the memory allocated for any buffer (including on the heap, not just on the stack). For example, if you write past the end of an array allocated from the heap, you've caused a buffer overflow.

http://stackoverflow.com/quest...

Dangerous recursion!

[mveloso]

From the slides:

"Toyota used dangerous recursion"

Not like that safe recursion that other vendors use.

Re:Live in a cave

[NiteTrip]

I had my car suddenly accelerate on me before. I was driving along and suddenly the pedal felt really strange and it start accelerating, even when I took my foot off the pedal. I turned off the car and pulled over. Turns out the rubber mat I put in to protect the inside of my car from wet/snow had somehow managed to flop on top of the pedal and pushed it down. When I heard about these Toyotas accelerating on their own, it's the first thing I thought of.

Re:In Canada Engineers Are Required to Write the C

[Cassini2]

Almost all safety systems in Ontario have not been designed or written by a PEO licensed engineer. The PEO is the same organization that tried to get Microsoft to stop using the term "Microsoft Certified Systems Engineer (MCSE)" and largely lost. If you start analyzing real and deployed systems, you will be shocked at what you find.

Yes, there are a few very well designed machines out there that do hardware and software interlocks properly, and in an obviously safe fashion. These are the exception, and I am delighted to find the few exceptions that exist.

However, if you want excellent examples of obviously unsafe things, consider:

- The gas pumps at Shell, Esso, and Petro-Canada. How many brands have an Emergency Stop button? One?

- Toyota cars have a push to start button that is also a push and hold to stop button. So how do you stop the car quickly? Shouldn't a car that has push-button start, also have a push-button stop, that is a different button and works quickly? Why would Toyota follow the Microsoft standard of using a start button to stop, instead of following the very well thought out emergency stop button standards?

- Hospitals have implemented a number of computer systems that are networked, and make the job of nurses quicker, easier and more productive. This reduced nursing costs considerably, and fewer nurses are looking after more patients. However, these systems are not reliable, and the official backup plan is that a nurse will step in and do the job manually if the system fails. Unfortunately, many of these core systems are also running on Microsoft Windows (often Windows XP.) One virus, or one bad update, written by a non-engineer, to wipe out many core systems. A major hospital had its Internet linked systems disrupted because too many people watched Olympic hockey (over the critical internal network.) Has any engineer approved any of this? Does any hospital have enough nurses to cover off in the event of a computer failure?

- Most servo-motor drives are sold with a "not recommended that power be cut by an emergency stop/safety system" warning buried somewhere in the documentation. Ignoring this, and assume braking resistors are used, and power is really cut. Most motors will follow an exponential stopping curve, and appear to coast to a stop. A mechanical engineer doing a PSHSR (Pre-Start Health and Safety Review) will expect the machine to stop quickly, and not coast. The cheapest way to do that is to dynamically brake into braking resistor under full software and transistor control. The second cheapest way is to use a parking brake, but those are not rated for safety and only a fraction of the servo-motor market uses parking brakes anyway. How many PEO licensed mechanical engineers doing PSHSR reviews have passed systems with incorrect software E-STOP circuits and Safety Circuits, and failed E-STOP circuits and Safety Circuits that cut power to the motors in hardware?

Do not depend on the PEO and statutes to keep you safe ...

Re:"Stack Overflow" not good for discussion site.

[firewrought]

Technically knowledgeable people often give very poor names to their efforts.

I thought "Stack Overflow" was great branding for a website aimed at helping programmers solve technical problems. It's a distinctive, cheeky in-reference understood by its intended audience. (And honestly, it didn't hurt that most developers enjoy being made to feel clever about themselves.) That's what a brand is suppose to do, and it partially* explains their overwhelming success. And hey, much better branding choice than ExpertSexChange.com!

*Of course, branding is just one of many things they did right. They also filled a unique niche, understood their community (because it was started by programmers, for programmers), and made the site super-easy to use by (here comes the important part...) NOT crapping over the UI with a fake paywall that sought rent for years' worth good-faith user contributions. However, they are sort of starting to be dicks about subjective questions (such as help with API choices, etc.). That may provide a niche for a new competitor to fill...