Slashdot Mirror
Toyota's Engineering Process and the General Public
Doofus writes "The Washington Post has published in today's paper an article titled 'Why it's so hard for Toyota to find out what's wrong' by Frank Ahrens on the Toyota situation and the difficulties of adequately conveying to Senators and Representatives — most of whom are non-technical — the debugging process. Ahrens interviews Giorgio Rizzoni, an 'expert in failure analysis' at Ohio State, who describes the iterations of testing that NHTSA will likely inflict on the Toyota sample cars they have purchased, and then moves into the realm of software and systems verification: 'He explained that each vehicle contains "layers of computer code that may be added from one model year to next" that control nearly every system, from acceleration to braking to stability. Rizzoni said this software is rigorously tested, but he added: "It is well-known in our community that there is no scientific, firm way of actually completely verifying and validating software."' Ahrens ends the piece with a quote from a 2009 LA Times interview with former UCLA psychology professor Richard Schmidt about how user reports are often unreliable: 'When the driver says they have their foot on the brake, they are just plain wrong. The human motor system is not perfect, and it doesn't always do what it is told.'"
Toyota is currently planning an event to challenge evidence presented by professor David W. Gilbert that called into question Toyota's electronic throttle system.
> Toyota is currently planning an event to challenge evidence ...
Macroscopic events generally don't challenge evidence. They challenge the politics of evidence.
One challenges evidence with small, discrete, verifiable events.
-- IANAL, this isn't legal advice, and definitely isn't legal advice for you. Also, Squee!
"It is well-known in our community that there is no scientific, firm way of actually completely verifying and validating software."
How wrong can you be? Yes there is. Software is fundamentally the composition of many mathematical functions. Its results can be formally proven if the hardware it is running on is assumed (or preferably also proven) to be error free. Don't get me wrong, it would be incredibly cost, labor and time expensive, and require real computer scientists, but it is certainly possible.
Why exactly is there a congressional case going on about this? It becomes even more worrying when you realize that the US government has a controlling interest in most of Toyota's competitors in the USA. In short, why, in a country where states are going bankrupt, privacy is an illusion, healthcare reform has boiled down to if you are pro or anti Obama, rampant spending and tax increases. In short, why do I care about this? File a class action lawsuit and let the courts settle it. Nothing is worse then a bunch of politicians knowing nothing about engineering, with stock in competitor's companies and large problems they haven't solved wasting their time with this crap.
Taxation is legalized theft, no more, no less.
When the driver says they have their foot on the brake, they are just plain wrong. The human motor system is not perfect, and it doesn't always do what it is told.'
This was true with Audi in the 80's, when 60 Minutes did a report where, among other things, they faked a car accelerating out of control (the car was modified extensively.) And yes, a large number of drivers, particularly the elderly, hit the wrong pedal all the time.
However, there are cases where driver reports are plenty accurate. A great example of this would be the problems Volvo V70R and S60R owners have with brake failure while going up hills.
I've experienced it three times in the 6 months or so that I've owned my car. Each time, I was headed up a hill towards a stop sign, put my foot on the brake, and there was nothing there- I had to push so hard I was pulling against the steering wheel for leverage. This is a car with big, high-performance brakes that can stop on a dime.
Volvo claims there's no problem, despite numerous reports on the V70R.com and Swedespeed forums. No other models demonstrate the behavior.
Please help metamoderate.
come on, it's just a big conspiracy.
it's not like 100, 200, one thousand toyotas are
skidding of the highway and into a tree everyday.
there are like a handful of incidents.
-
naw, this is just a big PR campaign of american motor
industry to smear superior japanese tech.
the prius is like a 5 year old car model and in all this
time american "muscle" motor never came up with an answer.
-
big oil and big car a big happy american family.
-
the engine (sic) that drives the (u.s.) capitalistic machine needs
consumption and waste, not innovation and thriftiness.
I find it odd that the systems in vehicles do not have a default "debugging" which should basically trigger the vehicle to stop.
Why does the vehicle ABS (from what I know from the news) get tripped up on instant breaking? Really? ABS... the thing that is supposed to pump the break to allow for cleaner stops triggers breaking problems and increased acceleration?
I just think bad coding in general here. Regardless of "testing"
Of course Toyota is right. The most likely cause of these "sudden acceleration" problems is humans with their foot on the gas pedal. I've owned plenty of Toyotas, and I wish that my current Toyota was in need of replacing right now, because now is a great time to buy one. Unfortunately, my current Toyota only has 150K miles, meaning that I have a good 5-10 years of life in my vehicle. After that... I'll buy another Toyota.
I don't respond to AC's.
... being in control of braking and acceleration.
If you disagree with me on social issues, then it's pretty clear that you are a narrow-minded bigot.
"It is well-known in our community that there is no scientific, firm way of actually completely verifying and validating software."
Um ... did this guy ever heard of formal verification? Or is math proof not good enough for him?
It is well-known in our community that there is no scientific, firm way of actually completely verifying and validating software.
Looks like Toyota's suffering from a halting problem. ;)
Humans are fallible. You can't dismiss user reports. You can review them skeptically, or examine them for trends.
EVERYBODY knows that cell phones cause cancer. So, why hasn't somebody fixed that?
EVERYBODY knows that vaccines cause autism. So, why hasn't somebody fixed that?
EVERYBODY knows that they're smarter than average. So, how did the last few presidents get elected? :)
Interestingly, the heat death of the universe provides an alternative solution to the Toyota braking problem: It will probably stop the cars. (I say "probably" because I don't have time to do a formal verification.)
Less than 100 cars out of 8,000,000 have had this problem. That is a 0.001% failure rate.
Of those 0.001% of cars that had the problem, how many times did someone drive them before they failed?
I don't want to say this is user error, but I have seen some users do stupid stuff and not even know they did it.
Im a gamer, not a grammer major. This post is full of spelling and grammer mistakes.
General Motors has been making cars with poor reliability literally since I was a child. Read your library's old copies of Consumer Reports for verification.
Insufficient attention was given to the poor reliability of G.M. cars, in my opinion.
As long as G.M. cars could continue to be sold, making unreliable cars was more profitable. That's similar to making a sloppy computer operating system that is vulnerable to attacks. The sloppiness helps sell new versions.
If neutral won't work- you can also turn off the ignition, but don't turn the key completely off, or you'll engage the steering lock(ie, go to the 'accessory' position.) You will not "lose steering"; at any speed over about 2-3MPH, steering assist becomes less and less necessary, particularly if you don't have very wide tires.)
If you "ride" the brakes, the pad and rotor will heat up and "cook"; consumer, mass-market pads are designed to have good "cold" (ie instant) grab, be easily modulated, quiet, not cause excessive wear on the rotor, and not generate brake dust that is impossible to remove from the wheels. Racing pads are designed for higher temperatures (where among other things, you get much more heat transfer from the rotor to the air blowing past/through it), but they have very lousy "cold" bite. Also, heat up the calipers enough, and you will cause the moisture in the brake fluid to boil (your brake fluid should be changed at a MINIMUM every 2 years, because it is hygroscopic), and that boiling will result in "vapor lock"- no brakes. The brakes MUST be bled after such an incident.
Audi successfully defended itself from several lawsuits and even won a countersuit in a case where a mother crushed her boy against their garage wall (after going through the garage door!). Interviewed by an officer afterwards, she repeatedly said she'd hit the wrong pedal. They sued a few months later claiming the car had "gone out of control". As someone who knows Audis well, particularly the mid-80's 5000 turbo series- the idle stabilization valve (the only way the car computer can increase engine speed) simply cannot allow enough air to bypass the throttle enough to cause the car to lay down burnt rubber, crash through a garage door, and embed itself in a house wall.
The problems with the Volvo "R" models have been reported in a number of other european cars; you'll also see the words "ice mode" thrown around occasionally. Many ABS controllers since 1990 or so have an accelerometer to detect when all the wheels stop simultaneously but there is no corresponding negative acceleration. "Ice mode" is supposedly some sort of variant of this, and there has been great debate as to whether this "mode" is internet folklore, but you'll find many, many posts on all sorts of varying car enthusiast forums.
Please help metamoderate.
The most relevant thing I've read about the problems with Toyota vehicles is this quote from the bottom of page 3 of that PDF linked above:
"... it was determined that [Toyota] Electronic Control Module (ECM) malfunction detection strategies were not sufficient to identify all types of fundamental APP sensor and/or circuit malfunctions. Some types of Electronic Throttle Control (ECT) circuit malfunctions were detectable by the ECM, and some were not. Most importantly, the Toyota detection strategies were unable to identify malfunctions of the APP sensor signal inputs to the ECM. APP sensor signal circuits must be undeniably correct to electrically convey the appropriate driver commands to the ECM."
Next paragraph:
"With the two APP sensor signals shorted together through a varying range of resistances, all four Toyota vehicles tested thus far reacted similarly and were unable to detect the purposely induced abnormality. The types of signal faults introduced into the APP circuit should have triggered the vehicles' ECM to illuminate a warning lamp within seconds."
Bottom of page 4:
"In addition, the shorted APP signal circuits were connected momentarily to the sensor's five-volt supply circuit with the vehicle in drive. In all test vehicles, the ECM did not set a DTC and the engine speed increased rapidly to full throttle. This result shows that unusual or sudden unintended acceleration of the vehicle was possible in the ETC test vehicles."
Erroneus wrote:
Wow. Just wow. Never has a nick been so apt.
This isn't a Toyota thing. It isn't even exclusive to the auto industry. System complexity was where so many cliches like "Fast, complete, cheap: pick any two" come from.
Sure, we can put missile-guidance software protocols into all sorts of software development; If I remember the metric, every line of code costs 10x as much as in general industry.
Another thought: Airbags took 15 years to get acceptance from their 1970's invention -- the industry quickly realized their safety value, but nobody wanted to pony up $800 (1980 estimated per-car cost) or increase the cost of a car to eat that cost.
And don't even get me started on FAA vs. adequate safety. Or Seldane and the FDA.
tl;dr: Toyota *DOES* test extensively. Shit happens.
The simple fact is that overall a Prius with its minor brake transfer problem is far safer than any pre-ABS/traction control car. The fault is far less serious than, say, brake fade in drum brakes. And I don't even own a Toyota. You don't need any kind of tinfoil hat to think this is about bashing the part of the motor industry that is not US-owned.
From scarped cliff or quarried stone she cries "A thousand types are gone, I care for nothing, no not one."
Opinions on verifying code as a means to tell whether a Toyota will have 'sudden acceleration' above are UTTERLY, well, let us say, ill thought out in my opinion, in most cases. Code is only ONE part of an almost hopelessly complex system when ALL THE POSSIBLE VARIABLES are analyzed.
Failure analysis may start with code, but these systems then can encounter intermittent connections, power surges, static generated by multiple known and unknown items (including the rare intermittent connections), induced currents in parallel wires, temperature induced changes, faulty seals & water/condensation intrusion, etc. By the time an accident investigator looks at a vehicle that had a problem, the transients are long gone.
Intermittent Mechanical (& thus often electrical) changes & failures are an absolute bane of complex systems.
In my opinion, the only way you can find these rare transient problems is to find vehicles who have been reported to have these problems (& didn't crash) and then you load them up with data loggers and drive the hell out of them in all sorts of environments.
Personally, I really like a 1972 Blazer...with a manual transmission. Minimal plastic, no electronics beyond the turn signal module, fix it myself and I can start it with a bit of a downhill run. Yup, I drive my Highlander, but I'm thinking of putting a 72 Blazer back in as new shape.
Here is an example of a person that brought a car to the dealer while it was pegged - mechanic played with pedal and studied the situation:
http://www.leftlanenews.com/feds-investigate-toyota-electronics-for-unintended-acceleration.html
``Dismissing user reports is what got Toyota in trouble in the first place. Keep doing that. See how far it gets you.''
Right. Nobody I know about actually has a problem with there being a defect in the vehicles. The defect should not have been there and it's a great shame that it was, but everybody understands that it happens. If it happens too often, that gives you a poor reputation, but it doesn't happen to Toyota a lot so their reputation there is good.
Where Toyota went wrong is in how they handled the incident. What they should have done was err on the side of caution, notify people of a possible issue, and encourage them to be careful and report anything that might be related to Toyota to help them investigate the issue. Only after they would have done their best to confirm the issue could they have concluded that the issue does not actually seem to occur, and even in that case they should not have told people that there is no issue, especially not the people who report experiencing it.
What they did instead was deny that there was an issue before they had properly investigated it, and effectively called the reporters of the issue liars. Calling your customers liars is a very bad idea, and doing so with those who report a rarely occurring issue not only insults them, but also deprives you of an important source of information. It's probably the very worst thing they could have done.
Figuring out the parallel between this and full disclosure in computer security is left as an exercise to the reader.
Please correct me if I got my facts wrong.
The thing you're missing, is the level of those defects. The problems that GM had with quality were almost never safety related (And when they were, they weren't major and were fixed rapidly). Say what you want that their cars sucked, but in the 100 years they have been selling cars in the USA, they have never had as major of an issue such as this. Ford has (Remember the exploding gas tanks?). Chrysler has (They had an issue with cruise control that caused some accidents). I'm not saying that GM is good (I got rid of my last GM car 2 years ago, and I don't know if I will buy another one). What I am saying is that comparing quality by shear number of defects (As consumer reports does) is ignoring the much more important bigger picture...
If a man isn't willing to take some risk for his opinions, either his opinions are no good or he's no good
The whole Toyota situation has become irrational. People knowingly sell and buy cars with varying levels of safety every single day. The safety differences between all the different models of cars on the road, of varying sizes, ages, and safety features, utterly swaps any marginal risk Toyota is even alleged to have caused. Go ahead and take the model Toyota has recalled the most of, and I guarantee I can find many, many other makes/models with many more deaths per million miles driven. Again, certainly Toyota should fix it. But at some point, paranoia on one small issue just diverts resources away from other bigger problems.
Toyota should be more forthcoming with the black box info on these cars to validate exactly what the driver was doing at the time of the accident. But they won't because lawyers would be all over that data to file lawsuits. still, knowing the truth is best for all involved. Far less finger pointing; far better remediation of the problem.
"I believe in Karma. That means I can do bad things to people all day long and I assume they deserve it." : Dogbert
Car&Driver did some tests and found that even with the throttle wide open the brakes can still stop a car, even a 500hp muscle car. With a normal car the distance wasn't even significantly greater than with closed throttle.
No sig today...
The real problem is people who think that not having any sort of actual linkage is a good idea.
A mechanical linkage is not necessarily more reliable or safer. The fact that you can put your hands on it doesn't by itself make it better or worse. You are making an assumption based on your intuition that you cannot back up with data.
Vehicles have only become more and more problematic since the late 70s due to increased reliance on electronics in place of actual mechanical parts.
Nice sound bite but problematic in what way? Cars today are in general demonstrably more reliable, last longer, rust less, are (generally) safer in crashes, more powerful, and emit less pollution. At one point I made my living selling classic cars from the 70s and earlier. I'm very familiar with them first hand. You might like the styling better but performance-wise they are inferior to modern cars in almost every way I can think of.
One of the design "features" of the Toyota product involved in the 2009 fatal accident in San Diego was that the driver needed to press the engine start button for three seconds to kill the engine. Can you imagine any machine tool company making a product that required the emergency stop switch to be depressed for three seconds to turn off the machine?
Another issue with that car was that getting the tranny into neutral was not trivial (sport shifting option).
Toyota screwed up big-time here.
A Shadeless room is a brighter room.
While the tone could have been nicer, the AC was correct at least here:
if you have enough time to call 911 you have enough time to stop the car
Yes you probably might forget "the trick" they taught you in driver's ed when you're panicking. I probably would.
Yes people are being tremendously callous when they scoff that "Duh, why didn't you just put it in neutral lolz"
Still, if you really can't come up with SOMETHING to avert plowing into an intersection at 135 mph in the 60+ seconds they had, and you seriously expect someone miles away to get to you in two seconds, well, you were probably living on borrowed time anyway.
Information theory is life. The rest is just the KL divergence.
He wasn't discussing cars as a whole, just the aspects relevant to the Toyota fiasco[1].
No he wasn't. He said "The real problem is people who think that not having any sort of actual linkage is a good idea." That has nothing whatsoever to do with Toyota specifically.
On old cars there's nothing second guessing you.
That doesn't automatically translate to better or safer. It's simpler but that is all you can say for certain unless you want to compare specific cases. Just as newer is not always better, older is not always safer.
Yes, obviously some things are better on modern cars, but that's not the point here
No that's exactly the point. The grandparent post was implying that a mechanical linkage is intrinsically safer while providing no evidence to back up that assertion. If you are going to declare drive-by-wire to be more dangerous than the alternatives, you had better back up that declaration with data.
I've seen this "mechanical linkages are safer" argument before and I've never seen anyone making it actually back it up with facts. They just pre-suppose that the simpler, older technology is safer. It may be or it may not be but I've yet to see anyone prove it.
Unsafe at Any Speed: The Designed-In Dangers of the American Automobile
Wrong. Cars have become MUCH more reliable over the years. Lots can go wrong with mechanical systems. A spring breaks, a rod binds, whatever. A friend had a car break the throttle return spring on a old muscle car and it took off like a rocket, hit a k-rail, ripped off both front wheels, went airborne and landed on a nice Cadillac.
Know what a tune-up is? You used to have to do one at least once a year to keep your car going. Not really done anymore.
I could go on like this for quite a while. I like working on old cars because they are simple. But the new cars are more reliable.
Anarchists never rule