Slashdot Mirror


NHTSA Has No Software Engineers To Analyze Toyota

thecarchik writes "An official from the National Highway Traffic Safety Administration told investigators that the agency doesn't employ any electrical engineers or software engineers, leaving them woefully unable to investigate correctly what caused the most recent Toyota recall. A modern luxury car has something close to 100 million lines of software code in it, running on 70 to 100 microprocessors. And according to consultant Frost & Sullivan, that number will rise to 200 to 300 million lines within a few years. And the software that controls the 'drive-by-wire' accelerators of Toyota and Lexus vehicles is one potential culprit in the tangled collection of issues, allegations, and recalls of many of those vehicles for so-called 'sudden acceleration' problems."

91 of 459 comments (clear)

  1. With all the recent US layoffs ... by WrongSizeGlass · · Score: 4, Informative

    ... there is plenty of talent out there for them to hire - even if only on a project by project basis.

    1. Re:With all the recent US layoffs ... by Marxist+Hacker+42 · · Score: 2, Insightful

      I know plenty who are laid off for other reasons- such as their C-level executives being slackers and the whole bloody company going under.

      --
      SJW: a person who perceives an injustice, and while correcting it, commits a greater injustice.
    2. Re:With all the recent US layoffs ... by ryanvm · · Score: 2, Insightful

      Not in the auto industry - it's mostly union. There is no correlation between ability and likelihood of employment.

    3. Re:With all the recent US layoffs ... by Marxist+Hacker+42 · · Score: 3, Funny

      No

      --
      SJW: a person who perceives an injustice, and while correcting it, commits a greater injustice.
    4. Re:With all the recent US layoffs ... by ottothecow · · Score: 2, Funny

      Are you a liar?

      --
      Bottles.
    5. Re:With all the recent US layoffs ... by Marxist+Hacker+42 · · Score: 2, Funny

      yes

      --
      SJW: a person who perceives an injustice, and while correcting it, commits a greater injustice.
    6. Re:With all the recent US layoffs ... by Marxist+Hacker+42 · · Score: 2, Funny

      yes- This exact comment has already been posted. Try to be more original...

      --
      SJW: a person who perceives an injustice, and while correcting it, commits a greater injustice.
    7. Re:With all the recent US layoffs ... by toastar · · Score: 3, Insightful

      Well I don't know if "talent" is the right word. The people who get laid off are the worst 10%. Usually the real slackers.

      I thought the most expensive got laid off first

    8. Re:With all the recent US layoffs ... by sconeu · · Score: 5, Funny

      But if he is lying then he is telling the truth
      and if he is telling the truth he is lying...

      NORMAN COORDINATE!!!!!

      --
      General Relativity: Space-time tells matter where to go; Matter tells space-time what shape to be.
    9. Re:With all the recent US layoffs ... by Red+Flayer · · Score: 4, Informative

      He's already proved he can create government jobs. The federal government is now larger than it ever has in history.

      [citation needed]

      Are you aware that there were more federal government employees in the 1980s under Reagan than there are today?

      Are you aware that there were more government employees in the 70s under Nixon, Ford, and Carter than there are today?

      Go take your horseshit somewhere else.

      Sources: Article on Bush increasing the federal employment rolls, just to point out your misplaced ire.
      All fed employees, 1962 to 2008 Here you go. What's that? Federal employment peaked at the end of Reagan's term and decreased under Clinton, only to increase again slighlty under Bush? How can that be, in your misinformed little world?
      An article pointing out the increase in federal employees due to Obama's stimulus packages as of last September. It was newsworthy that 25k federal employees were added from Dec 08 to Aug 09. FYI, more have been added since, with 33k added in Jan 2010 as an example. Still far under what we had in the 80s under Reagan.

      Get a clue. Dig into the numbers before you make erroneous claims parroting your stupid right-wing ideological leaders.

      --
      "Trolls they were, but filled with the evil will of their master: a fell race..." -- J.R.R. Tolkien on Olog-hai
    10. Re:With all the recent US layoffs ... by Stradivarius · · Score: 2, Insightful

      This is the same problem faced by businesses who need a 'software' person. Without having a good software person in the company already, how can they tell the difference between candidates? They can't.

      If you're an organization looking to hire your first expert or two, you do it the old-fashioned
      way. You consider their degree and the institution that granted it. You consider their work experience. And perhaps you rely on a referral from a trusted contact who knows more about the field than you do.

      If you're looking for some fine-grained specialization in a particular technology, there are a number of certification programs out there. If you're looking for broader skills sets, there are both BS and MS programs available in disciplines such as Computer Science, Computer Engineering, and Software Engineering. What does a licensing regime get you that certificates and degree programs do not?

    11. Re:With all the recent US layoffs ... by DocHoncho · · Score: 2, Funny

      The A level executives are the Illuminati controllers who give the C levels their marching orders.
      I'm not really surprised you hadn't heard of them, they like to hide in the shadows being Vampires and all.

      --
      Celebrity worship is a poor substitute for Deity worship and costs more to boot.
    12. Re:With all the recent US layoffs ... by GodfatherofSoul · · Score: 3, Insightful

      That's a pretty ignorant statement, isn't it? But, since it aligns with your ideology I guess reality doesn't matter. I guess it was all those union guys in Wall Street that ran our economy into the dirt?

      --
      I swear to God...I swear to God! That is NOT how you treat your human!
    13. Re:With all the recent US layoffs ... by MightyYar · · Score: 2, Insightful

      Outsourcing!

      The "reductions" in federal head count are just politicians beating their chests - the employees all get replaced by contractors.

      The government is not smaller today than it was in the 70s or 80s... just look at the budget!

      --
      W..w..W - Willy Waterloo washes Warren Wiggins who is washing Waldo Woo.
    14. Re:With all the recent US layoffs ... by Bartab · · Score: 2, Insightful

      Hint: There exists state jobs, which are massively in excess at the moment - compared to lossage in every other field.

      Basically, the "stimulus" has been used to shore up failing state budgets to avoid public employee layoffs. Then these jobs are listed as "saved or created", and Obama takes a bow. Meanwhile, productive jobs in the private sector are experiencing 10% unemployment - that's people looking for work, the official Unemployment Rate. Alternative measures reaches as high as 18% in the month of January 2010.

      http://www.bls.gov/webapps/legacy/cpsatab15.htm
      Select U-6.

      Public sector jobs experience "only" a 4% unemployment rate.

      http://mercatus.org/publication/public-vs-private-unemployment

      Shouldn't the least productive, public tax fed jobs be pruned first?

      Oh but wait, those jobs are unionized - primarily - and the system allows the union to get their representatives on both sides of the negotiation table.

      --
      Any sufficiently advanced technology is indistinguishable from a rigged demo.
  2. Here come the shackles. by HungWeiLo · · Score: 4, Interesting

    Here comes DO-178B for cars.

    I wonder what the cost is per line of code?

    --
    There are a huge number of yeast infections in this county. Probably because we're downriver from the bread factory.
    1. Re:Here come the shackles. by Beardo+the+Bearded · · Score: 5, Insightful

      Dunno.

      My kids were runover by an out-of-control Mustang about four years ago. There was nothing mechanically wrong with the car. Maybe it was driver error. I don't know, but apparently the accelerator was still stuck to the floor when the police got there. I remember how the cruise control on the cars I've owned will lower the accelerator when the CC is accelerating.

      I've always blamed the firmware. Maybe that's because I'm an EE who used to write firmware for a living. (Firmware that's been in use in life-critical applications for five years with a 0% failure rate.) Odds are the code is shit and there's an edge case that nobody thought about. Maybe there's an uninitialized variable in there. I've seen it happen before. Of course, I'm not Woz-brand, so my opinion doesn't mean a thing.

      For some reason, the various regulatory agencies (i.e. Engineering Associations) have been rolling over and letting the manufacturers put any code they want into public use without any thought that hey, maybe we should get someone with some credentials to look into it. I've tried to mention it to mine, no results. Maybe they're dinosaurs who think that engineering is about roads and sometimes other things, like buildings and handrails. Software can't hurt people, can it?

      This problem is not limited to Toyota, and we've only just seen the beginning. I guarantee that other manufacturers are clenchinging their butts hoping that nobody in the media wonders about all the intermittent "floor mat" problems.

      --

      ---
      ECHELON is a government program to find words like bomb, jihad, plutonium, assassinate, and anarchy.
    2. Re:Here come the shackles. by gr8_phk · · Score: 2, Interesting

      It could be software, it could even be hardware. Whatever drives the pedal to the floor is probably driven by a MOSFET. If proper FMEA isn't done people will overlook that a failed-short condition might pull the pedal down. I once worked at a company where I pointed out something similar but much less likely to cause problems and was greeted with anger. Another concern I had, they just didn't see how it related to safety - it was like talking to a rock. I've also worked at places that poured rather large amounts of money into investigating failure modes where the outcome was uncertain. As a personal non-scientific observation, the OEMs take safety seriously and so do some tier-1 suppliers, but it gets worse the farther down the food chain you go. People are human and can make mistakes - and that's why the industry does LOTs of testing on real vehicles.

    3. Re:Here come the shackles. by b4dc0d3r · · Score: 2, Informative

      I've seen that feature, basically it helps when switching from cruise control to manual. You put your foot on the gas and release CC, and you can maintain speed. I'm not sure if the CC presses the accelerator in place of a human, or if the CC controls fuel flow and then adjusts the accelerator to match.

      What I do want to know is how many crashed cars had the cruise control "on" but not set. My CC light can be on but not controlling speed until I hit "set". And if I hit the brake or clutch (it's a manual) it goes from "set" back to just "on" where I can control the pedal. I'm betting this is one of those cases where you turn on CC, disengage it through brake/clutch, and at some point CC confuses whether it's "set" (controlling speed) or "on" (waiting to take over).

      There is a variable which keeps track of the current target speed, whether it's engaged or not. You can hit the brake and then hit 'resume' and it remembers the speed. There's a separate variable for whether it should be engaged or not. This variable should be correct at all times, and never changed as a side effect of something else.

      I wouldn't be surprised to see this implemented as the "remembered speed" variable, which Resume uses, and the "current speed" variable, which is 0 meaning disengaged, and positive meaning engaged at that speed. That way you don't have to check :

      if (engaged && speed > 0)

      instead you check just:

      if (speed)

      Embedded systems requiring optimization, someone might be tempted to do this. All you need is an edge case as you say to set this negative (there is a 'decrease/increase' feature on most CC), or faulty memory, or even bits flipped by nearby electromagnetic equipment. You don't even need badly written code, just poor insulation.

      Ah screw it, give me the firmware and I'll disassemble it.

    4. Re:Here come the shackles. by Cassini2 · · Score: 2, Interesting

      The testing was very rigorous, even after a couple of lines of code there were code coverage tests, unit tests, static code analysis, tests of the hardware with Vector CANoe, you name it.

      None of these tests are entirely effective when dealing with embedded applications.

      Bluntly, software tests can only prove the existence of a software bug relative to the specification. For an embedded application, toss the specification out, and start looking at real-world failure scenarios. Glitches on the reset line can cause all sorts of interesting results ... and that is just one possible failure mode.

      On a well designed embedded system, most of the dangerous failure modes involve complex unexpected system level interactions.

    5. Re:Here come the shackles. by Bodero · · Score: 2, Interesting

      My kids were runover by an out-of-control Mustang about four years ago. There was nothing mechanically wrong with the car. Maybe it was driver error. I don't know, but apparently the accelerator was still stuck to the floor when the police got there. I remember how the cruise control on the cars I've owned will lower the accelerator when the CC is accelerating.

      I had a Mustang with an out of control acceleration problem. I was driving down a country road when all of a sudden it kept accelerating. I stomped on the brakes and managed to bring it to about 10 mph, pulled off the road, then turned off the ignition.

      The culprit? I had stored it all winter long (this was the spring) and squirrels had used my engine compartment as their own winter storage. An acorn had lodged itself in the throttle cable and held it wide open.

      Sometimes what sounds like it might be something more complicated is simple.

    6. Re:Here come the shackles. by jhol13 · · Score: 2, Informative

      No clue, but I very much doubt the figure.

      100 million lines is more than in a normal Linux installation (with OS, openoffice, gnome/kde, firefox, etc.)

  3. consultants by N7DR · · Score: 3, Insightful

    Surely it would be a serious inefficiency for NHTSA to maintain on staff a large number of specialists to handle this kind of problem? Isn't that exactly what (properly qualified) consultants are for?

    1. Re:consultants by Hatta · · Score: 4, Informative

      Given how much of our vehicles are run by computer, I don't think there should ever be a lack of demand for software engineers at the NHTSA.

      --
      Give me Classic Slashdot or give me death!
    2. Re:consultants by TubeSteak · · Score: 2, Insightful

      Surely it would be a serious inefficiency for NHTSA to maintain on staff a large number of specialists to handle this kind of problem? Isn't that exactly what (properly qualified) consultants are for?

      I agree that it'd be inefficienct to have a large number of EEs & SEs on staff, but they have no one to do even a simple sanity check on the hardware and software that is being certified for public roads. And that strikes me as a failure of their organizational mission.

      --
      [Fuck Beta]
      o0t!
    3. Re:consultants by sjames · · Score: 4, Insightful

      Given that there isn't a car made today whose safety can be properly evaluated without the skills of EE and software engineers, why would it be inefficient for the agency responsible for that evaluation to have people with those skills on staff? It's not like next years cars won't have even more of the same complete with modified firmware to examine.

      Given that the safety evaluation will involve interactions between mechanical, electrical and software systems, you'd want a cohesive multi-disciplinary team, not a revolving door.

    4. Re:consultants by rainmayun · · Score: 4, Interesting

      I can promise you have independent verification and validation contracts are bread & butter in the federal contracting world. The federal government has made huge strides in the direction of outsourcing almost all technical expertise, and quite a bit of management expertise (google "federal PMO contracts" for lots of random examples). The few civil servants left in many agencies are a kind of sheepherders, managing vast groups of contractors.

    5. Re:consultants by fuzzyfuzzyfungus · · Score: 3, Insightful

      If 100,000,000 LoC is common(albeit probably concentrated in more or less irrelevant things like the fancy display console, rather than the ECU) there is no such thing as a simple sanity check... And new cars and models are coming out all the time, from a variety of manufacturers, who are presumably constantly tweaking.

      Under the circumstances, you pretty much have two options. The radical, future-looking one is to say "Ok, clearly complex software is the future. We are going to do whatever it takes, build up a serious software engineering team, impose standards that would make medical device makers cry, sponsor research in automated verification, whatever. Yeah, it sucks that we have do deal with that complexity; but so it goes." The traditional conservative(and, much more likely to fit within your budget and not ruffle feathers) option is to throw up your hands and treat the software as a black box. Have your existing test engineers use their existing techniques, or limited variants, to run the vehicles through test conditions, hoping that, if the test conditions effectively model the real world, any real world critical bugs will appear in testing, at which point you can kick it back to the people who wrote the code and tell them to fix it.

      It seems pretty clear that the NHTSA has pretty much gone with option two. And, frankly, it is hard to blame them under the circumstances. Even at the best of times, technical regulation is a pretty unsexy legislative priority, and tends to be funded accordingly. It wouldn't take an actively antiregulatory corporatist to raise an eyebrow at a request for the sort of resources that you'd need to seriously audit the code in each new car coming off the line. And, if you don't have the resources to properly evaluate code from a CS or formal verification perspective, empirical black-box testing under real world-ish conditions is about the best you can do.

  4. Welp by Pojut · · Score: 3, Interesting

    Such is the cost of more complicated technology. Although, I will admit, this problem seems awfully widespread for Toyota to have not caught this at some point in their QC/QA process.

    I'm reminded of the "recall" speech in Fight Club...

    1. Re:Welp by Monkeedude1212 · · Score: 4, Funny

      Which car company do work for?

      A major one.

  5. Heads better roll by dave562 · · Score: 4, Funny

    If the statement in the article is true then this country is in even worse shape than I thought. It seems like rarely a handful of months can go by without the realization that yet another Federal department is completely incompetent. How in the hell does the NHTSA even do their job?! They are supposed to ensure that vehicles are safe but they don't even have the staff to do that.

    What the hell is wrong with our country?

    1. Re:Heads better roll by happy_place · · Score: 2, Insightful

      Q. What's wrong with our country? A. The price to make you perfectly safe, six times over, is prohibitively expensive. This seems like a stupid approach to the issue. I mean, just how many engineers need to be hired to make you feel safe? And exactly how do they test all 200 million lines of code? If Toyota's engineers missed something like this, do you honestly think that the government is going to magically find it? It's not like Toyota engineers did this sort of thing on purpose. They made a mistake. It's now costing lives. That's killing Toyota too.

      --
      http://www.beanleafpress.com
    2. Re:Heads better roll by tonywong · · Score: 5, Insightful

      What is wrong is that everyone started believing the mantra that smaller government is better government. This isn't just limited to the United States.

      In Canada, the province where I live (Alberta), derives a major part of its revenues from oil and gas. In the same conservative government 35 years ago, we had 2 independent arms of the government who could determine how much royalties were owed to the government from the oil and gas producers.

      Today, we have no one in our government who is able to determine how much we should be collecting and therefore have to rely upon the oil and gas companies to tell use how much they are supposed to remit. Our own government auditor believes we have been bilked out of billions yet somehow we have a leaner and, ahem, more efficient government.

      Just remember that the only thing to stand up to a big business nowadays is big government, and the goal of any big business is to convince everyone that a small government can watch over big business just like a big government can.

    3. Re:Heads better roll by dave562 · · Score: 2, Insightful

      It has to be deeper than just the President. The NHTSA lacking EE's and SE's is institutionalized fail. They don't even have the talent to meet their mandate. It required a full blown Congressional investigation into dozens of fatalities for someone to stand up and basically say, "By the way, we can't do our job."

    4. Re:Heads better roll by je+ne+sais+quoi · · Score: 5, Interesting

      I don't why I even respond because I'm sure to get a troll mod but I'd just like to point out that one of the major political parties solution to bad government is no government at all. This poorly functioning government is a direct result of the dual conservative mantras: 1) deregulation of markets is necessary for them to perform well and 2) less government is better. We saw how well #1 worked in the banking industry, this is more of the same. #2 results in chronically understaffed government agencies, or government agencies not able to do what they're supposed to do (e.g. the Republican senators holding up Obama's appointees right now).

      My parents both worked for the FDA and if the NHTSA operates in any similar way to the FDA, it's a shadow of itself in the 1970s. For the FDA that means that there are less food inspectors and no surprise, there is a rise in food poisoning incidents. I wouldn't be surprised if NHTSA is also chronically understaffed. Additionally, even if individual government workers wanted to do their jobs, they are often prevented by doing so because that is not perceived as "business friendly". The political appointees who run the show are in the thrall of private industry, in fact, they are often people taken directly from private industry (e.g. big pharma lobbyists often run the FDA). This "government capture" is the fault of the democrats just as much as the republicans, e.g. Obama lied about hiring lobbyists in his campaign. Basically, we have a non-functioning government and one party's answer to this is the get rid of the thing all together. That is one solution but that wouldn't prevent things like this incident with Toyota.

      I'm sure Toyota will do the right thing though, because that would be in its interests as a good corporate citizen. *snicker*

      --
      Gentlemen! You can't fight in here, this is the war room!
    5. Re:Heads better roll by dr2chase · · Score: 4, Insightful

      Big picture, it's not costing that many lives. Bad drivers are much deadlier, and simply sitting on your butt in the car and not getting enough exercise is deadlier yet.

    6. Re:Heads better roll by Areyoukiddingme · · Score: 4, Insightful

      I mean, just how many engineers need to be hired to make you feel safe?

      One would be a good start. Oh hell, let's get wild and crazy and say.. 2.

      Certainly more than zero.

    7. Re:Heads better roll by The+End+Of+Days · · Score: 2, Insightful

      The problem, of course, is that nothing can stand up to big government. That's a tiny problem though, it's not like the government would ever abuse its power to grab control of the citizenry, right?

    8. Re:Heads better roll by wisnoskij · · Score: 4, Insightful

      "How in the hell does the NHTSA even do their job?"
      Like every other safety certification organization. The car companies pay for a certificate, NHTSA takes some of the blame when something happens, and the general population feels safe knowing their is an entire organization dedicated to protecting them.

      --
      Troll is not a replacement for I disagree.
    9. Re:Heads better roll by Chris+Mattern · · Score: 2, Insightful

      Just remember that the only thing to stand up to a big business nowadays is big government

      I'm sorry, but if you think the antidote to big business is big government, you're delusional. Big government is big business's *partner*. It's always been that way, and it'll always be that way. Handing government more power means that there will be plenty of regulations. You *do* know that a regulation-heavy environment favors big business, not small business, right? Small business can't afford the compliance department you need.

    10. Re:Heads better roll by eh2o · · Score: 4, Informative

      Years of deregulation and resource starvation have strangulated our regulatory agencies to the point where they are unable to act.

      Much of this based on Greenspan-style Libertarian philosophies that market forces can correct any problem including fraud and crime, a position which he himself has now renounced and we as a people have yet to heed.

      Since the late 80s we have been riding on a giant ponzi scheme and its all coming crashing down right now. And yet, nothing. I expect things to get much worse.

    11. Re:Heads better roll by rainmayun · · Score: 3, Insightful

      The government's job in this case is not to duplicate the testing done by Toyota engineers, but rather to provide oversight and verify that Toyota's engineers ARE doing it, to a degree of completeness and correctness that satisfies statutes and regulations. Clearly that task requires substantial technical expertise, but it's not the same task.

    12. Re:Heads better roll by roman_mir · · Score: 3, Insightful

      Just remember that the only thing to stand up to a big business nowadays is big government, and the goal of any big business is to convince everyone that a small government can watch over big business just like a big government can.

      - I mean, really? Wake up, is there anyone home? The government that you like so much consists of a system of people, who like to remain in power. To do so takes money. Lots and lots of money. Where do you get the money? It's the system - the bribes real and implied etc.

      Government today is in it with the large corporations. They are one government. In Canada it is a bit different from the US but the principles are the same. Big money wants more money, to do so it needs to corrupt the government and it works on that day and night. Big government wants to stay in power, to do so it needs contributions and various other things money can buy, they do this day and night.

      It's like that Alien vs Predator: no matter which one of them wins, who do you think is going to lose?

    13. Re:Heads better roll by operagost · · Score: 2, Insightful

      Much of this based on Greenspan-style Libertarian philosophies that market forces can correct any problem including fraud and crime, a position which he himself has now renounced and we as a people have yet to heed.

      The Federal Reserve would not exist in a libertarian society. I guess someone modded you up as "funny" because "ignorant" isn't an option. The first and second banks of the US served their functions (until the second became corrupt and President Jackson rightfully killed it), but the Fed is corrupt, enigmatic, and detrimental to the republic. Since FDR, it has allowed the USA to essentially print money at will and rob the people through inflation.

      --

      Gamingmuseum.com: Give your 3D accelerator a rest.
    14. Re:Heads better roll by istartedi · · Score: 4, Insightful

      Wrong. There's a reason Theodore Roosevelt is on Mt. Rushmore. Go back in history and read up. In the USA, we are re-living many of the same issues, roughly 100 years later.

      --
      For all intensive purposes, "whom" is no longer a word. That begs the question, "who cares"?
  6. Computer Engineers needed by HalWasRight · · Score: 4, Insightful

    They don't need Electrial Engineers or Software Engineers. They need Computer Engineers, people who are trained to understand both sides of the hardware/software boundary.

    --
    "This mission is too important to allow you to jeopardize it." -- HAL
    1. Re:Computer Engineers needed by Deltaspectre · · Score: 2, Funny

      Even better, this one only costs $12.99!

      --
      My UID is prime... is yours?
    2. Re:Computer Engineers needed by Anonymous Coward · · Score: 2, Insightful

      Speaking as someone with a CMPE degree, employers see me as under-qualified to do EE work and over qualified to do programming work. What they need is either EEs with heavy embedded programming experience or software engineers with (guess what) embedded programming experience. The title isn't that important.

  7. 100 million lines of code?? by Anonymous Coward · · Score: 2, Insightful

    I find that extremely hard to believe. Jurassic Park ran on just two million lines of code. I doubt all the lifetime output of all the readers of this thread, combined, equals 100 million. I further doubt that such complexity is remotely necessary to run a car, and that it is remotely possible to debug that much complexity to the standards of, say, the airline industry. And that NHTSA could audit that code in any respectable amount of time. I hope beyond hope the number is wrong.

    1. Re:100 million lines of code?? by quantumplacet · · Score: 4, Informative
    2. Re:100 million lines of code?? by saccade.com · · Score: 3, Informative

      I strongly suspect the "100 million lines of code" is BS. Most of the "ECUs" are small microcontrollers that would be lucky to hold 5,000 lines of code, let alone millions. Either the professor is inflating the code size estimate to make himself seem important, or the systems are designed by complete idiots.

  8. 100 million lines? Sure, we will get right on it by jeffmeden · · Score: 4, Insightful

    What exactly would the NHTSA do with a set of engineers? Audit all 100 million lines of code for each and every car they suspect has a safety issue with the computer system? Yeah, that sounds like a worthwhile endeavor. How about they do it the old fashioned way; collect the reports, identify the risk, and sanction the manufacturer to find/fix the problem. Thinking that an NHTSA coder (or a hundred) would have gotten to the bottom of this Toyota issue in any reasonable amount of time is a joke!

  9. How many microprocessors was that again? by jdgoulden · · Score: 2, Insightful

    70 to 100 microprocessors? I imagine that this is true only if you employ a fairly broad definition of "microprocessor" and note that the vast majority are single-purpose devices in self-contained systems. I doubt that the "microprocessors" and "lines of code" that run the stereo or the climate-control system - or even the airbags - have any connection with the driveline.

  10. Microsoft Hotline by imscarr · · Score: 2, Funny

    Can't they just call Microsoft's toll-free number and ask someone over there to look at it?

    --
    Like the beaver, it's just Dam one thing after another
  11. Re:Huh! by megamerican · · Score: 4, Interesting

    If the NHTSA didn't exist Toyota would have had to spend money to fix the problem instead of paying ex-regulators to quash multiple investigations.

    Toyota (TM) hired ex-government regulators to kill at least four investigations into problems with its cars in the U.S. That's the conclusion of an investigation by Bloomberg. The news service reports that, "Christopher Tinto, vice president of regulatory affairs in Toyota's Washington office, and Christopher Santucci, who works for Tinto, helped persuade the National Highway Traffic Safety Administration to end probes including those of 2002-2003 Toyota Camrys and Solaras, court documents show. Both men joined Toyota directly from NHTSA, Tinto in 1994 and Santucci in 2003. "

    The same goes for Wall Street. Most of the financial regulators are former high level executives from Goldman Sachs or strong ties to them and other financial institutions.

    I don't understand why we need so many useless regulators who are usually wolves being put in charge of the hen house when the courts could easily handle this. It's going to end up being prosecuted in a court of law anyway and not solved by some magic regulation hand-waving.

    --
    If you have something that you dont want anyone to know, maybe you shouldnt be doing it in the first place -Eric Schmidt
  12. Re:This all story starts to look like swine flu by binarylarry · · Score: 4, Interesting

    http://abcnews.go.com/Blotter/RunawayToyotas/toyota-acceleration-problems-new-evidence-imprisoned-minnesota-toyota-camry-owner/story?id=9903455

    This guy apparently killed a few people and got put in jail for it. Now it looks like he was telling the truth when he said the car wouldn't stop.

    --
    Mod me down, my New Earth Global Warmingist friends!
  13. This is the government, not an engineering firm by rm999 · · Score: 2, Insightful

    I totally disagree: the NHTSA shouldn't hire engineers. NHTSA should not do the job of Toyota's engineers and testers; they were created to set policy and propose safety laws. The NHTSA should hire economists, policy makers, and maybe some scientists. But the job of ensuring the nuts and bolts of a car are safe should fall on the car-maker, with strict repercussions if they fail.

    My biggest problem with all this is what people on Slashdot should already know: looking through and understanding millions of lines of code would take an engineer a few lifetimes - how many engineers are we proposing NHTSA hires? They could learn Toyota's software system, but then what about Ford cars? Or BMW? All for a government organization with 600 employees...

    In cases like this, NHTSA should force Toyota to hire a third party (objective) consultant to create a technical report. Maybe a small team of engineers could remain on staff to read and understand those reports.

    1. Re:This is the government, not an engineering firm by kidgenius · · Score: 2, Interesting

      Why not? The FAA hires engineers. With the way cars are going, I am scared to think of how much computer control is being done (drive by wire, brake by wire, etc), with little to no oversight from an regulatory agency ensuring the safety of the cars. I work in aerospace and my boss is an FAA DER. The amount of safety review done on an airplane is insane. I think that at least some of that analysis should be applied to cars, now that we are giving up so much of the control in the vehicles to them. someone quoted DO-178B for cars...not necessarily a bad idea.

    2. Re:This is the government, not an engineering firm by rahvin112 · · Score: 3, Insightful

      Why not simply require that any software in an automobile be OSS (not FOSS). In fact that requirement should seem to be an extension of mechanic laws that required car makers to provide parts and knowledge to service vehicles outside dealerships. All software in such a critical item should be OSS so it can be reviewed for errors and be reprogrammed by mechanics who wish to offer such services.

  14. Re:List of software powered cars by Chyeld · · Score: 4, Informative

    Anything street legal without a needing a special waiver for emissions.

  15. Re:100 million lines of code? by mcgrew · · Score: 2, Interesting

    I've seen the comment about a modern car having something like 100 million lines of code in articles before. Now, I am not in any way qualified to say that number is to large or to small. But as an embedded systems software developer, that seems like an INSANE amount of code.

    Someone posted a link to this article that confirms it. I can't find the comment with the link; someone must have modded him down past my threshhold. But the article linked itself confirms that it is indeed an insane amount of code, insanely implimented.

    The avionics system in the F-22 Raptor, the current U.S. Air Force frontline jet fighter, consists of about 1.7 million lines of software code. The F-35 Joint Strike Fighter, scheduled to become operational in 2010, will require about 5.7 million lines of code to operate its onboard systems. And Boeing's new 787 Dreamliner, scheduled to be delivered to customers in 2010, requires about 6.5 million lines of software code to operate its avionics and onboard support systems.

    These are impressive amounts of software, yet if you bought a premium-class automobile recently, "it probably contains close to 100 million lines of software code," says Manfred Broy, a professor of informatics at Technical University, Munich, and a leading expert on software in cars. All that software executes on 70 to 100 microprocessor-based electronic control units (ECUs) networked throughout the body of your car.

    It gets worse.

    And unlike most commercial aircraft, which have strict firewalls between critical avionic systems and the in-flight entertainment systems, there is more commingling of information between the electronic systems used to operate the car and those for entertaining the driver and passengers. According to a Wharton Business School article entitled "Car Trouble: Should We Recall the U.S. Auto Industry?," a few years ago, some Mercedes drivers found that their seats moved if they pushed a certain button; the problem was that the button was supposed to operate the navigation system.

  16. Re:List of software powered cars by ircmaxell · · Score: 4, Informative

    Go to a car dealer. Look. Every car sold since 1996 (At least in the US, and I assume the rest of the world) today has at least an ECM (Engine Control Module) which is just a fancy name for a computer controlling the engine. That's what the government mandated OBD-2 program was (OBD == On Board Diagnostics). The number of cars that are completely computer controlled (drive by wire) is far lower, but higher than you'd think.

    I had an '05 Chevy Cobalt that had "computer assisted" electromechanical power steering. Basically, what I found out from the dealer after the computer controlling it failed (and I lost all power steering) is that the computer (BCM, Body Control Module) takes inputs from the ABS system, Traction control (if equipped), speedometer, accelerometers and about a dozen other sensors and computes the way it thinks you want to be steering. Then it provides an "intelligent" boost in that direction. I must say, it worked really well in the snow and when fishtailing (it made if VERY difficult to over-correct and put it into a spin). But when it failed, I'd be in the middle of a curve on the highway when all power steering went out... Luckily they were smart enough to put a kill switch in to prevent it from coming back on while the car was moving (I could just imagine struggling through a corner when all of a sudden it came back)... It turns out that it was a software issue in the first place (they updated the software, and it never happened again). I got rid of the car a few years later for other, more significant reasons...

    The benefits of computer control are good, but there needs to be intelligent fail-safes put in place to prevent disaster when something does go wrong (not if, when)...

    --
    If a man isn't willing to take some risk for his opinions, either his opinions are no good or he's no good
  17. Re:100 million lines of code? by 0100010001010011 · · Score: 3, Interesting

    Most of that code is auto generated. Except for some low level stuff, nothing is written by hand in assembly or C. It's all auto coded from some sort of control toolbox. Most likely Matlab/Simulink.

    Sure enough this is one of the first hits on Google.

    Writing that many lines of code would be damn near impossible in the relatively short development cycle.

    Even a simple PID controller could take up a few dozen lines of code even though on screen it's simply represented by 3-4 blocks.

  18. Re:100 million lines? Sure, we will get right on i by bhtooefr · · Score: 5, Interesting

    And they said in a modern luxury car.

    So that's all the code in the following computers:

    Engine (controls throttle and such)
    Transmission
    Collision avoidance (ABS, traction control, etc. TPMS is usually here, too, because it's sometimes part of the ABS system to save costs)
    Safety (airbags, seatbelt pretensioners, etc.)
    Central convenience (security system, power locks, power windows, cabin illumination, in some cars even the exterior lighting goes through central convenience)
    HVAC
    Instrumentation (yep, there's a computer dedicated to that - and some security functions are sometimes in there)
    Entertainment (navigation, stereo, DVD, etc., etc.)

    And all these systems are interconnected.

    You get in your car (central convenience deactivates security upon receiving the signal, and when you open the door, it illuminates the cabin, alerts the engine computer that a start is imminent, possibly starting fuel pumps, on diesel cars turning on the glow plugs, etc., etc., and notifies the instrument cluster that the door is ajar.)

    You insert your key into the ignition (yes, I know about push-button start,) and start the engine (engine computer starts up, after which the instrument cluster polls the RFID chip on the key. If it can't get a read, it immediately requests that the engine computer shut down.)

    You decide that you want a little heat before you set off, so you use your steering wheel controls (which go through instrumentation) to set HVAC settings, and then you figure some music won't hurt (entertainment.) Then, you remember that you don't know where you're going, so you punch the address into the navigation system, and it feeds directions back to the instrument cluster.

    Now, you put the car into gear. The transmission computer notifies the other computers about this, and the engine computer adjusts the idle fueling to compensate. The instrument computer reflects the gear change. The central convenience module turns on the daytime running lights. The entertainment system might prevent you from using the touchscreen interface. The safety computer may become more persistent about reminding you that you didn't put on your seat belt, and will notify the instrument cluster of this, to annoy you more.

    After you put your seatbelt on, you let off the brake and pull out of your parking space. Obviously, the engine computer and transmission computer are working together here, the instrument cluster is constantly updating the status of those (and the entertainment computer, which is noting the changes in vehicle position.) After you hit 10 MPH, the engine or transmission computer sends a request to the central convenience module to lock the doors.

    Now, you're going down the freeway, and right in front of you, a semi truck loses control, and flips onto its side. You jam on the brakes, which kills engine power immediately (engine computer, and the transmission computer is affected as well, and this all gets fed back to the instrument computer.) Collision avoidance computer activates ABS and (as you're attempting to swerve out of the way) stability control, and notifies the central convenience computer that you're undergoing a panic stop, and to activate the hazards.

    Unfortunately, you don't have enough time and room to stop, and you hit the semi. The safety computer notices this, and fires the seatbelt pretensioners and the appropriate airbags. Once that's done, there's some less immediate concerns. It would be a bad idea to leave the engine running, so the safety computer requests an engine shutdown. The transmission computer may be requested to shift to neutral, to make moving the wreck easier. The entertainment system will be told to stop playing music, and if it's got a system like OnStar (which used to be yet another TWO separate computers off of the entertainment system,) an emergency call initiated. Instrumentation is of course updating the status of all of this. HVAC may be set to off. The collision avoidance computer will still be trying to keep t

  19. unlikely, given most networks are separated by SuperBanana · · Score: 5, Interesting

    Here comes DO-178B for cars.

    The vehicle drivetrain network is very often, if not always, separate from the "entertainment" network; Audi, for example, runs two separate CAN busses for them. The original story hypes things a bit; there may be 70-100 microCONTROLLERS, but half or more of them are "body" (ie windows, sunroof, etc) or "entertainment"(audio, navigation) related and thus don't really need to be reviewed.

    The vast majority of them do very, very simple things, mostly sending CAN bus messages or responding to CAN bus commands. Ie, you move the wiper stalk. The microcontroller for the steering wheel controls says "the stalk moved" either to the wiper motor interface or a 'body control' computer, which then sends a command to the wipers.

    The code review for most of the modules, as a result, is extremely simple- they're just (mostly digital) I/O boxes. Some of them are things like fuel pump modules, which at most have some diagnostic capabilities (like current draw from the pump, pressure sensor, etc.)

    The code review will not be very problematic for engine computers, because (gasp!) they're not made by car manufacturers. Bosch, Magnetti Marelli, Hitachi, and a couple of other companies are the primary producers. And guess what? The code is largely the same car-to-car. Parameters are changed- code doesn't, so much. And car companies share "platforms", which further simplifies things.

    It's not nearly as scary as it sounds.

    1. Re:unlikely, given most networks are separated by aaarrrgggh · · Score: 2, Informative

      While there is truth in what you are saying on complexity, as someone who has invested a lot of time understanding why Bosch has some fuel pumps failing in a non-passive fashion on stationary engines... there are a lot of assumptions built in, and many problems are only found by trial and error.

  20. GS by zogger · · Score: 2, Informative

    Clinton signed the law repealing glass steagall. Whether a veto by him would have been overturned is moot, he still signed the thing. They should have called it the "let wall street and the casino bank hustlers go crackhead apeshit with your money" act. That's one of the biggees, not the only, but one, of the reasons we are in an economic mess now.

    I'm a small government guy by nature, but some regulations are always in order. Pure anarchy market forces lead to monopolies and cartels, and that's about it. Because predatory crooks rise to the top levels of giving orders.. and that's business and ggovernment, both.

      That's why there needs to be oversight, and why we need more pure government "kick em all out!" efforts occasionally, and why we need but don't have yet "corporate death penalities". The crooks eventually take over, it always happens, not much you can do to prevent it, so all you can do is slow them down a little. And even then, with oversight and slowing them down, they eventually get firmly entrenched at all the order giving levels, so you have no choice other than starting over again from scratch. Very broadly historically speaking of course.

  21. advice for anyone with a runaway gas pedal by Kargan · · Score: 3, Funny

    Shift into neutral. I haven't seen this anywhere as part of the many Toyota-related discussions around the world, so figured I'd mention it.

    --
    Palaces, barricades, threats, meet promises
    1. Re:advice for anyone with a runaway gas pedal by Cassini2 · · Score: 2, Interesting

      There are two major problems with the "shift to neutral" solution:
      1. It doesn't always work.
      2. Only a few auto-mechanic and maybe some race car drives have the reflex to shift the car into neutral.

      Most people will not think of shifting to neutral when a problem is encountered, simply because they never need to do it. I'm an engineer, and if my car takes off, it will take me a while to think of shifting to neutral. A car at full acceleration can cover much ground in less than 1 second.

      The other problem is that I doubt that auto-transmissions will consistently disengage under *fault* conditions.

      Your best chance is to be driving a manual transmission. Every manual transmission driver knows to hit the clutch and brakes at the same time, and will do it instinctively. Additionally, the manual transmission is less vulnerable to simultaneous failure modes than the modern computer controlled automatic transmission. For instance, if you are high gear in a manual transmission, it won't automatically down-shift to apply more torque to the wheels when you brake the car to slow it down. Additionally, if the manual transmission is in low gear, it won't up-shift automatically if the car engine takes off. The engine may rev-high, but in low gear, at least you won't be going fast. The manual transmission is much safer in runaway engine conditions.

  22. Not news to me by VGR · · Score: 2, Interesting

    I can't say I find this surprising. Anyone who has ever worked on software for a US government contractor, or US military contractor, knows the government/military has no one who can analyze the product they pay for. Nearly every software product I've seen delivered is of absurdly poor quality. It would be laughable if the implications of the software's use weren't so disturbing.

    --
    The Internet is full. Go away.
  23. Re:This all story starts to look like swine flu by Anonymous Coward · · Score: 2, Interesting

    Wow, if this is true, then Toyota with-held evidence that could have kept an innocent man out of jail...

  24. Why? by Chicken_Kickers · · Score: 2, Insightful

    Why the need to over complicate a relatively simple mechanical construct that is the car? The old adage still hold true: if it ain't broke, don't fix it. Modern fighter jets are purposely designed to be unstable for manoeuvrability or due to the effects of stealthy design and thus requires fly-by-wire capability. Cars don't need such complexity. Why would I need my steering wheel to be mechanically decoupled from the wheels or my brake pedals to the actual brake discs? This introduces more intermediate steps in the process and therefore increases the chances of failure somewhere along the line. The previous hydraulic systems worked just fine and gives fairly instantaneous feedback. What's more, you couldn't tinker with the car yourself any more and have to send them to expensive specialist mechanics. This is all just an unhealthy infatuation with technology and shoe horning them where they are not needed.

  25. Fire all the bridge and tunnel inspectors, too by FranTaylor · · Score: 2, Insightful

    If you think that the government should not get involved in engineering.

  26. Don't reinvent the wheel by Tisha_AH · · Score: 2, Insightful

    The NHTSA does not need to evolve a new set of standards out there to address part of this problem. Just require that all automobiles meet the FCC Part 15, Class B standards for electromagnetic susceptibility. It is stupid that this is not done already.

    There are plenty of critical pieces of equipment that cannot turn up their noses and fail because of electromagnetic interference. Medical equipment is tested to at least this standard every day. There are hundreds of testing laboratories throughout the world who manufacture products that have to meet these specifications. There are thousands of engineers who already do this type of testing.

    Now lines of code and software is a different animal. In a hundred million lines of code there are certainly bugs and flaws.

    --
    Tisha Hayes
  27. Re:List of software powered cars by YrWrstNtmr · · Score: 4, Informative

    1. A car designed for manual steering is quite different than one designed for power steering.
    2. There is a wide range of speed and turn radius conditions between straight freeway and parking lot.

  28. Re:List of software powered cars by Monkey_Genius · · Score: 2, Informative

    ...you will never struggle with turning anything at freeway speed.

    Unless of course the hydraulic assist from the power steering pump is lost due to a pump failure or broken belt -essentially the same thing. Then the steering becomes very difficult as you have to supply the 'power' necessary to force the hydraulic fluid through the steering gear and the failed pump. This is also made more difficult as most power assisted steering has a higher ratio -fewer number of turns lock-to-lock- than a manual (non-power assisted) steering gear.

    --
    I've got your sig, right here.
  29. Re:List of software powered cars by CapnStank · · Score: 2, Informative

    A professor in my first year of university told me something that has stuck with me for years:

    "You can never design a product that will never fail. Whether it is your incompetents or someone else's the product will fail. As an engineer it is your duty to provide fail safes as to not cause any bodily harm to the user or others."

    I still wonder where the engineers where who saw the flaws in the system two years ago. I don't believe that this 'software' issue went unnoticed for THAT long.

  30. Regulation != Bad by sjbe · · Score: 4, Insightful

    Most of the financial regulators are former high level executives from Goldman Sachs...

    Some are but most are demonstrably not. Many are financial industry insiders but that's by necessity. Do you really want an financial regulator who has no knowledge of the industry he/she is regulating? The only place to get people with the appropriate financial experience is from the finance industry.

    I don't understand why we need so many useless regulators who are usually wolves being put in charge of the hen house when the courts could easily handle this.

    While I admire your faith in the court system, in truth the courts are woefully ill-prepared to deal with the sorts of issues the SEC and other regulating bodies deal with. The court system is sloooooowww, expensive and can only effectively deal with misconduct after it has occurred. The courts are a poor monitoring system. The court system also is not heavily staffed with financial experts who understand the issues involved. Trust me, you REALLY don't want financially illiterate judges deciding financial regulations.

    The reason the industry insiders often end up as regulators is precisely because they are the only ones who really understand what is going on. Finance is really, really complicated. Yes it's not perfect but that's why the regulators are accountable to other bodies including the President and Congress. If anything the problem with the regulators isn't (usually) that they do poor quality work but rather that they aren't given enough resources to really do a great job. The SEC for instance is badly understaffed given it's mandate. If you really want to keep a better watch on the finance industry, lobby congress to increase funding to the SEC and other watchdog agencies.

    It's going to end up being prosecuted in a court of law anyway and not solved by some magic regulation hand-waving.

    Spoken like someone who has no experience whatsoever in the financial industry. I won't argue that all regulations are good or well enforced but relying on the court system alone to solve the issues that regulators deal with daily would be insanity. If you really want to screw up the financial system, get rid of the regulators. Our current financial mess is due in significant part to a lack of regulation.

  31. Borrow some from the FDA by AmericanGladiator · · Score: 2, Insightful

    Surely they have some with all the safety-critical code (e.g. from pacemakers) that must pass through their review process.

  32. It's time... by GrahamCox · · Score: 2, Insightful

    It's time:

    a) for a global safety-critical standard for drive-by-wire software.
    b) for an open industry standard for interfacing for servicing, fault codes, etc, to end the scam of lock-in to specific manufacturers servicing tools and dealers.
    c) to open source it.

  33. 100 MLOC? They should have used Lisp. by istartedi · · Score: 2, Funny

    The car function is built in.

    --
    For all intensive purposes, "whom" is no longer a word. That begs the question, "who cares"?
  34. 100 million lines of code on 70 processors? by goffster · · Score: 2, Interesting

    I would be more interested in the process of how
    Toyota develops/maintains code. Do they rewrite code for every car?
    When they reuse code, how do they retest assertions?
    How do they do code verification?
    What is their culture when coding problems interfere w/deadlines ?
    Is there a whole crap load of unused code in there because
    they are scared shitless to remove it ?

    etc.

  35. Re:100 million lines? Sure, we will get right on i by ColaMan · · Score: 3, Insightful

    It would be a pretty crappy car if it engaged the seat belt PREtensioners POST-impact.

    Pretensioners are fired after the initial contact, whilst the very front of the vehicle is still crumpling away. How the hell do you think the computer knows that it has hit something otherwise? Radar? Not on your $10K cheapo. Magic? No, a little ball + spring combo live underneath your front bumper and the last thing they tell the vehicle before they are crushed in an accident is "something big is heading your way".

    I'd also rather it didn't "kill engine power" every time I hit the brakes.

    We're not just talking about 'every time', we're talking about the two-feet-on-the-brake-pedal-jesus-christ-I-want-to-stop-NOW kind of braking that will activate ABS. Once ABS (and it's cousin, stability control) are running the show, engine power can (and will) be modulated as they see fit in attempt to keep the vehicle going where you want it to go. If you think you can simultaneously control brake force and engine power separately to each wheel whilst in an emergency to do the same, than you go right ahead. I'll take the bus.

    although "traction control" systems might retard timing if severe wheel slip is detected.

    Traction control is a lot smarter than you seem to think now, and retarding timing went out of fashion about 15 years ago. Now if the traction control system wants less power it simply requests the engine computer to reduce power output by X percent and the engine computer will choose between:
      - Simply closing the throttle body, if it has control of it.
      - Killing fuel injection on a few cylinders to drop power.
      - Dropping boost if it's a turbo'd vehicle.
      - Cutting (or yes, retarding) ignition. Bit of a last resort due to unburnt fuel getting out the other side of the engine.

    And what $20,000 compact automatically turns on hazard blinkers, mutes the stereo, and opens windows?

    My Peugoet 307 turned on the hazards and muted the music if you hit the brakes hard enough to activate its electronic brake force assist system. I did it a couple of times in the two years I had the car, but never got into a collision to find out about the windows.

    --

    You are in a twisty maze of processor lines, all alike.
    There is a lot of hype here.
  36. Legitimate checks to power by copponex · · Score: 5, Insightful

    The government doesn't have to do anything complicated. It just has to have the ability to strike fear into the hearts of the business community it's supposed to regulate.

    This requires a few things: an independent media, which we don't have; a civically informed populace that takes it's democratic duties seriously, which we don't have; and a culture that values human dignity over profits, which we don't have.

    In cultures that do have all of these things, government regulation works very well and fosters progress, since you don't have to constantly worry about getting screwed over, you don't have to wonder if you'll have access to medical care, or a good public school, or a good safety net to get you back on your feet if your fall ill, get in an accident, or whatever.

    Clear and concise regulation with real penalties for breaking those regulations fosters competitive markets. Diminishing the government to the point where it can be bought and sold by businesses usually leads to fascism. The markets destroy themselves with greed, destabilize the economy (and eventually the whole society), and further concentrate wealth and power until you have a virtual oligarchy sprinkled with political theater.

  37. Re:Huh! by Eravnrekaree · · Score: 2, Insightful

    Regulations are to prevent safety problems in the first place. The reason why regulation is not that regulation *never works*, but instead the regulators are not independant, there are too many conflicts of interests, and it will be the case until we implement some real compaign finance reform so that politicians are not helped to be elected by corporations through all of their generous donations. The fact is, without regulations, matters would not be any better, in fact they would likely be worse, as you want. When we have broken regulators we do not get rid of all regulation but fix the problem as to why its broken. We need more independant experts in regulatory agencies, they need to be run by people never employed by corporations and not paid or influenced by them in any way.

  38. More needed than just source code review by MillenneumMan · · Score: 2, Insightful
    I used to write software for the US Dept of Defense, and our office had a fairly good sized team that all day every day manually compared expected results to actual results when compiling our programs. I was amazed at how frequently that team uncovered errors. Most of the time they found subtle errors in how the compiler program performed its translations, but it was not unusual for them to find logic errors embedded in the computer chips themselves. All of these things had to be corrected, even it if meant re-engineering a computer chip, before our software could be deployed, and for obvious reasons: you cannot allow a weapon to fire due to a computer error.

    This drive-by-wire stuff is very serious. I seriously doubt that any car manufacturer validates their computer software and hardware as rigorously as the Dept of Defense; in fact they probably don't do compiler or chip logic validation at all. I bet the aviation industry could give them guidance in this arena.

  39. Drive-by-wire has been around for a while. by cvtan · · Score: 2, Informative

    BMW has had drive by wire throttles in production since 1988 750iL V12. Slowly migrated down to cheaper models over the years. Not much in the way of serious problems. Stepper motors running the throttle can fail, but this is more of an annoying expense than a safety disaster. My MINI Cooper has drive by wire and works fine. Makes it easy to implement cruise control and traction control. Throttle control is by dual redundant pots that "vote" on throttle opening. If something acts screwy, it goes into limp-home mode. The only throttle control problems I've had were with cars with mechanical linkage that got bound up from rust/old age.

    --
    Sorry, but gray text on gray background is making my eyes bleed.
  40. Standards, not analysis. by rickb928 · · Score: 3, Interesting

    And safety, not peformance.

    Instead of testng code, evaluating the design process, pretending the NHTSA can even begin to become expert in software design, how about applying the old standards to the new systems?

    For instance, braking safety. I was listening to and reading the testimony from Rhonda Smith, where she even describes shifting her Lexus into neutral. Neutral?

    A simple test, and I'm not an engineer, but shouldn't a car come to a stop with 'maximum' brake effort, despite the acclerator position? This is solvable in software - if the brakes are going into lock, and ABS is engaged, engine power and/or transmission state have to be compelled to answer the driver's command to stop. Traction control is already being used in many cars; NHTSA should be able to make a test capable of verifying that even multiple malfunctions are overcome.

    Crap, my wife's 1995 Saab 900SE has a mode where the ECU shuts down the fuel pump if the engine stops running, on the assumption that something is terribly wrong, and spewing gas to a stopped engine is pointless if not dangerous. How do I know this? Her car developed a habit of stalling at stops. The real cause was a defective vapor recovery canister, causing loss of vacuum and low RPMs, and the ECU saw that as a stopped engine and made sure it stopped.

    Certainly there are other states that can be tested for performance and safety, not some quality of performance standard. Most cars have 'safe' or 'cripple' modes to protect the drivetrain if something seems wrong, like the transmission in a gear that should not permit the indicated speed. My '95 Explorer does that, and it's only an OBD-I system. Acclerator position, wheel speed, and transmission mode should all correlate, and if something is wrong the system needs to cripple - slow down, set a max speed, etc.

    Aircraft flight control systems are held out as an example of safety and reliability. Most of these, if not all, have to at least ensure the aircraft doesn't exceed the flight envelope and exceed safety limits. This is the sort standard and evaluation the NHTSA needs to focus on.

    Maybe NHTSA needs to borrow a few investigators from the FAA and the military? They should be looking to Boeing, McDonnell, Electric Boat, General Dynamics for expertise in verifying safety in vehicles. Maybe even some NASA people. At least NASA seems to have turned the Shuttle program around a little too late. They certainly have a cautionary tale to tell, and a jaundiced eye towards the assurances of the 'experts' and trusting management.

    Which would go a long way to reinstating a somewhat adversarial relationship between the regulators and the industry. There should be some tension there. Hiring your industry's former employees is not the way to go.

    We can do so much better. We just need to solve the real problems.

    --
    deleting the extra space after periods so i can stay relevant, yeah.
  41. "Resource starvation"? by PapayaSF · · Score: 3, Interesting

    Years of deregulation and resource starvation have strangulated our regulatory agencies

    Here's some recent data about the resources available to the DoT, the parent agency of the NHTSA: When the recession started, the Transportation Department had only one person earning a salary of $170,000 or more. Eighteen months later, 1,690 employees had salaries above $170,000. Plus the juicy benefits and pension plan. I'll bet all those managers and supervisors raking in the big bucks would agree that their agencies are "resource starved" and that if they only had more money and more power, they could hire two or three software engineers (for the cost of one manager).

    --
    Q: What does the "B." in Benoit B. Mandelbrot stand for? A: Benoit B. Mandelbrot
  42. I just HAVE to ask this question... by cyberjock1980 · · Score: 2, Insightful

    Bear with me for a second here...

    The three laws of robotics:

    1. A robot may not injure a human being or, through inaction, allow a human being to come to harm.
    2. A robot must obey any orders given to it by human beings, except where such orders would conflict with the First Law.
    3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

    I know that a car is not a robot. But the same rules should apply for ANY computer system that, in case of a serious bug, could result in any of those 3 laws being broken.

    This computer literally controls a rather large piece of metal that can travel at speeds sufficient to kill someone. So why is there no subroutine that ensure that brake pedal input will ALWAYS override the gas pedal input? It seems that even on the absolute most basic of level, adding this extremely basic concept could seriously mitigate these issues. Not to mention all of the legal responsibilities, public outcry, and other consequences of not having software or hardware with these "basic" concepts built in.

    Even when making a car and using this system on a test site somewhere. Wouldn't you want to have LOADS of extra code in there to make sure a bug in the software doesn't kill the driver at the test site? It seems to me Toyota's definition of "safety" is practically non-existent.

    Honestly, when seeing something like this, I have to question what kind of work ethic Toyota has and how much they value me as a customer.

  43. Throttle software causing sudden acceleration? by L4t3r4lu5 · · Score: 4, Insightful

    It's not a race condition, is it?

    I can't believe I'm the first one on this thread to make that joke. I'm not even a programmer.

    You should all be ashamed of yourselves.

    --
    Finally had enough. Come see us over at https://soylentnews.org/