Elon Musk Rolled Out Autopilot Despite Engineers' Safety Concerns, Says Report (theverge.com)

An anonymous reader quotes a report from The Verge: When Elon Musk announced last fall that all of Tesla's cars would be capable of "full autonomy," engineers who were working on the suite of self-driving features, known as Autopilot, did not believe the system was ready to safely control a car, according to the Wall Street Journal. The WSJ report sheds more light on the tension that exists between the Autopilot team and Musk. CNN previously reported in July that Musk "brushed aside certain concerns as negligible compared to Autopilot's overall lifesaving potential," and that employees who worked on Autopilot "struggled" to make the same reconciliation.

A major cause of this conflict has apparently been the way Musk chose to market Autopilot. The decision to refer to Autopilot as a "full self-driving" solution -- language that makes multiple appearances on the company's website, especially during the process of ordering a car -- was the spark for multiple departures, including Sterling Anderson, who was in charge of the Autopilot team during last year's announcement. Anderson left the company two months later, and was hit with a lawsuit from Tesla that alleged breach of contract, employee poaching, and theft of data related to Autopilot, though the suit was eventually settled. A year before that, a lead engineer warned the company that Autopilot wasn't ready to be released shortly before the original rollout. Evan Nakano, the senior system design and architecture engineer at the time, wrote that development of Autopilot was based on "reckless decision making that has potentially put customer lives at risk," according to documents obtained by the WSJ.

Comments

Not surprised

[Anonymous Coward]

After that one poor guy was denogginized through no fault of his own, we now find out apparently the autopilot was not ready for prime time? Wow. Heads should roll over this.

Re:Not surprised

[Mr D from 63]

Five magic words make this controversy go away;

Its just like plane autopilot.

Elon, you are off the hook.

Re:Not surprised

[K. S. Kyosuke]

through no fault of his own

Are you referring to that idiotic youtuber? Through what leap of logic does one conclude that he wasn't at fault?

Re:Not surprised

[cheesybagel]

There's been more people than those mentioned here who left Tesla. Chris Lattner is one.

Re:Not surprised

[Anonymous Coward]

His " full self-driving car" (by Musk's own admission) drove into the side of an 18 wheeler in the middle of the day with perfect weather conditions.

Seems pretty clear to me.

Re:Not surprised

[sexconker]

If it doesn't autonomously pilot the vehicle, it's not autopilot and should not be called such.

And autopilot for planes is far, far, far more advanced, capable, robust, and reliable than the shit Elon is selling. The people using it are explicitly trained on what it can and can't do. People buying Teslas only get the marketing speak shoved in their face.

Re:Not surprised

[AmiMoJo]

That's hardly the only time. There was an incidental in China where autopilot drove full speed into a road sweeper that it apparently couldn't see.

Tesla seem to have admitted it doesn't work as originally advertised, by repeatedly increasing the amount of effort it makes to keep the driver alert.

Re:Not surprised

[alvinrod]

Better yet don't call it autopilot. Even if you try to explain it, there are plenty of fools who won't get it. Call it "drive assist" and people might be a little less foolish. Some assholes will still misuse it, but you can't stop someone hellbent on stupidity.

Re:Not surprised

[Spy Handler]

Except those words are false. It is NOT just like plane autopilot. Your Tesla on autopilot going down highway 101 is in a vastly different environment than an airplane flying on autopilot in the middle of the Pacific ocean at 30,000 feet altitude.

Plane autopilot is safe because it is well understood by all parties that it is not to be engaged in an environment containing tractor-trailers and pedestrians in your path. Telling people Tesla autopilot is just like plane autopilot is dangerous snake oilmanship.

Re:Not surprised

[Anonymous Coward]

If it doesn't autonomously pilot the vehicle, it's not autopilot and should not be called such.

And autopilot for planes is far, far, far more advanced, capable, robust, and reliable than the shit Elon is selling. The people using it are explicitly trained on what it can and can't do. People buying Teslas only get the marketing speak shoved in their face.

When any autopilot specifically informs a human driver to put their hands on the wheel and pay attention, and the fucking idiot behind the wheel ignores that, I'd say it's pretty damn obvious who and what is at fault.

Airline pilots are intelligent and highly trained individuals. That is why they are not found on every street corner, and are worth more than a dime a dozen. The main mistake Elon made was assuming that the average driver wasn't a fucking idiot.

Re:Not surprised

[K. S. Kyosuke]

No, he was instructed not to do what he did. And he did it anyway. Seems pretty clear to me.

Re:Not surprised

[Mr D from 63]

I was being sarcastic. I guess I'd better go ahead and state that now

Re:Not surprised

[whoever57]

"Is it safe?" is the wrong question.

The real question is: "are the fatalities, injuries and accidents that occur per passenger mile greater or less when compared to a human driver?"

"Safe" is a subjective and unmeasurable term, unless you define it in unreasonable terms (for example if your definition of safe is zero accidents).

Re:Not surprised

[K. S. Kyosuke]

And autopilot for planes is far, far, far more advanced, capable, robust, and reliable

That's mostly because it is also much simpler. The comparison with it is correct, it just omits the part where you have to additionally recognize roads, vehicles, traffic signs, pedestrians...all the things airplanes don't have to worry about. Airplane-level capability just isn't enough.

Re:Not surprised

[burtosis]

When the car can drive in snow, heavy rain, fog, or around road construction let me know. As it is you can't even compare autonomous cars to people it's like gangsters to space stations as the only miles on self driving cars are sunny freeway miles and carefully planned city routes. No study I've seen compared them in the same situations.

Re:Not surprised

[Anonymous Coward]

If it doesn't autonomously pilot the vehicle, it's not autopilot and should not be called such.

It autonomously pilots the vehicle to about the same extent an airplane autopilot does. In neither case is the pilot or driver free to stop paying attention. The pilot is expected to monitor the behavior of the autopilot, which follows simple instructions such as, "fly a track to this radial to this VOR", or "climb to FL30 and level off, increasing speed to mach 0.81". It can chain those together and fly a preprogrammed route, but no matter what, the pilot must maintain situational awareness. Autopilots can and do fail, or develop problems such as entering a descent when level flight is commanded. Autopilots can and will disengage if something happens they aren't prepared to deal with.

Autopilot here means the roughly what it means in airplane land, which is why the name is tolerably apt. "It will help you with the drudgery of steering the vehicle around, but you the human must pay attention to what's happening and be prepared to intervene".

Re:Not surprised

[Rei]

It's a lot simpler than that:

This article is bullshit.

Sorry to be so blunt, but it's journalistic malpractice. The author is confusing Enhanced Autopilot (EAP) with Full Self-Driving (FSD). To be clear:

  * Some safety features related to autopilot, such as automatic braking and the like, are available to everyone for free.

  * EAP is an optional add-on available today ($5k on the Model 3 if purchased at the time of buying the vehicle, $6k as an over-the-air upgrade) which provides lane-following (requires hands on the wheel and driver attention) and driverless summon features (very low speed, "back out of / into a tight space / drive down the parking lot" stuff without a driver in the car). More to the point, there are two entirely different versions that have existed over the year. AP 1.0 was used on earlier vehicles, based on software and hardware from Mobile Eye. Tesla and Mobile Eye split in a contract dispute. Mobile Eye claims that Tesla wasn't using their hardware right. Tesla says that Mobile Eye found out that Tesla was working on an in-house Autopilot system and demanded that they stop as a precondition to get to continue to use their hardware. Mobile Eye says they knew about Tesla's internal work but didn't feel threatened by it. Regardless, Tesla was forced to switch to their internal version, AP 2.0, which was a step backward. AP 2 is just now catching up to the features of AP1.

  * FSD is Tesla's current goal, where the vehicle can drive itself without you having to have your hands on the wheel or paying constant attention. You cannot use FSD, even if you buy it. It costs $3k on the Model 3 ($4k as an over-the-air upgrade later). The article is talking about FSD being rolled out before engineers think it's ready. To reiterate: you can only buy FSD right now, you can't use it until it's ready. Tesla apparently tried to clarify this for the author:

When reached for comment, a Tesla spokesperson referred back to the company website, where a disclaimer for Autopilot reads that “self-driving functionality is dependent upon extensive software validation and regulatory approval.”

The author apparently nonetheless still failed to understand what that means. You Cannot Use FSD. Period. If engineers are complaining about FSD being rolled out too soon, they're complaining about Tesla selling something that its drivers aren't going to be able to use for too long of a period of time. And you know what, I fully agree with the engineers in that regard - I think it's wrong of Tesla to sell something that there's a big question as to whether they'll be able to get it working reliably enough or pass the serious regulatory barriers in its way.

But if engineers are complaining about FSD, then it's not complaints about EAP. Because the two are very distinct things. EAP isn't perfect, don't get me wrong - and the 1.0 / 2.0 switch was a big setback (they still don't use all of the cameras on the vehicle). But it also pesters drivers enough if they show signs of not paying attention to the road (e.g. not holding onto the wheel) in order to overcome its imperfections (the level of pestering was significantly increased after AP1's fatal accident, in which the driver was apparently watching movies during most of his trip).

Re: Not surprised

[Anonymous Coward]

There's also the motorcyclist in Norway that was run over from behind by a tesla that couldn't detect it.

Re:Not surprised

[BasilBrush]

Chris Lattner wasn't there long enough to get started. We don't know why be backed out.

Personally I could never see why a compiler guy was being hired as head of one of the most complicated AI projects anyway. Different field.

Re:Not surprised

[Anonymous Coward]

"he was instructed not to do what he did"

The car just plain didn't see anything, and your talking about a much more recent version of Telsa's software/hardware, and as AmiMoJo said:

"Tesla seem to have admitted it doesn't work as originally advertised, by repeatedly increasing the amount of effort it makes to keep the driver alert"

Re:Not surprised

[Anonymous Coward]

I don't see how explaining that Telsa is now officially selling something that doesn't yet exist helps their record of misleading the public on "auto-pilot".

Re: Not surprised

[Anonymous Coward]

Oddly, planes don't autopilot to the terminal. Why? It's just parking. That's a solved problem.

Re:Not surprised

[Mr D from 63]

No, he was instructed not to do what he did. And he did it anyway. Seems pretty clear to me.

Yes, two things are very clear;
1) The driver was irresponsible and legally at fault
2) Tesla autopilot was not good enough to stop the car from plowing into the truck on its own.

Re:Not surprised

[Sperbels]

And autopilot for planes is far, far, far more advanced

This comment demonstrates an glaring ignorance of how plane and automobile autopilots work.

Re:Not surprised

[Grishnakh]

And autopilot for planes is far, far, far more advanced, capable, robust, and reliable than the shit Elon is selling.

That's total bullshit. Autopilot for airplanes has been around for many decades now. It just maintains a heading and altitude. It's roughly analogous to cruise control on cars in technological terms, and maybe automatic lane-keeping in actual functional terms (since cars have to follow a road, planes don't; of course, technologically, lane-keeping is far, far, far more advanced than the autopilot in a typical Cessna).

Yes, there are very advanced autopilots in today's newest passenger planes like the 787, but the term is not exclusive to those, and there's countless decades-old Cessnas and Pipers out there with autopilots that are quite primitive.

No, autopilot in planes does not autonomously pilot the plane. It doesn't take off or land, it doesn't fly around bad weather, it doesn't check METARs and PIREPs, it doesn't watch for other traffic, it doesn't handle radio calls to ATC when you cross into class B airspace, it just keeps you flying straight and level.

The only thing in your post that's correct is the bit about pilots being trained to use their equipment. That isn't true for Teslas, but it also isn't true for any other car either. How many drivers on the road today got explicit training to use the cruise control in their car? Cruise control has been around a few decades now too. Or how about the more advanced features we have not, like adaptive cruise control, lane-keeping, and operating the infotainment system? Every car is different, with different controls and different quirks. Airplane pilots aren't even allowed to fly a plane (solo) unless they've been specifically trained for that model and received a rating for it. Perhaps we should do that for cars....

Re:Not surprised

[Grishnakh]

Airline pilots are intelligent and highly trained individuals. That is why they are not found on every street corner, and are worth more than a dime a dozen.

Actually, this isn't true. Pilots start out their careers as instructors ("those who can, do, those who can't, teach"), and make peanuts. After that, they might get a job as a copilot for a regional jet company. Last I heard, the starting salary for one of these guys is $18k. Yep, barely above minimum wage. It takes many years for them to work up to any kind of decent salary approaching 6 figures. Then, when they hit 60 years old, they're forced to retire.

Being a pilot is for people who are independently wealthy (e.g. trust fund, or has a spouse willing to support them), or for people who love it so much they're willing to sacrifice everything just to have that job.

Re:Not surprised

[Anonymous Coward]

The one that happened before this, using Tesla's prior "auto pilot" technology? And who ignored repeated warnings from the system to put his hands on the damn wheel and take control before a crash occurs?

Re:Not surprised

[Rei]

They implied that Tesla is currently having people drive something that its engineers deem unsafe. This is simply not the case at all. If the engineers were complaining about selling FSD, they're not complaining about anything that consumers are actually driving.

Everyone who buys FSD does so on their assessment of how likely they think it is that Tesla will actually deliver. There is zero confusion among anyone who buys it about the fact that they can't use it right away; the option always includes the "you can't use this until it's finished and legally approved" disclaimer next to it. It all comes down to how optimistic or pessimistic you are about the technology. I'm a pessimist, and will not be buying it. Some of Tesla's engineers working on it are apparently also pessimists. I'm not surprised. It's a crazy-hard task, and very different from human-supervised autosteer / EAP.

Re:Not surprised

[Octorian]

The author is confusing Enhanced Autopilot (EAP) with Full Self-Driving (FSD).

Every single person I talk to, who only casually observes Tesla-related news, assumes that EAP == FSD. Every friggin single one. Usually in knee-jerk comments, when I mention something about having to drive my car somewhere. (I have a Model S /w AP v1... Love it in stop-and-go highway traffic and long highway trips, but I'm quite well aware of its capabilities and limitations.)

Re:Not surprised

[Richard_at_work]

Aeronautical autopilots are not safe, precisely because they have significant nuances and problems which have resulted in numerous crashes through misuse and over reliance. You do not turn on an aircrafts autopilot and then sit back and relax for the rest of the flight.

Which is pretty apt considering what we are discussing and what is being claimed on all sides...

Re: Not surprised

[Anonymous Coward]

^--- This. My brother is a pilot. My parents spent about $100k sending him to Embry-Riddle 25 years ago, then spent ANOTHER $100k renting planes for him so he could get his 1,000 hours of multi-engine time and actually become employable by an airline.

The thing that really sucks about being a pilot is that you can't independently afford the training & certifications you need to get hired, but airlines won't pay for the training & certs until after they hire you

It's kind of like the catch-22 film students had prior to cheap HD video... nobody would fund your films until you were good, but you couldn't afford the gear, film, and services to BECOME good in the first place unless you and/or your parents were independently wealthy.

Re:Not surprised

[aix tom]

I think the "on it's own" it really the thing.

If a driver can detect 95% of hazards, and the car can detect 70% of hazards, then there is improved security, because they both would have to miss a hazard. But switching completely from the driver in charge to the car in charge decreases security.

Kinda reminds me of some people I encountered in IT back in the 90's. "Hey, I don't need to backup anymore, I have RAID now"

Re:Not surprised

[speedplane]

Better yet don't call it autopilot. Even if you try to explain it, there are plenty of fools who won't get it. Call it "drive assist" and people might be a little less foolish. Some assholes will still misuse it, but you can't stop someone hellbent on stupidity.

Agreed, this was a marketing fiasco of Tesla's making. The technology is good, but they marketed it as something it wasn't, and at least one guy died.

Re:Not surprised

[HuguesT]

Autopilots in planes have been able to land and exit the runway for quite a while. I'm pretty sure they should be able to take off as well if the plane is well aligned at the start of the runway, but they can't taxi (drive the plane on the ground to the start of the runway).

Re:Not surprised

[drinkypoo]

Agreed, this was a marketing fiasco of Tesla's making. The technology is good, but they marketed it as something it wasn't, and at least one guy died.

Actually, they marketed it as what it was, and that was confusing for people, and at least one guy died expressly ignoring the warnings of the manufacturer, of which he literally could not possibly be unaware since he had to attend a safety lecture before he was allowed to use the feature. He ignored the warnings of the people who produced the system, and that is what killed him.

Re:Not surprised

[AmiMoJo]

I'm a bit worried by their FSD technology. The current auto-steering/speed control needs to use map data to work, i.e. its maps tell it where sharp bends are so it can slow down, that sort of thing. For full self driving it needs to be able to work without accurate maps, e.g. there might be some roadworks or a new road layout that isn't on the map yet.

At best the car would be forced to stop, possibly stranding the passengers if they were unable to legally drive it. At worst it might not slow down in time.

Maybe they will get rid of this requirement before release. The fact that they are selling the feature right now is worrying though, because either it's screwing people over with something that won't be released for years, maybe even before they sell/scrap the car, or because they are planning to release it way too early.

Re: Not surprised

[Anonymous Coward]

Autopilot in planes is far simpler than what is needed in cars, that's why it came decades ago. If a car had a plane's autopilot it would have cruise control and would drive in a straight line.

Re:Not surprised

[Mr D from 63]

I think the "on it's own" it really the thing.

If a driver can detect 95% of hazards, and the car can detect 70% of hazards, then there is improved security, because they both would have to miss a hazard. But switching completely from the driver in charge to the car in charge decreases security.

Kinda reminds me of some people I encountered in IT back in the 90's. "Hey, I don't need to backup anymore, I have RAID now"

Agree. That last part, the human factor element, is the hard part. As people feel safer they naturally take greater risks. We all do it, some more than others. Its those 'some' that do it more that we have to worry about.

Re:Not surprised

[Grishnakh]

Autopilots in planes have been able to land and exit the runway for quite a while.

Some can, many can't. It's just like cruise control in cars: some can detect cars in front of them and slow down ("adaptive cruise control"), most aren't at all that advanced.

I'm quite sure that all the people with 60s-era Cessnas do not have autopilots capable of auto-landing.

Re:Not surprised

[swillden]

Being a pilot is for people who are independently wealthy (e.g. trust fund, or has a spouse willing to support them), or for people who love it so much they're willing to sacrifice everything just to have that job.

Or for people who are willing to do a stint in the military. That's the best route to becoming a commercial pilot: Join the Air Force and let Uncle Sam pay for your training and flight hours. If you're smart, go for cargo jets rather than fighters, because after an eight year hitch as a C-5 pilot (for example), you'll have lots of heavy multi-engine hours which are very expensive to buy.

Re: Not surprised

[LifesABeach]

Maybe the musk could define what his autopilot is?

Re: Not surprised

[LifesABeach]

Also various critters.

Re:Not surprised

[Anonymous Coward]

Of course let's ignore that people drive all the time in snow, rain, fog and road construction conditions when they are told it is unsafe to do so. Look no farther than Texas where hundreds if not thousands of cars are trapped underwater, when they were told to stay off the roads. Every time there is a big snow storm, with road conditions that are too hazardous for people to drive in there are hundreds, if not thousands, of people who ignore the warnings, drive anyway, and have accidents, some of them fatal.
People speed through construction zones and every year dozens of workers are killed.
A self driving car that refuses to move because the environmental conditions are too bad for it to drive is actually better than a manually driven car that allows a stupid person to race down a road that is too hazardous to be driven on. Typically we are on the cusp of having technology that is better than all but the best professional drivers, and it will not be able to drive under conditions that people insist on driving under. Conditions that are too hazardous for them to be driving, but of course they know better.

Re:Not surprised

[speedplane]

Actually, they marketed it as what it was, and that was confusing for people, and at least one guy died ....

If a company markets something that confuses people, it's the company's fault.

The End for Elon

[Anonymous Coward]

Kill Musk, take his money, and pay the rest of us our Basic Income.

Re:The End for Elon

[Anonymous Coward]

I have a better plan.

Kill Musk, take his money, keep it.

When did Musk get his MBA?

[Anonymous Coward]

This is MBA behavior, not engineer behavior. I didn't see anything in the newsfeed about it though. I guess the transformation was inevitable though, in hindsight. Someone remove his pocket protector and have him turn in his scientific calculator on the way out.

Re:When did Musk get his MBA?

[HornWumpus]

Engineer behavior outside of competence is indistinguishable from MBA behavior.

But the engineers are supposed to know better. The MBAs are _trained_ that 'competence is irrelevant' they can manage anything.

Re: When did Musk get his MBA?

[Anonymous Coward]

As someone with a niche skill set, only good engineers know that. There's plenty who aren't very good.

Re:When did Musk get his MBA?

[Mr D from 63]

The software engineer will deploy it at some given milestone and then work out the bugs. The MBA will re-define and accelerate the milestone.

Re:When did Musk get his MBA?

[JohnFen]

This is MBA behavior, not engineer behavior.

That's funny, because when I read this the first thing I thought was that he sounded like one of the new-school engineers. You know, the ones with the motto "move fast and break things".

Re:When did Musk get his MBA?

[Dutch Gun]

No competent engineer, new or old, has ever uttered those words or advocated what they represent. Such a motto only works when you're involved in shit that doesn't really matter. In other words, it's perfect for Facebook, or a small Google team working on a new project that will be abandoned as soon as it's acquired a few loyal users, or a Silicon Valley startup no one's heard of writing an app that no one cares about.

Re:When did Musk get his MBA?

[HornWumpus]

'Engineer' is not just a job title.

Re:When did Musk get his MBA?

[JohnFen]

No competent engineer, new or old, has ever uttered those words or advocated what they represent.

I completely agree. However, it's a sentiment that I hear quite commonly these days, usually from people in software engineering positions who are in their 20s.

Re:When did Musk get his MBA?

[JohnFen]

Absolutely right, but I was talking specifically about people holding software engineering positions.

"Move fast and break things" started as the engineering motto at Facebook (thanks, Zuck). Unfortunately, there's a whole bunch of people who still think that sounds great.

Re: When did Musk get his MBA?

[Anonymous Coward]

Which is precisely why most programming and devlopment positions shouldn't have the word engineer in them.

Re:When did Musk get his MBA?

[Anonymous Coward]

They Write the Right Stuff

I'm not a software engineer, just the old fashioned brick and mortar kind (the ones who build physical systems for the real world that people depend on for life). But this was required reading when I started my job.

Re:When did Musk get his MBA?

[Anonymous Coward]

Engineer behavior outside of competence is indistinguishable from MBA behavior.

and I suppose this particular niche of behavior comparison is your area of competence?

Re:When did Musk get his MBA?

[Dutch Gun]

They Write the Right Stuff

I'm not a software engineer, just the old fashioned brick and mortar kind (the ones who build physical systems for the real world that people depend on for life). But this was required reading when I started my job.

Brilliant read, thanks. I loved reading the part about how errors that slip past are meticulously analyzed. When a bug is found and fixed, the entire process is looked at to discover *why* the bug slipped past in the first place.

This is something I instinctively do as a developer, but have never really heard formalized. When I see a mistake that was made, I like to take a step back and ask "was there something I could have done to prevent that mistake from occurring in the first place? What part of the process can be tweaked for fixed to guarantee it doesn't happen in the future?" Essentially, turn each bug into a positive feedback loop for making your software better.

Also, as a bonus, I think we just figured out how to get women back in the tech industry:

Otherwise, the hour-long meeting is sober and revealing, a brief window on the culture. For one thing, 12 of the 22 people in the room are women, many of them senior managers or senior technical staff. The on-board shuttle group, with its stability and professionalism, seems particularly appealing to women programmers.

Re:When did Musk get his MBA?

[JohnFen]

Also, as a bonus, I think we just figured out how to get women back in the tech industry:

Maybe this explains why I've not personally experienced a lack of women at the places I've worked. Of the last three companies, the one with the fewest women engineers had about 1/3 women.

But I am extremely picky about where I work. If I get a hint of poor management, or I disagree with their engineering approach, I pass.

Re:When did Musk get his MBA?

[werepants]

It is a good read, but there are a couple things worth noting. The process described in the article isn't really a description of software engineering - it's a description of how aerospace engineering is done, period. It's also a description of why the U.S. spends a ludicrous amount of money on defense. At the end of the day, you've always got a balance between risk and budget. Risk will never reach zero, and for any substantive piece of software it will never be perfectly tested and perfectly bug free. So the question is, what's your risk tolerance? And even more importantly, can you afford to pay down the risk to that level?

At these aerospace places, the risk tolerance is very, very low. So they spend 100x as much to get something that fails 10% as often, typically. You have to pay for an army of requirement jockeys and endless layers of bureaucracy and auditing to try to control that process. And since the risk tolerance is so very low, change is extraordinarily painful.

And the unfortunate truth is that, at least in some areas of testing and product assurance, you can easily spend 100x as much money but get no real improvement to your risk profile. There are a ton of consultants out there that will invent fake problems, and then come up with "solutions" to the fake problems that require hundreds of thousands of dollars to implement. How do you know, when you spend that money on testing or product assurance or adding checks and balances to the process, that you have actually caught the biggest risks? Or that you've improved the risk at all? You really don't.

So what ends up happening is that it's easy to point at how some of these engineering groups do a really amazing job with process control and they catch a lot of bugs. But then you also get less talented groups that try to emulate the same approach, and superficially they add the same bureaucratic checks and religiously follow Process, but they don't really understand that process or they don't know how to apply it in the right place, and so you can end up with all the inertia and cost and dogma of a strictly controlled engineering environment, but without any of the actual benefits to the project.

Jail time

[Anonymous Coward]

100 years dungeon, no trials!

Negligence du jour

[Anonymous Coward]

And people wonder why I don't think arrogant salesmen like Musk should have the power to make OTA software updates without my permission.

There's always one or two voices....

[Anonymous Coward]

.... not disputing that people may have had a valid point here, but anyone who's tried to execute a change or deliver an outcome will always find one or two dissenting voices in any organisation of scale. Once you take away the marketing fluff, what are the actual facts or reasoning behind their concerns? In my mind its' a discussion that runs very similar to:

1) Company develops new anti-cancer medication. Reduces mortality by 85%.
2) In 5% of cases people will die from a non-cancer related reason, as a result of taking the medication.

Musk appears (though I'm struggling to find specific evidence to support) to have suggested that the net balance is in favor of the roll-out, because you can never deliver anything both perfect and safe. Engineers tending to be cautious in nature, in small numbers have reservations about #2. I think some more clear messaging within the vehicle when you engage autopilot would put this to bed:

'You must remain attentive and prepare to take control of the vehicle at any time'.

It already stops you from taking your hands off the wheel for more than a few seconds, after a few folks have abused the system previously (the chap watching Harry Potter and eating lunch springs to mind) - so aside from semantics around marketing, this feels like lawyers trying to put the screws on Telsa for not shipping a product that prevents people ending themselves through idiocy.

Re: There's always one or two voices....

[Anonymous Coward]

Most licensed engineers have agreed to hold the public safety Paramount.

If they do not do that because a new toy is really cool, that's an actionable breach of professional ethics.

Re:There's always one or two voices....

[JohnFen]

anyone who's tried to execute a change or deliver an outcome will always find one or two dissenting voices in any organisation of scale.

Absolutely true. And it's equally true that it's foolish to not take those dissenting opinions very seriously (even if, after careful consideration, they don't change your plans).

In any organization, there is a strong "rah-rah" tendency, and people tend to suppress their own doubts. Nobody wants to be a wet blanket or potentially risk their career by not seeming to be a "team player". So the voices of those who point out problems need to be listened to much more carefully than the voices of those who say "everything's great".

Re:There's always one or two voices....

[Mr D from 63]

.. this feels like lawyers trying to put the screws on Telsa for not shipping a product that prevents people ending themselves through idiocy.

Why should Tesla be treated different than any other company that must prevent idiots from killing or injuring themselves with their products? In Tesla's case, one could argue Musk was encouraging them to take risks based on his early descriptions of its capabilities.

Re:There's always one or two voices....

[bobbied]

Ah yes, but marketing is reality to a lawyer looking for a payday. You cannot SELL something with color glossy printed materials if what you are actually getting isn't as described in said color glossy materials.

You can bet that the lawyers will have their way with Tesla should the "autopilot" thing turn out to have faults (or even limitations) that might kill you and these are not plainly disclosed in the advertisements. You can also bet that Tesla will now require signatures (in blood) that indemnify them of all liability for any faults or limitations, past, present or future to make it harder for their customers to sue...

Re:There's always one or two voices....

[Mr D from 63]

Yes, particularly when human safety is at the heart of the matter. OF course, sometimes there is enough evidence to move forward if one feels the concerns are addressed, when doing so there should be some validation.

Aside from that, I think Musk is guilty of misrepresenting safety numbers. He repeatedly stated cars on autopilot were safer than cars without, even stated a number, but used apples and oranges data to do that comparison, with no normalization of the non-Tesla data to make in comparable. I think he's smart enough to know exactly what he was doing, and was happy to intentionally mislead the public. Probably because he believed the system truly was safer even though there was not sufficient data to make that claim.

Re:There's always one or two voices....

[Anonymous Coward]

You sound anti-profit. Are you anti-profit?

Re:There's always one or two voices....

[drinkypoo]

In Tesla's case, one could argue Musk was encouraging them to take risks based on his early descriptions of its capabilities.

Every owner that wants to enable the autopilot functionality is required to attend an orientation in which the limitations of the system are explained in some detail, so there is literally no valid argument in that direction.

Re:There's always one or two voices....

[Mr D from 63]

In Tesla's case, one could argue Musk was encouraging them to take risks based on his early descriptions of its capabilities.

Every owner that wants to enable the autopilot functionality is required to attend an orientation in which the limitations of the system are explained in some detail, so there is literally no valid argument in that direction.

Oh, there is. There were plenty of videos out there that showed T drivers doing amazing no hands driving. Did Musk make ANY attempt to warn folks that such driving was not responsible, or did he happily enjoy the PR attention that came along with it? So those classes you talk about clearly were not sufficient.

Re:There's always one or two voices....

[ElizabethGreene]

Did Musk make ANY attempt to warn folks that such driving was not responsible, or did he happily enjoy the PR attention that came along with it?

Yes, the car nags you not to do no-hands driving, and in the post-one-drove-under-a-truck-crash software updates have made it more aggressive about annoying you.

The only way to no-nag drive a Tesla hands free now is to hang your water bottle off of the side of the steering wheel.

Re:There's always one or two voices....

[Mr D from 63]

Publicly he didn't. He issued zero statements when all those videos were coming out. It wasn't till after he started getting blowback that he toned things down.

Deep neural nets will never give us full autonomy

[Anonymous Coward]

Denials notwithstanding, a deep neural net is just a fancy rule-based expert system. The rules are in the form IF A THEN B where A is a pattern and B is a category or label. It's GOFAI redux. Deep nets suffer from the same problem as all rule based expert systems: they are brittle. Given a situation for which there is no rule, they fail catastrophically.

Full driving autonomy will not happen until AGI gets here. Unfortunately for Musk, nobody knows how to do that.

I'm sure his space program

[Anonymous Coward]

works the same way

explosions on launch much?

Full autonomy would be unsafe

[backslashdot]

The current hardware doesn't have side facing cameras (or lidar or radar) on the front sides of the car. There are cameras in the side front fenders (in the logo) but they are looking backwards. They need to have side cameras close to the front of the car and high up as possible because before proceeding onwards from a Stop sign you need to see what's coming at you from the left or right side. The camera mounted in the middle post of the windows doesn't have an adequate view.

Re:Full autonomy would be unsafe

[BasilBrush]

There are side facing cameras in the B Pillars. This is only a few inches further back than where a human driver's eyes are, and approximately the same height.

Re:Full autonomy would be unsafe

[backslashdot]

That's not good enough. We want it to be better than what a human driver would be able to work with. 40% of fatal accidents are at stop signs -- mostly side impact. That is, tens of thousands of people are killed in side impact collisions every year. Reference: https://safety.fhwa.dot.gov/in...

Anythng to eliminate that would be good. If a collision is imminent the early warning may help the car decide to speed up or brake such that the passenger compartment is safe. Having a camera in the B pillar may help ensure that the Tesla wasn't at fault but what good is that to the dead occupant?

Re:Full autonomy would be unsafe

[BasilBrush]

You are only looking at the cons and not the pros. 6 inches further back is neither here nor there. You get a far greater variance than that for human drivers based on the model of car they drive and how long the hood (bonnet) is.

Meanwhile on the plus side, you have 360 degree coverage at all times, rather than the limited field of view of a human driver, and the blind spots caused by pillars.

You don't have the selective vision of humans, where the driver simply "didn't see" a motorcycle coming, because they were looking for cars, and forgot about motorcycles.

And you don't get the bad behaviour, where the human driver may not even stop at a STOP sign, because he thinks he knows better, assumes it's clear, or he's just an asshole.

Re:Full autonomy would be unsafe

[Anonymous Coward]

Putting a camera on the front fender just behind the lights would be a difference of about 5 feet, not 6 inches. That is a difference of about 1/10th of a second for a car going at 35 mph, that may be barely enough of time to prevent being killed by a person who is about to blow past his stop sign.

Re: Full autonomy would be unsafe

[Anonymous Coward]

I am not arguing against having a camera in the B pillar. That's needed. But I am saying a side-facing pair is needed further up front as well, just behind the lights.

Mr Musk

[Trogre]

Can you please focus on making your electric cars and the batteries that run them better and more affordable instead of getting sidetracked with these bullshit features that nobody wants?

Thank you,
Everyone

Re:Mr Musk

[PoopJuggler]

Nobody wants self-driving cars?

Re: Mr Musk

[sound vision]

The guys who program the self driving features are probably different from the guys who are working in materials science trying to improve the batteries. You might be able to move funds from one department to the other, but my guess is that Tesla already has the best minds in the right places, and they would hit diminishing returns trying to bring in fresh blood. Not that there isn't more talent out there, but that Tesla already has as much talent as it can support efficiently. Hiring more now would lead to diminishing returns. Beyond hiring and moving funds around, I don't know what else you expect an organization like Tesla to do to encourage development in specific areas.

Re:Mr Musk

[bobbied]

Can you please focus on making your electric cars and the batteries that run them better and more affordable instead of getting sidetracked with these bullshit features that nobody wants?

Actually, I think everybody wants the stuff the pied piper is attempting to sell (I'd love to have all that stuff he keeps dreaming up), he just doesn't have it at a price anybody can really afford. I cannot afford a Tesla, even the stripped down model myself.

Of course, Musk's issue is that a Tesla is way too expensive BECAUSE of all this wiz-bang cool stuff he keep stuffing in them, and THAT is why Tesla will continue to struggle as a company until this kind of madness stops.

Take an example from history. The beauty of the Ford Model-T wasn't that it was available only in black and only in one configuration from the factory, but that it was CHEAPER than the hand built cars of the day and the average Joe could afford one. The average Joe cannot afford a Tesla, which is why he cannot sell enough of them. There is something to learn from Ford.

Re:Mr Musk

[LynnwoodRooster]

Motorcycle riders don't given they miss motorcycles at a rate 10 times higher than cars or pedestrians...

Re:Mr Musk

[Anonymous Coward]

Thank you for your comment, Mr. Everyone.

As always, you seem to have something against Mr. Nobody. I don't know why you denigrate and seek do deny Mr. Nobody features that he might want, but please be assured that nobody cares.

(captcha: egotism. Just brilliant!)

Re:Mr Musk

[Mr D from 63]

Of course, Musk's issue is that a Tesla is way too expensive BECAUSE of all this wiz-bang cool stuff he keep stuffing in them, and THAT is why Tesla will continue to struggle as a company until this kind of madness stops.

That is a valid point, but the other side is that the cars are going to be very expensive regardless simply because of the batteries, and that wiz-bang stuff is helpful in getting the target market to part with their $$$$ by further distinguishing the product.

Re:Mr Musk

[BasilBrush]

Evidence?

Re:Mr Musk

[BasilBrush]

Nobody wants? I can't wait!

I suppose you imagine no one wanted cruise control either.

Re:Mr Musk

[bobbied]

Perhaps, but Musk is playing a loosing hand on this then... With Oil and Natural Gas prices at bargain basement prices and no real upside in sight, who can afford an EV anyway? The ROI on the investment just isn't there now.

For me (with a 15 min commute one way) an EV would be great, but there is zero chance I'm going to get one anytime soon. I simply cannot afford the extra expense of the purchase when even if I had free electricity to charge it with the cost savings on fuel wouldn't make up the difference in price in nearly a decade. By the time it paid for the extra cost to buy it, I'd be putting new batteries into it to keep it on the road. I'm sorry, my pickup truck is cheaper...

Musk needs to figure this out if he intends to sell his cars in volume. If he cannot get the cars cost competitive with his competition, then he's severely limited his target market and is going to struggle to turn much profit, selling a niche car to a niche market. To make money selling cars, you have to sell lots of them...

Re:Mr Musk

[Mr D from 63]

I pretty much agree with you on the ROI thing. That's why Musk was brilliant to sell a high end vehicle where ROI wasn't a fatal concern for the customer. I'd also love to have an EV, it would be a blast to drive, but the ROI is fatal to my buying decision as well, as are my travel needs. Not only that, but I'd have to clean out my garage to park it by a charger and that's a showstopper right there.

Re:Mr Musk

[rtb61]

Self driving with zero wireless inputs thank you very much and add in a system verify feature at startup, to ensure no hacking. So multiple systems, rather than an all in one hackathon special, that take you pick of three letter agencies, can hack to drive you straight off a cliff, or into a train or on the other side of the road towards a semi that is also accelerating because it has been hacked to kill you. So self driving, not remote, who the fucks wants to get into a remote controlled vehicle. The Tesla marketing is also pretty clear, autopilot is not installed but the system can at some time in the future be upgraded to autopilot and I am not too sure on that language as to whether or not, that is just a system firmware upgrade without any additional hardware needing to be added or updated.

What I do not, is both the fossil fuellers and existing fossil fuel auto manufacturers will attack Tesla at every opportunity because competitor and psychopathic capitalism. So crappy twisted interpretive articles, are to be expected, that is the norm for modern business. Tesla can of course counter with attacks on fossil fuel vehicles, like warnings not to start you fossil fuel motor in you garage because if you should say faint for any reason, that car will now out and out kill you, you should push you fossil fuel car out of the garage and only start it's infernal combustion motor when it is safely out in the open and of course how about taking a fuel tank, stirring it up and then igniting it and watching how it blows a car to bits, equivalating a fuel tanks explosive potential to how many sticks of dynamite you are carrying. Not to forget the pollution or the fossil fuel wars, mass murder to fill your tank with a horribly toxic product, that can quite readily kill you ;D.

Re:Mr Musk

[LynnwoodRooster]

here you go.

Re:Mr Musk

[Anonymous Coward]

Nobody whose actually thought about it for more than three seconds wants self-driving cars.

Re:Mr Musk

[Trogre]

No.

Cruise control amounts to a throttle with a memory so you can dial in a speed until you intervene. Occasionally useful, often overused, occasionally dangerous but very rarely leads to deaths when it goes wrong.

Self-driving cars involve you entrusting your life to software that:

1. Should be able to determine the best route from your location to your destination, with any intermediate waypoints, and maintain a speed in compliance with local restrictions. For travel on common roads, software is getting pretty good at this.
2. Must identify potential hazards correctly, including changes in weather conditions, road conditions (including melted tar, black ice, floodwater), signage, and all types of traffic, and take appropriate action every time, weighing priorities where multiple decision branches exist. Despite what marketing material you may have seen, no software in the world can actually do anything close to this in anything other than the most useless contrived, controlled, conditions. Of course, humans often get it wrong too, but they tend to fail in mostly predictable ways.

Of those two aspects, which do you think is more important?

There's more to driving than Gran Turismo, you know.

Re:Mr Musk

[BasilBrush]

OK, so you have no evidence for motorcycles being missed 10 times as often as cars of pedestrians.

The first two were stories about collisions. So what? Human drivers often miss motocycles and hit them. You've presented no evidence that Autopilot does it more often.

Re:Mr Musk

[BasilBrush]

None of that reflects the post you were replying to, which was about the fact people want autopilot, just as they wanted cruise control in previous decades.

And the Gran Turismo jibe - I've been driving for 30 years. I never play driving games. So duh!

Re:Mr Musk

[Anonymous Coward]

Wow, you again... must be either heavlily invested in tesla or even on their team. Or just brainwashed appologist.

Re:Mr Musk

[Trogre]

Now you've got me curious - what exactly do you think "autopilot" means, when applied to cars and not aircraft?

Re: Mr Musk

[speedplane]

I don't think the problem is wasting talent, the problem is wasting goodwill. If Tesla continues to act recklessly, pushing products that are not quite ready for market, and people get hurt, they may delay the development of self-driving cars altogether. I want to see them as much as anybody, but if they keep making dubious claims about the technology, people will lose interest, excitement will ebb, and funding and development will slow.

Re:Mr Musk

[Anonymous Coward]

> For me (with a 15 min commute one way)

There is an amazing invention called bicycle, it runs on cereals and shake, no gas needed. But if you are too lazy for that, a Honda or Piaggo scooter would do just fine, no need for a war chariot on such a short trip. In fact you could ride a horse cowboy style as well.

Re:Mr Musk

[drinkypoo]

Self driving with zero wireless inputs thank you very much and add in a system verify feature at startup, to ensure no hacking.

Yep. I'm not getting into an AV without a glass break tool. At least that way if someone tries to use it to kidnap you, you might be able to escape. I'm not particularly worth kidnapping, but lots of people who aren't really worth kidnapping get kidnapped anyway because it's cheap and so many people have nothing to lose. AVs will make it both cheap and easy.

Re: Mr Musk

[Anonymous Coward]

It took me more than three seconds of thought to decipher your gibberish.

Re:Mr Musk

[Whibla]

Motorcycle riders don't given they miss motorcycles at a rate 10 times higher than cars or pedestrians...

Evidence?

No, hyperbole!

Evidence would require research:

Number of Tesla cars on the road. Number of non-Tesla cars on the road. Number of motorcycle accidents involving a Tesla. Number of motorcycle accidents involving a car that wasn't a Tesla.

And some basic understanding of statistics:

Given the proportionally low number of Tesla vehicles compared to total vehicles you'd probably want to limit data to particular locales (i.e. continental USA), and, further, correct for 'error magnification'

Based on my observations there's little evidence GP is engaged in anything other than anti-Tesla FUD.

I will say that I tend to agree with thread parent though. By all means, Tesla, continue the research into fully autonomous driving but a little more focus on production of a reliable electric vehicle, and the scaling up of that production process, without sacrificing quality, would probably be sensible right now. The vehicles Tesla makes are already desirable, the bells and whistles they already incorporate are more than sufficient for 99% of their market. Full on autopilot, while being "the-time-is-right-sci-fi", and thus a draw to potential customers, is not the main reason most people are ordering and buying your cars (I'd actually go as far as to say that were it included as standard it'd put off as many people as it would attract - I love driving, I dislike, intensely, being driven).

Re:Mr Musk

[HuguesT]

The Tesla forum discussion (3rd link) makes it clear that the the Tesla software is not properly aware of overtaking motorcyclist and is actually confused by them into thinking that the *car* ahead has accelerated. Which causes the Tesla car to accelerate in turn, when there is a very real slow car ahead. This is a known issue which at the time of the forum had not been fixed (July 2015). Since then, Tesla Motors has parted ways with MobileEye and reports are that the newer software is not as good.

Re:Mr Musk

[BasilBrush]

Any issues with MobileEye is irrelevant now, as current AutoPilot is completely different. Yes in some ways it's still catching up on features. But you don't know from that old source that there is any motorcycle problem in current Teslas. And it certainly does nothing to confirm LynwoodRoosters fabricated claim that they miss motorcycles 10 times more often than human drivers.

Re:Mr Musk

[BasilBrush]

Good post. Except for the last paragraph. Because you personally love driving, I think you are underestimating the number of people that don't take any pleasure in driving - at least real driving, much of which is stop/go taking place in congested traffic, down the same route twice a day every working day.

If cost wasn't an object I'd say most people would want a system that could driver the commute for them. And inevitably cost will not be an object for long.

Re:Mr Musk

[bobbied]

15 miles on a bicycle when it's 104 out? During Rush hour on a freeway? I don't think so, I'm not interested in a quick and painful death, even at this age.... I did the no AC thing for over a decade in my 65 VW Beetle, back in the days I couldn't afford more than a $600 car and now you suggest I ride a bike?

Get off my lawn! {-smile-}

Re:Mr Musk

[Anonymous Coward]

I still don't understand why people want cruise control. It might make sense with networked traffic where you have a row of cars all agreeing to go a set speed. But in regular traffic, you can't count on anyone going a set speed. Sure, you get that one guy doing a constant 50 in the left lane, but he'll speed up to 80 when anyone tries to pass him and then slow right back down to 50 afterward. The second you think it's safe to set a constant speed, the guy in front of you will slam on his brakes for no reason. So now you need a fancier cruise control to automatically brake and adjust speed as conditions change, which requires more oversight because the consequence of failure is higher. Might as well just drive the car normally at that point.

Re: Mr Musk

[BasilBrush]

And yet there are not an undue amount of people getting hurt due to AutoPilot. i.e. There are not more people being killed or injured in Teslas with AutoPilot than in cars without AutoPilot.

And there are no shortage of people interested in AutoPilot.

Re:Mr Musk

[ElizabethGreene]

I want a self driving car. I'm a terrible driver. This could literally save my life.

Re:Deep neural nets will never give us full autono

[K. S. Kyosuke]

Your mom is just a fancy rule-based expert system.

Re:Deep neural nets will never give us full autono

[Anonymous Coward]

Human's aren't that great in situation's for which they don't have experience either...

Who would have thunk it?

[niittyniemi]

That the fountain of BS that is Elon Musk allegedly put personal profit ahead of people's lives?

It seems incredible to me that any engineer who values their career would do any work for him in any shape or form.

I've heard him described as "a latter day Edison".

Unfortunately, Musk knows as much about engineering as Sonny Bono did about downhill skiing.

I think he's a naturally slothful person, sluggish and indolent, a dawdling flaneur, content to waste his life spread eagled on pillows forever indulging himself in the pleasures of the palm.

TL;DR: A complete wanker.

Re:Who would have thunk it?

[Anonymous Coward]

Very funny.

Re:Who would have thunk it?

[Anonymous Coward]

At least he's willing to push the envelope and move a stagnant industry forward.

Re:Who would have thunk it?

[speedplane]

That the fountain of BS that is Elon Musk allegedly put personal profit ahead of people's lives?

... TL;DR: A complete wanker.

Maybe a wanker, and maybe risking people's lives, but don't think he's doing it for profit. He's doing it to advance the state of the art. Still not a valid excuse, but better than profit.

Re:Deep neural nets will never give us full autono

[Anonymous Coward]

We are amazingly great in new situations. In fact, most of the patterns we experience on a daily basis are encountered for the first time. We know how to generalize.

risk assessment

[Anonymous Coward]

If the potential pay out from the crashes that may happen is less than the money gained from the boost in sales due to this feature then musk made the right choice..

after all you must break a few eggs to make an omelet.

and for those that are going to hate all over elon for this, why arent you grabbing your pitchforks and going after all of those pharma CEO's? the side effects of some of the medications i have heard about are ridiculous.

Simple fact, it is impossible to make a 100% fool proof autopilot given the technology we currently have, either we take a risk and learn from the mistakes we are bound to make or we perpetually put it off until it is perfect (which is effectively never)

capcha: quagmire

Re:risk assessment

[Anonymous Coward]

Cars don't have to go through multiple-stage, medium-large people trials and show safety and efficacy that improves over the current treatment standard....

Re:risk assessment

[bobbied]

You assume Musk's motives are about selling cars... I'm not so sure that's true.

I actually think that Musk's driving force is more about PR than running any of his business ventures the most productive way possible. I suspect that he craves the attention that comes from having that flashy idea, and the money that comes from the starry eyed investors who flock to his door to "invest" in them. I don't think he's a snake oil salesman, only that he's not opposed to throwing plausible ideas up on the wall and see what kind of attention sticks to it.

In short, he only really cares about the attention. Which doesn't mean he's not about turning a profit, he wants that too, but because it brings him positive attention. Which, in Musk's case, hasn't been a bad thing for him.

Think Howard Hughes, only as an extrovert...

Of course.. Your mileage may vary...

Re:risk assessment

[Anonymous Coward]

I think Musk is about pushing the technology. Not that the self-promotion isn't a factor, but I think that he really cares about improving the human condition through the application of technology. Think Tony Stark, without the iron suit, but with the same ego.

Re:Deep neural nets will never give us full autono

[Fwipp]

Really, who isn't?

No shit

[Anonymous Coward]

There is always people that will say something is unsafe and risky. It's also true that everything has some risk. Doing nothing and sitting in a cave has risk.

When you buy any vehicle you assume some risk. If you are buying a vehicle and believing it to be 100% risk free you are an idiot who shouldn't be on the road.

Re:No shit

[bobbied]

IF the color glossy sales literature leads you to believe that the thing you are buying is capable of safely driving itself and the company doesn't go out of it's way to dispel such misconceptions, one could imply that the company either advertised falsely and/or produced a product that wasn't safe.

Change products for a second... Let's say you market a drug to treat some sickness. You clearly say it CURES the illness in your advertisements, and in most cases that's true. But in a small percentage of cases, it doesn't cure anything but kills the patient. Do you think you, the manufacturer, are going to be sued? You betcha, lawyers are going to be beating down the doors of every civil court in the country to file suit on behalf of those unfortunate people who die. Why do you think RX companies have all those warnings on their stuff? You think they do that because the FDA makes them? Maybe, but I don't think so.

Same thing for an "auto pilot" in a car.... Kill one person while on autopilot and queue up the civil suits...Follow that with large disclaimers and liability wavers for everyone...

It's why we cannot have nice things...

Re:Deep neural nets will never give us full autono

[K. S. Kyosuke]

Trump?

Re:Deep neural nets will never give us full autono

[Anonymous Coward]

What about the patterns we experience while driving? I think they are fairly common. In exceptional circumstances, stopping or swerving to safety sounds reasonable, and there's no reason to believe we need AGI to reach human level capabilities there...

Re:Deep neural nets will never give us full autono

[sexconker]

Human's aren't that great in situation's for which they don't have experience either...

Like pluralizing thing's?

Re:Deep neural nets will never give us full autono

[PoopJuggler]

It won't matter because AGI will mark the extinction of humans.

Re: Deep neural nets will never give us full auton

[Anonymous Coward]

Wrong!

He is very disciplined, he has the best rules.

Re:Deep neural nets will never give us full autono

[Anonymous Coward]

Actually, I have a lot of experience with grammatical mistakes.

Re: Deep neural nets will never give us full auton

[Anonymous Coward]

Case in points

Re:Deep neural nets will never give us full autono

[bobbied]

I can come up with a few examples that I STRONGLY suspect don't meet that standard... Mostly they drive cars around here, but some of them are in public office...

Re:Deep neural nets will never give us full autono

[bobbied]

We know how to generalize.

Hmmm... Ain't that the truth...

Autopilot!

[sexconker]

You know, a country with money's a little like a mule with a spinning wheel. No one knows how he got it and danged if it knows how to use it.

Heh-heh, mule.

The name's Musk, Elon Musk. And I come before you good people tonight with an idea. Probably the greatest... Aw, it's not for you. It's more of a China idea.

Now, wait just a minute. We're twice as smart as the people of China. Just tell us your idea and we'll give you subsidies for it.

All right. I'll tell you what I'll do. I'll show you my idea. I give you the Tesla Autopilotl!

I've sold autopilots to Plymouth, Oldsmobile, and Studebaker, and by gum, it put them on the map!

Well, sir, there's nothin' on earth like a genuine bona-fide electrified one-car autopilot! What'd I say?

Autopilot!

What's it called?

Autopilot.

That's right! Autopilot!

Autopilot, autopilot, autopilot, autopilot, autopilot...

I hear those things are awfully new.

It's user tested, but not by you.

Is there a chance the car could crash?

It's not your life, so splash the cash.

First adopters must be brave...

They'll be given early graves.

Will this venture fund new green jobs?

No, good sir, I'm the new Steve Jobs.

We've killed off our whole space program.

Fund my SpaceX, my good man.

I swear, it's the country's only choice! Log in to PayPal and raise your voice!

Autopilot

What's it called?

Autopilot

AUTOPILOT!

But the economy's still all fucked and broken.

Subsidies, this man has stolen!

Autopilot
Autopilot
Autopilot!
Autopilot!!!

Auto...*CRASH*

Re:Deep neural nets will never give us full autono

[LynnwoodRooster]

And thus she earns a label?

Words are important

[presidenteloco]

I don't have the exact words Musk has used, but I distinctly remember that he said that all Teslas will come equipped with the HARDWARE necessary for fully autonomous self-driving (computer power, sensors), but that the actual functionality would depend on a future software upgrade.

Now you and I, as software-related techies most of us, know that that will have to be one MASSIVELY COMPLEX and not really invented yet by any stretch of the imagination software upgrade, but technically, what he said is not false.

Re:Words are important

[Carewolf]

I don't have the exact words Musk has used, but I distinctly remember that he said that all Teslas will come equipped with the HARDWARE necessary for fully autonomous self-driving (computer power, sensors), but that the actual functionality would depend on a future software upgrade.

That is the new line. Elon was much bolder in the past, and the whole thing have always been intended to give the impression of being fully automated even if it wasn't. You can see the confusion all over older threads discussing the Tesla.

Re:Words are important

[AmiMoJo]

I'm not expecting this upgrade any time soon. When they moved from the V1 to V2 sensor package the current auto-steering system got a lot worse, and the car's ability to read its surroundings was severely degraded too.

For example, on the V1 package the car could tell the difference between motorcycles, cars and trucks. The V2 sees motorcycles as cars and often mistakes trucks for cars as well. It also seems to detect them much later, and not see nearly as far ahead as V1. The auto-steering seems much more prone to over and under-steer as well, sometimes crossing into oncoming traffic if the driver isn't paying attention.

That set-back seems to have massively delayed their self driving plans. Clearly if they can't get auto-steering and their vehicle detection sensors to work even as well as the old hardware, they aren't going to push out to full self driving any time soon.

Someone always has to make the tough call

[RhettLivingston]

I've been in engineering organizations releasing new products that had life saving or threatening potential. It is always an agonizing, scary hard call as to when you've passed the threshold of risk.

There is a bell curve with a peak. You rarely hit the peak. If you make the call too late, you cost the lives of those you might have saved - too soon, you cost lives of those who might have saved themselves.

Even if you hit the peak perfectly, you'll always be able to truthfully argue that some people are being saved who would have died and some are dying who would have lived. The peak is a point of balance between the two - not a perfect elimination.

I can remember many times hating my bosses when they released a product that I didn't feel was ready. As an engineer, I have to be over-focused on the problems and stand no chance of seeing when I am perfectly perched on that probability peak. They had to pry the projects from my hands to get them out the door. I actually begged in tears once. But, in retrospect, I can't think of any case where my bosses weren't right in releasing the product that I was concerned about releasing.

What we need to force progress is for attorneys to get smart and start figuring out how to file more effective suits for lack of progress toward autonomy. How many are dying today because we don't have it? We need to focus hard on that.

Re:Someone always has to make the tough call

[Anonymous Coward]

What we need to force progress is for attorneys to get smart and start figuring out how to file more effective suits for lack of progress toward autonomy. How many are dying today because we don't have it? We need to focus hard on that.

That sounds a lot like slavery

Re:Someone always has to make the tough call

[niittyniemi]

As an engineer,...

You are obviously not an engineer in the way I'm familiar with in the UK. ie. chartered and at a minimum a member of your professional institute: mech, civil, electrical, whatever; they've all got their own professional body. Except software "engineers", of course.

I've been in engineering organizations releasing new products that had life saving or threatening potential. It is always an agonizing, scary hard call as to when you've passed the threshold of risk.

The fact that you agonize over whether to release a possibly dangerous product to the public strongly suggests to me that just like Mr. Musk you are not an engineer at all but just another wannabe.

If you were a professional, chartered engineer, there would be no "agonizing". You do not sign-off on a possibly dangerous product. Period.

Why? Because if somebody gets seriously hurt, or worse dies, you or your company will be on the receiving end of a very expensive lawsuit. Multiply the expense x10 in US, at least.

But what's considerably worse for you, is that you will be kicked out of your institute, you will no longer be able to practice and your career will lie in burning ruins around you.

As for any future employment whatsoever, see how: "killed and maimed people" goes down on your CV/resume with prospective employers.

That is why Musk's engineers are bailing and why I said in my previous post that no proper engineer would work for him if they valued their social standing, employment prospects, financial wellbeing and conscience.

Re:Someone always has to make the tough call

[RhettLivingston]

Having the license in this country is often career-ending, much like having a PhD. It can make it very difficult to get a job. I've been in corporations that had thousands of engineers and never met anyone I knew to have it. I think they tend to be in certain structural and mechanical, and most civil and architectural engineering areas. The electrical, aeronautical, and computer engineering professions have much less of this.

Regardless, there is no such thing as a vehicle on the road today that does not make some safety compromise. Not one single vehicle uses the best-known safety mechanism for every single aspect of the car. No one could buy it if they did, and it wouldn't meet other necessary criteria if every compromise was made in the safety direction. Our government often has to force the matter by making regulations like the ones coming down the pipe soon to require all vehicles to have automatic braking technology. This is tech that has been available for a while, but many engineers must be signing off on vehicles that are killing people, otherwise, the government wouldn't have to be stepping in.

Everything engineered makes these compromises. For example, every building might be designed to handle a 500-year quake, but what happens if a 5,000-year quake comes along?

Airbags are an interesting example. Even the best airbag systems kill some people who would not have died without airbags. But they save many more that would have. So, you accept the compromise. Many years ago, seatbelts did the same and still do. Yet, we have them, and are even required by law in most places to wear them.

With the autonomous vehicle question, it is ready to deploy when it will save more people than it will kill when measured versus human drivers (all of them, not just the competent ones). To wait any longer would be killing those people that it might have saved. Of course, determining when that point is is a near impossibility. The hard call will either be made or the vehicles will never be made because the engineers will never be able to say with any product that it is not flawed in some situation - often in which it is being misused by the consumer.

Realistically, we do wait longer than the point of net balance because the public does not understand statistically-based decisions very well. When it is your family member that died because the tech failed you want to blame the tech without looking at the whole picture. We often don't even know when our family member died because the tech that could have saved them was held back because it was being over-engineered.

Often, these hard decisions are the reason for regulation - not to protect the public but to allow the companies protection in deploying something that a big picture organization like the government has determined will be a net benefit to the public while being a detriment to some individuals. The engineers then have the excuse of having met the regulation. It seems to work better with our minds.

Absent specific regulations and tests to target (which is the ideal situation in a free society), the business leaders are usually the ones who make the tough calls.

Re:Someone always has to make the tough call

[Anonymous Coward]

I'm guessing that you're not an engineer, either. A phrase like "You do not sign-off on a possibly dangerous product." is meaningless.

Every product is POSSIBLY dangerous, and many are absolutely dangerous. If you work for a company that makes transportation products (airplanes, cars, motorcycles, bicycles, etc.) people will die using them. That is guaranteed if they are used in any reasonable number.

Likewise, people will die on the roads that most civil engineers design. People will have adverse reactions to medicine. Some products, like guns, are intended to cause harm.

All of these items could be made safer at some cost, be it monetary, convenience, functionality, etc.

As the parent pointed out, there is a choice to be made between risk and benefit. A straight, divided highway is safer than a winding, 2-lane mountain road. Is the civil engineer who signed off on the mountain road going to be ruined because there will be more fatalities per mile traveled on it than on the divide highway? How could he or she have signed off on such a hazardous road! Far better that the little community at the end of it remain isolated than to build something that we know a priori will cause deaths.

Re:Someone always has to make the tough call

[drinkypoo]

Our government often has to force the matter by making regulations like the ones coming down the pipe soon to require all vehicles to have automatic braking technology.

Too bad that's a garbage example, since the automakers voluntarily chose to implement AEB by that time, without being forced. The example you want is seatbelts. And actually, many government safety mandates are crap. The rears of vehicles are creeping upwards to meet rear impact crash test requirements, that's a natural process. But the fronts of vehicles are being mandated to specific dimensions in the name of passenger safety. Instead of having test requirements to meet, the government is forcing specifications on engineers which may or may not best meet the goals they claim to address, like pedestrian safety.

Re:Someone always has to make the tough call

[AmiMoJo]

In practice what happens is the engineer only certifies for very limited use cases in controlled environments, and the management/sales people push it further. Then a few years later in court the engineer produces their documentation to show that they didn't support it being used that way and tried to warn people of the impending disaster.

Re:Someone always has to make the tough call

[Whibla]

A well thought out and reasoned response to an inflammatory post.

Airbags are an interesting example. Even the best airbag systems kill some people who would not have died without airbags. But they save many more that would have. So, you accept the compromise.

Perfect example!

Realistically, we do wait longer than the point of net balance because the public does not understand statistically-based decisions very well. When it is your family member that died because the tech failed you want to blame the tech without looking at the whole picture. We often don't even know when our family member died because the tech that could have saved them was held back because it was being over-engineered.

Often, these hard decisions are the reason for regulation - not to protect the public but to allow the companies protection in deploying something that a big picture organization like the government has determined will be a net benefit to the public while being a detriment to some individuals. The engineers then have the excuse of having met the regulation. It seems to work better with our minds.

Absent specific regulations and tests to target (which is the ideal situation in a free society), the business leaders are usually the ones who make the tough calls.

And insightful!

Just wanted you to know, your efforts were appreciated. :-)

Re:Someone always has to make the tough call

[Anonymous Coward]

Lawsuit for NOT advancing the state of the art? OK, Then potentially everything can be improved so YOU should be sued as an engineer for not improving EVERYTHING. Why you? Why not? This is essentially slavery by decree and you DO say you are an engineer. Millions don't have abundant and safe drinking water. Clearly you are responsible for not having engineered a solution to this problem. Pencil tips are pointy and can result in an undesirable stabbing. Clearly you are responsible for not having engineered a solution. Ice cream can give a vicious headache. Again YOU are responsible. Is there no end to your negligence?

Your suggestion is clearly absurd and untenable.Of course companies have been sued for not applying tech, but that is simply applying already existing tech. Companies are also sued for inherent risk of products and may endeavor to find remedies to reduce that risk. But that is not the same as suing for not developing tech that makes products safer. Basically the argument that an engineer can be expected to solve virtually any problem regarding safety is unsupportable.

Re:Someone always has to make the tough call

[Anonymous Coward]

Do you not understand what this guy is saying? If you are slightly behind the peak, more people will die if you release the product. If you are slightly ahead of the peak, more people will die if you don't release the product. It is impossible to know where this point lies. Vehicular fatalities are so common that releasing a defective product that will lead to some deaths could still end up saving lives in the long run.

Consider an autonomous driving computer may have a flaw that causes 4 fatal accidents per 100,000. This may be an unacceptably high risk to send to market, especially when any accident involved will be very high profile on the news. Yet, these autonomous cars will not run red lights or stop signs. They will not swerve between lanes and cut people off to make minuscule gains in gridlock traffic. Their near 360 degree view of the world will make it so they never miss pedestrians while trying to make turns at intersections. These autonomous cars will never drive drunk. They will not be susceptible to road rage. Sure, it's bad that this car has a flaw that kills people. Yet, the engineer could very well be leading to more deaths by not releasing the technology.

This is the point autonomous driving is at this moment.

Re:Someone always has to make the tough call

[polar bear]

Excellent post.

Just to add, I'm a UK-based Chartered Engineer and a Fellow of my professional institute (IET, as it happens). I work exclusively in this area (software-intensive safety and security), and have for 20+ years.

And of course I agonise over releases! As you rightly point out, under the parent's policy we wouldnt have air bags, and may other things.

I'm afraid I doubt the parent, and would like to see their experience laid out!

Re:Someone always has to make the tough call

[Anonymous Coward]

Having the license in this country is often career-ending, much like having a PhD. It can make it very difficult to get a job.

Firstly, a PhD is a degree not a license. If it makes it harder for you to get a job it's because they think that you're overqualified and can't afford you.

Secondly, having a PE makes it easier to find new jobs as you immediately stand out among others who can't be bothered to get it. There are quite a few states in the USA where you can't even put the title Engineer on your business card without being licensed. Architect is also restricted in such a way (and it actually takes even longer to get that license.)

That aside a PE doesn't mean that you're an expert, it means that you passed an exam certifying a certain minimum level of competency expected of someone of 4 years of experience and that you have references backing up your experience (4 years minimum under other PEs, 2 years needing to be in a lead role.) It provides some quick evidence when you claim to have X years of experience that you really do rather than really Y number of years X number of times.

Re:Someone always has to make the tough call

[RhettLivingston]

My thought isn't that there should be more suits for not developing tech, but that there should be more suits for not deploying tech that was already reasonably available. Every stage / step toward autonomous driving will save lives and some are already reasonably available.

In a broader fashion though, often, a tech isn't deployed because someone has a patent and is unreasonably charging for its use or even keeping it for their own use at all costs due to a product advantage it gives. I think that could be attacked as having caused an unreasonable death.

A possibly appropriate example recently mentioned in the media is the circular saw safety feature that detects when a blade has contacted skin and instantly stops it. It has been around for more than a decade and would pay for its costs many times over in the cost of the injuries it will stop. However, tools are usually manufactured at a very low-profit ratio - often less than 10%. The owner of the patent has always wanted 8% or more of the gross, not of the profit. Even 8% of the profit is too much to ask for a commodity low margin product.

I think it possible that a lawsuit for an injury that has occurred with a product not using that patented technology could succeed if brought equally against the manufacturer and against the owner of the patent whose unreasonable price (as shown by nobody paying it) has cost the limbs and sometimes lives of hundreds if not thousands of people.

Re:Deep neural nets will never give us full autono

[presidenteloco]

artificial deep neural nets know how to generalize too. What's your point?

Terrible for Tesla if true

[Anonymous Coward]

While not entirely surprising, it's disappointing that Musk has promoted it in the way it has. It's obvious to most people that it shouldn't be though of as a fully autonomous system, for several reasons.

licensed engineers may be need for autodrive softw

[Joe_Dragon]

licensed engineers may be need for autodrive software or something like it.
The FAA does code audits on autopilot software.

Rush to market

[Rick Schumann]

This is a prime example of what I'm talking about when I say so-called 'self driving cars' are being rushed to market.
NONE of them are really ready and won't be for quite some time -- if ever.
Meanwhile people really don't want them anyway.

Re:Rush to market

[ShnowDoggie]

Cherry pick data much?

"However, just over 70 percent would ride in a car that was partially autonomous. Gartner defined partially autonomous vehicles as those that could drive autonomously, but allow a driver to retake control of the car if needed."

That describes the Tesla Autopilot.

Re:Rush to market

[Anonymous Coward]

The only asked if you would ride in one, there's a few other more important questions it would be interesting to know the answers to:

- Would you want to drive an autonomous vehicle?
- Would you want to own an autonomous vehicle?
- Would autonomous features impact your purchase decision?
- How much would you be willing to pay for autonomous features?

Here's my answers:

No, driving is super fun, in my top ten things to do in life.
Not really, just continue to make it harder for me to kill myself or others without sacrificing the fun of driving in any way.
Maybe but not the current offerings, don't want them at all, more likely to cause accidents or kill me in a new a never before seen way.
Zero more dollars, if you paid me I would probably take it but if it causes the car to cost any more than it would otherwise I have zero interest, thanks but no thanks.

Re:Rush to market

[speedplane]

The article you cite says people don't want autonomous cars because they don't trust them. As the technology improves, trust will also improve.

Re:Rush to market

[f00zbll]

It's not that simple right?

If too many people die or get in serious accidents because the technology was deployed before it is ready, there will be a backlash and slow progress. Google has been at autonomous cars longer than any other public company. They didn't rush it out and took a methodical approach. Elon created a problem for himself by having annual hype cycle. He setup the expectation that every year he had to give a big announcement. Elon is his own worst enemy. If he had the discipline to not fall for the hype cycle, and taken a more conservative approach, a person wouldn't have died in a Tesla. If they required the driver to be in the driver's seat and use face recognition to make sure they are paying attention to traffic, the accident could have been avoided. Google's steady approach is the right way to go.

Re:Rush to market

[Rick Schumann]

I do not believe that it will ever 'improve' enough for people to just strap themselves into a seat and hope they don't get killed by a runaway malfunctioning machine. It's basic human nature and it's not going to change anytime in the next 10,000 years. Control over one's own body is one of the things basic to being Human.

Re:Rush to market

[Rick Schumann]

'Partially autonomous' means there are controls and you're doing most of the driving. So long as I can turn the automated system completely OFF so it doesn't interfere with me, that's fine (even if I have to pull a fuse somewhere to accomplish it); in fact it would be a deal-breaker requirement that I be able to completely turn it off. Also I wouldn't pay a single penny extra for any vehicle with these capabilities.

Contradictory

[DontBeAMoran]

Electric cars cool, self-driving cars bad.

If Musk really wants to "save the planet", drop the self-driving crap already. It makes the car more expensive so less people can afford one, meaning they keep their old polluting car or even buy a brand new polluting car.

Re:Contradictory

[speedplane]

Electric cars cool, self-driving cars bad.

If Musk really wants to "save the planet", drop the self-driving crap already. It makes the car more expensive so less people can afford one, meaning they keep their old polluting car or even buy a brand new polluting car.

I disagree, self-driving cars have an even better ability to "save the planet" than electric cars. But I would say "Electric cars cool, self-driving cars cooler, but currently unrealistic." That said, unrealistic is not in Elon's lexicon.

Re:Contradictory

[drinkypoo]

I disagree, self-driving cars have an even better ability to "save the planet" than electric cars.

They really don't, because they're still cars, and cars still suck. PRT would be dramatically superior and it would even permit the existing automakers to continue to exist (just like AVs will) but big changes are scary.

Re:Contradictory

[Anonymous Coward]

How in the world do you justify the statement that electric cars will do ANYTHING at all to "save the planet?" The net impact of self driving vs human driven cars is pretty much zero. Oh, you could imagine some change to human needs, desires and behavior and calculate some smaller number of vehicles that are shared. But that's really not going to happen unless forced upon the populace. This leaves no net effect at this time. Self-driving cars are not an actual functional reality at this time. Cars that are grabbed by a human once the going gets tough or confusing can't really generate reliable safety record comparisons until honesty and rigor is present. It does not appear Tesla meets those two criteria to a degree required to make any proper assessment. Electric cars have a clear and present advantage over IC powered cars as far as environmental impact.

Re:Deep neural nets will never give us full autono

[BasilBrush]

Neural Nets are very specifically NOT rule based. They are trained.
GOFAI was pretty much a phrase invented to label stuff that IS NOT the neural net approach.
Autonomous vehicles do not need AGI. It's very much a single domain system. You don't need your autonomous car to be able to diagnose diseases for example.

He thinks the way we think!!!

[Anonymous Coward]

Musk "brushed aside certain concerns as negligible compared to Autopilot's overall lifesaving potential,"

Popular /. talking point right there. No wonder so many fanboys. Musk is the Donald Trump of tech.

Re:He thinks the way we think!!!

[ElizabethGreene]

I've had three at fault accidents in twenty years of driving, and was lucky enough to walk away from all three.

For some of us the "overall lifesaving potential" isn't a hypothetical concept.

AUTONOMOUS KILLING MACHINE

[Anonymous Coward]

Now I understand.

When Musk opposes autonomous killing machines, he's just trying to reduce the competition. He wants a monopoly!

Re:Deep neural nets will never give us full autono

[Gavagai80]

What if there are two people on different parts of the road in front of the car, and swerving to avoid one will mean hitting the other? The car needs to be able to diagnose which of the two people has a terminal disease, in order to select to hit that one.

Re:Deep neural nets will never give us full autono

[Anonymous Coward]

Adversarial patterns prove that you are wrong.

Re:Deep neural nets will never give us full autono

[Anonymous Coward]

Actually that's false when it comes to new *dangerous* situations. Research has shown that most people's minds take them to what they know when their lives are on the line, even if it runs counter to survival.

Re:Deep neural nets will never give us full autono

[Anonymous Coward]

If the complexity of the tax code keeps increasing then yes, AGI will mark the extinction of Americans, at least, due to how long it will take to actually compute that number.

Re: Deep neural nets will never give us full auton

[Corbets]

Might be useful, actually. You pull over and the hooker gets in. Car scans for disease and automatically hits the eject button before you pay.

Re:Deep neural nets will never give us full autono

[Anonymous Coward]

Oh, trolly problem time. What if the terminally ill person has several years to live, and the other person is suicidal and wants to be run over?

Re:Deep neural nets will never give us full autono

[Anonymous Coward]

LOL.... and what does the training do? Nothing more than program a set of rules which we are not able to even understand or decompose (that's why NNs can't be used in some applications, where the consumer must understand WHY the specific outcome).

Whenever someone starts getting sanctimonious abou

[ishmaelflood]

Whenever someone starts getting sanctimonious about safety I ask them if they fit the best possible tyres to their car for the next journey. If not then they are prepared to sacrifice safety for cost or convenience.

Did you read the 3rd article?

[Anonymous Coward]

3rd article questions if Tesla can see motorcycles:

"Tesla Model S with Autopilot rear ends motorcyclist because the system didn’t “see” the rider"

Re:Deep neural nets will never give us full autono

[speedplane]

Agreed that Neural Nets are not "rule based", but I do think they need far higher level of intelligence than anything we've seen to date. The car is going to need to identify traffic police and follow instructions, they need to interpret human instructions. If they get on the loud-speaker and say "pull-over" vs. "pull-over at the next exit", they'll need to understand that. Not saying it's impossible, just many years off.

Re:Deep neural nets will never give us full autono

[speedplane]

What if there are two people on different parts of the road in front of the car, and swerving to avoid one will mean hitting the other? The car needs to be able to diagnose which of the two people has a terminal disease, in order to select to hit that one.

What if the two people are healthy, but the car can quickly identify which of the two has a higher net worth, so it can hit the poorer one. No moral problems there at all.

I know it's a currently an absurd result, but eventually, the computer will have to decide, or decide to ignore their net worth and use other factors, or to just flip a coin and hit one randomly. Every decision comes with it's own set of moral issues.

Re:Deep neural nets will never give us full autono

[speedplane]

It won't matter because AGI will mark the extinction of humans.

I'm pretty bearish on AGI. To get it in our lifetimes, it'll require continued exponential increases in processing power, i.e., it requires Moore's law to continue. However, many signs currently indicate that Moore's law is dead. It's entirely possible that our civilization may be hitting a technological plateau and that real AGI is not a few or dozens of years away, but hundreds or thousands.

Re:Deep neural nets will never give us full autono

[speedplane]

If the complexity of the tax code keeps increasing then yes, AGI will mark the extinction of Americans, at least, due to how long it will take to actually compute that number.

The tax code is large and opaque to most, but it has a bit of organizational beauty to it. Most people don't understand it because they never studied it. But if you can write software, you can understand tax law.

Re:licensed engineers may be need for autodrive so

[speedplane]

licensed engineers may be need for autodrive software or something like it. The FAA does code audits on autopilot software.

It's not exactly a code audit, it's more like the FAA certifies code to a certain level of robustness. For commercial airline software to get certified, they generally have to prove that every line of code has been covered by tests, and every branch has been taken and not taken. There are even higher levels of certification (usually for the OS), where the code must be symbolically expressed, and mathematically proved to be correct. Not an inexpensive undertaking.

Re:Deep neural nets will never give us full autono

[HuguesT]

If they are well trained on a dataset that covers all the relevant situation, they can interpolate, but they cannot really extrapolate all that well. No known machine learning technique know how to deal with a completely new and unexpected situation.

Re:Deep neural nets will never give us full autono

[HuguesT]

In other words we don't know.

Re:Deep neural nets will never give us full autono

[HuguesT]

More specifically, they are indeed rule based, but the rules are learned, not engineered.

Re:Deep neural nets will never give us full autono

[BasilBrush]

Well that's debatable, and I would come out on the other side of that debate.

But it's neither here nor there, because GOFAI is specifically done with Human readable rules. And there's no one suggesting Neural Nets can be interpreted as that. That is not debatable.

Re:Deep neural nets will never give us full autono

[Anonymous Coward]

Nah, you just hit both of them. Line up on the most open path and hope for the best.

Re:Deep neural nets will never give us full autono

[BasilBrush]

Actually I think you just need to be able to recognise emergency vehicles that have their flashing lights on. Pretty easy as they have lots of markings in addition to their lights.

When one of them is behind you, as a human driver you find a safe place to pull over and slow down so it can pass. If instead of passing it keeps following you, then you know you are the one they're after and if you;re law abiding, you come to a stop.

Automated driving can do the same.

Wow, wait until SpaceX's man-rated rockets

[kriston]

Wow, wait until SpaceX's man-rated rockets.

Customers of those do not take recklessness well.

Re:Deep neural nets will never give us full autono

[Anonymous Coward]

Exactly. Deep learning experts can jump up and down and claim otherwise but they are still doing symbolic AI aka GOFAI. They just figured out a way to automate the production of the rules with fast computers, back-propagation and gradient descent.

Re:Deep neural nets will never give us full autono

[presidenteloco]

"No known machine learning technique know how to deal with a completely new and unexpected situation."

Neither do humans, at least in car-operation time.

Try turning your head upside down and looking at the world. Hard to interpret, isn't it.