Slashdot Mirror
Elon Musk Rolled Out Autopilot Despite Engineers' Safety Concerns, Says Report (theverge.com)
An anonymous reader quotes a report from The Verge: When Elon Musk announced last fall that all of Tesla's cars would be capable of "full autonomy," engineers who were working on the suite of self-driving features, known as Autopilot, did not believe the system was ready to safely control a car, according to the Wall Street Journal. The WSJ report sheds more light on the tension that exists between the Autopilot team and Musk. CNN previously reported in July that Musk "brushed aside certain concerns as negligible compared to Autopilot's overall lifesaving potential," and that employees who worked on Autopilot "struggled" to make the same reconciliation.
A major cause of this conflict has apparently been the way Musk chose to market Autopilot. The decision to refer to Autopilot as a "full self-driving" solution -- language that makes multiple appearances on the company's website, especially during the process of ordering a car -- was the spark for multiple departures, including Sterling Anderson, who was in charge of the Autopilot team during last year's announcement. Anderson left the company two months later, and was hit with a lawsuit from Tesla that alleged breach of contract, employee poaching, and theft of data related to Autopilot, though the suit was eventually settled. A year before that, a lead engineer warned the company that Autopilot wasn't ready to be released shortly before the original rollout. Evan Nakano, the senior system design and architecture engineer at the time, wrote that development of Autopilot was based on "reckless decision making that has potentially put customer lives at risk," according to documents obtained by the WSJ.
A major cause of this conflict has apparently been the way Musk chose to market Autopilot. The decision to refer to Autopilot as a "full self-driving" solution -- language that makes multiple appearances on the company's website, especially during the process of ordering a car -- was the spark for multiple departures, including Sterling Anderson, who was in charge of the Autopilot team during last year's announcement. Anderson left the company two months later, and was hit with a lawsuit from Tesla that alleged breach of contract, employee poaching, and theft of data related to Autopilot, though the suit was eventually settled. A year before that, a lead engineer warned the company that Autopilot wasn't ready to be released shortly before the original rollout. Evan Nakano, the senior system design and architecture engineer at the time, wrote that development of Autopilot was based on "reckless decision making that has potentially put customer lives at risk," according to documents obtained by the WSJ.
Engineer behavior outside of competence is indistinguishable from MBA behavior.
But the engineers are supposed to know better. The MBAs are _trained_ that 'competence is irrelevant' they can manage anything.
John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
The software engineer will deploy it at some given milestone and then work out the bugs. The MBA will re-define and accelerate the milestone.
This is MBA behavior, not engineer behavior.
That's funny, because when I read this the first thing I thought was that he sounded like one of the new-school engineers. You know, the ones with the motto "move fast and break things".
anyone who's tried to execute a change or deliver an outcome will always find one or two dissenting voices in any organisation of scale.
Absolutely true. And it's equally true that it's foolish to not take those dissenting opinions very seriously (even if, after careful consideration, they don't change your plans).
In any organization, there is a strong "rah-rah" tendency, and people tend to suppress their own doubts. Nobody wants to be a wet blanket or potentially risk their career by not seeming to be a "team player". So the voices of those who point out problems need to be listened to much more carefully than the voices of those who say "everything's great".
Nobody wants self-driving cars?
That's hardly the only time. There was an incidental in China where autopilot drove full speed into a road sweeper that it apparently couldn't see.
Tesla seem to have admitted it doesn't work as originally advertised, by repeatedly increasing the amount of effort it makes to keep the driver alert.
const int one = 65536; (Silvermoon, Texture.cs)
SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
Better yet don't call it autopilot. Even if you try to explain it, there are plenty of fools who won't get it. Call it "drive assist" and people might be a little less foolish. Some assholes will still misuse it, but you can't stop someone hellbent on stupidity.
The guys who program the self driving features are probably different from the guys who are working in materials science trying to improve the batteries. You might be able to move funds from one department to the other, but my guess is that Tesla already has the best minds in the right places, and they would hit diminishing returns trying to bring in fresh blood. Not that there isn't more talent out there, but that Tesla already has as much talent as it can support efficiently. Hiring more now would lead to diminishing returns. Beyond hiring and moving funds around, I don't know what else you expect an organization like Tesla to do to encourage development in specific areas.
I don't have the exact words Musk has used, but I distinctly remember that he said that all Teslas will come equipped with the HARDWARE necessary for fully autonomous self-driving (computer power, sensors), but that the actual functionality would depend on a future software upgrade.
Now you and I, as software-related techies most of us, know that that will have to be one MASSIVELY COMPLEX and not really invented yet by any stretch of the imagination software upgrade, but technically, what he said is not false.
Where are we going and why are we in a handbasket?
"Is it safe?" is the wrong question.
The real question is: "are the fatalities, injuries and accidents that occur per passenger mile greater or less when compared to a human driver?"
"Safe" is a subjective and unmeasurable term, unless you define it in unreasonable terms (for example if your definition of safe is zero accidents).
The real "Libtards" are the Libertarians!
I've been in engineering organizations releasing new products that had life saving or threatening potential. It is always an agonizing, scary hard call as to when you've passed the threshold of risk.
There is a bell curve with a peak. You rarely hit the peak. If you make the call too late, you cost the lives of those you might have saved - too soon, you cost lives of those who might have saved themselves.
Even if you hit the peak perfectly, you'll always be able to truthfully argue that some people are being saved who would have died and some are dying who would have lived. The peak is a point of balance between the two - not a perfect elimination.
I can remember many times hating my bosses when they released a product that I didn't feel was ready. As an engineer, I have to be over-focused on the problems and stand no chance of seeing when I am perfectly perched on that probability peak. They had to pry the projects from my hands to get them out the door. I actually begged in tears once. But, in retrospect, I can't think of any case where my bosses weren't right in releasing the product that I was concerned about releasing.
What we need to force progress is for attorneys to get smart and start figuring out how to file more effective suits for lack of progress toward autonomy. How many are dying today because we don't have it? We need to focus hard on that.
You assume Musk's motives are about selling cars... I'm not so sure that's true.
I actually think that Musk's driving force is more about PR than running any of his business ventures the most productive way possible. I suspect that he craves the attention that comes from having that flashy idea, and the money that comes from the starry eyed investors who flock to his door to "invest" in them. I don't think he's a snake oil salesman, only that he's not opposed to throwing plausible ideas up on the wall and see what kind of attention sticks to it.
In short, he only really cares about the attention. Which doesn't mean he's not about turning a profit, he wants that too, but because it brings him positive attention. Which, in Musk's case, hasn't been a bad thing for him.
Think Howard Hughes, only as an extrovert...
Of course.. Your mileage may vary...
"File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
And autopilot for planes is far, far, far more advanced, capable, robust, and reliable
That's mostly because it is also much simpler. The comparison with it is correct, it just omits the part where you have to additionally recognize roads, vehicles, traffic signs, pedestrians...all the things airplanes don't have to worry about. Airplane-level capability just isn't enough.
Ezekiel 23:20
No competent engineer, new or old, has ever uttered those words or advocated what they represent. Such a motto only works when you're involved in shit that doesn't really matter. In other words, it's perfect for Facebook, or a small Google team working on a new project that will be abandoned as soon as it's acquired a few loyal users, or a Silicon Valley startup no one's heard of writing an app that no one cares about.
Irony: Agile development has too much intertia to be abandoned now.
It's a lot simpler than that:
This article is bullshit.
Sorry to be so blunt, but it's journalistic malpractice. The author is confusing Enhanced Autopilot (EAP) with Full Self-Driving (FSD). To be clear:
* Some safety features related to autopilot, such as automatic braking and the like, are available to everyone for free.
* EAP is an optional add-on available today ($5k on the Model 3 if purchased at the time of buying the vehicle, $6k as an over-the-air upgrade) which provides lane-following (requires hands on the wheel and driver attention) and driverless summon features (very low speed, "back out of / into a tight space / drive down the parking lot" stuff without a driver in the car). More to the point, there are two entirely different versions that have existed over the year. AP 1.0 was used on earlier vehicles, based on software and hardware from Mobile Eye. Tesla and Mobile Eye split in a contract dispute. Mobile Eye claims that Tesla wasn't using their hardware right. Tesla says that Mobile Eye found out that Tesla was working on an in-house Autopilot system and demanded that they stop as a precondition to get to continue to use their hardware. Mobile Eye says they knew about Tesla's internal work but didn't feel threatened by it. Regardless, Tesla was forced to switch to their internal version, AP 2.0, which was a step backward. AP 2 is just now catching up to the features of AP1.
* FSD is Tesla's current goal, where the vehicle can drive itself without you having to have your hands on the wheel or paying constant attention. You cannot use FSD, even if you buy it. It costs $3k on the Model 3 ($4k as an over-the-air upgrade later). The article is talking about FSD being rolled out before engineers think it's ready. To reiterate: you can only buy FSD right now, you can't use it until it's ready. Tesla apparently tried to clarify this for the author:
The author apparently nonetheless still failed to understand what that means. You Cannot Use FSD. Period. If engineers are complaining about FSD being rolled out too soon, they're complaining about Tesla selling something that its drivers aren't going to be able to use for too long of a period of time. And you know what, I fully agree with the engineers in that regard - I think it's wrong of Tesla to sell something that there's a big question as to whether they'll be able to get it working reliably enough or pass the serious regulatory barriers in its way.
But if engineers are complaining about FSD, then it's not complaints about EAP. Because the two are very distinct things. EAP isn't perfect, don't get me wrong - and the 1.0 / 2.0 switch was a big setback (they still don't use all of the cameras on the vehicle). But it also pesters drivers enough if they show signs of not paying attention to the road (e.g. not holding onto the wheel) in order to overcome its imperfections (the level of pestering was significantly increased after AP1's fatal accident, in which the driver was apparently watching movies during most of his trip).
Ever since, I've been suspicious of Jesus and very careful around chlorine.
Chris Lattner wasn't there long enough to get started. We don't know why be backed out.
Personally I could never see why a compiler guy was being hired as head of one of the most complicated AI projects anyway. Different field.
Neural Nets are very specifically NOT rule based. They are trained.
GOFAI was pretty much a phrase invented to label stuff that IS NOT the neural net approach.
Autonomous vehicles do not need AGI. It's very much a single domain system. You don't need your autonomous car to be able to diagnose diseases for example.
And autopilot for planes is far, far, far more advanced, capable, robust, and reliable than the shit Elon is selling.
That's total bullshit. Autopilot for airplanes has been around for many decades now. It just maintains a heading and altitude. It's roughly analogous to cruise control on cars in technological terms, and maybe automatic lane-keeping in actual functional terms (since cars have to follow a road, planes don't; of course, technologically, lane-keeping is far, far, far more advanced than the autopilot in a typical Cessna).
Yes, there are very advanced autopilots in today's newest passenger planes like the 787, but the term is not exclusive to those, and there's countless decades-old Cessnas and Pipers out there with autopilots that are quite primitive.
No, autopilot in planes does not autonomously pilot the plane. It doesn't take off or land, it doesn't fly around bad weather, it doesn't check METARs and PIREPs, it doesn't watch for other traffic, it doesn't handle radio calls to ATC when you cross into class B airspace, it just keeps you flying straight and level.
The only thing in your post that's correct is the bit about pilots being trained to use their equipment. That isn't true for Teslas, but it also isn't true for any other car either. How many drivers on the road today got explicit training to use the cruise control in their car? Cruise control has been around a few decades now too. Or how about the more advanced features we have not, like adaptive cruise control, lane-keeping, and operating the infotainment system? Every car is different, with different controls and different quirks. Airplane pilots aren't even allowed to fly a plane (solo) unless they've been specifically trained for that model and received a rating for it. Perhaps we should do that for cars....
Airline pilots are intelligent and highly trained individuals. That is why they are not found on every street corner, and are worth more than a dime a dozen.
Actually, this isn't true. Pilots start out their careers as instructors ("those who can, do, those who can't, teach"), and make peanuts. After that, they might get a job as a copilot for a regional jet company. Last I heard, the starting salary for one of these guys is $18k. Yep, barely above minimum wage. It takes many years for them to work up to any kind of decent salary approaching 6 figures. Then, when they hit 60 years old, they're forced to retire.
Being a pilot is for people who are independently wealthy (e.g. trust fund, or has a spouse willing to support them), or for people who love it so much they're willing to sacrifice everything just to have that job.
They implied that Tesla is currently having people drive something that its engineers deem unsafe. This is simply not the case at all. If the engineers were complaining about selling FSD, they're not complaining about anything that consumers are actually driving.
Everyone who buys FSD does so on their assessment of how likely they think it is that Tesla will actually deliver. There is zero confusion among anyone who buys it about the fact that they can't use it right away; the option always includes the "you can't use this until it's finished and legally approved" disclaimer next to it. It all comes down to how optimistic or pessimistic you are about the technology. I'm a pessimist, and will not be buying it. Some of Tesla's engineers working on it are apparently also pessimists. I'm not surprised. It's a crazy-hard task, and very different from human-supervised autosteer / EAP.
Ever since, I've been suspicious of Jesus and very careful around chlorine.