Slashdot Mirror
Dashcam Video Shows Tesla Steering Toward Lane Divider - Again (arstechnica.com)
AmiMoJo shares a report from Ars Technica: The afternoon commute of Reddit user Beastpilot takes him past a stretch of Seattle-area freeway with a carpool lane exit on the left. Last year, in early April, the Tesla driver noticed that Autopilot on his Model X would sometimes pull to the left as the car approached the lane divider -- seemingly treating the space between the diverging lanes as a lane of its own. This was particularly alarming, because just days earlier, Tesla owner Walter Huang had died in a fiery crash after Autopilot steered his Model X into a concrete lane divider in a very similar junction in Mountain View, California.
Beastpilot made several attempts to notify Tesla of the problem but says he never got a response. Weeks later, Tesla pushed out an update that seemed to fix the problem. Then in October, it happened again. Weeks later, the problem resolved itself. This week, he posted dashcam footage showing the same thing happening a third time -- this time with a recently acquired Model 3. "The behavior of the system changes dramatically between software updates," Beastpilot told Ars. "Human nature is, 'if something's worked 100 times before, it's gonna work the 101st time.'" That can lull people into a false sense of security, with potentially deadly consequences.
Beastpilot made several attempts to notify Tesla of the problem but says he never got a response. Weeks later, Tesla pushed out an update that seemed to fix the problem. Then in October, it happened again. Weeks later, the problem resolved itself. This week, he posted dashcam footage showing the same thing happening a third time -- this time with a recently acquired Model 3. "The behavior of the system changes dramatically between software updates," Beastpilot told Ars. "Human nature is, 'if something's worked 100 times before, it's gonna work the 101st time.'" That can lull people into a false sense of security, with potentially deadly consequences.
It's clear in the video the the Telsa is trying to take the left lane that has that strange signage showing it is closed. When the driver steers back to the right at that point it is heading towards the divider, but the car is trying to take that lane that goes to the left of the barrier. That's different than "the car is trying to steer into the lane divider".
In my 30+ years of driving I have never seen that kind of signage or markers that are apparently used to dynamically close lanes at certain times. I would wonder what I was seeing myself the first time I encountered that.
It looks like two things are going on:
1) The visual system of the Tesla does not understand that signage meaning a lane / offramp has been closed.
2) The GPS routing shows that is a viable route when it is somehow only intermittently open.
Better known as 318230.
The only thing more hubristic than assuming something will definitely work is assuming something will never work.
Of course autonomous driving software will have bugs in it, and those bugs will lead to accidents. The status-quo alternative (biology-based driving software) also has bugs in it, which regularly leads to accidents.
The difference is that bugs in the autonomous driving software will eventually be diagnosed and fixed. Bugs in biological driving software, OTOH, will never be fixed, because every new person has to learn to drive from scratch; even if someone eventually becomes a flawless driver, sooner or later that person will die and replaced by another newbie, who will repeat the same newbie mistakes as everyone else. Lessons "learned" by software (and software designers) OTOH, can stay "learned" indefinitely, as long as they don't lose the source code.
I don't care if it's 90,000 hectares. That lake was not my doing.
Duh. That's why anyone with a brain knows these things are deathtraps.
You can't debug a NN, not in any reasonable manner, certainly not one that you're constantly retraining and tweaking all the time. In this case, even providing heuristics ("Hey, there's a bridge near this GPS location, so don't think it's a wall" is literally what Tesla are putting into their software in some places because they can't train the behaviour out of the NN).
This has always been the concern of anyone that deals with such stuff since Tesla said they were using that technology.
You're basically training a black box on unknown criteria from limited test data, and then acting shocked when people say they don't understand how the black box works, can't predict what it will do, can't retrain or untrain it easily, and are surprised that even a million miles of road data aren't enough to let it drive safely across the entire world in perpetuity?
I'm a pilot. I fly a plane with an autopilot. I also drive a Tesla with their "autopilot".
The very expensive aircraft autopilot flies great. I can be hands-off the controls for extended periods of time, read a book, browse Facebook (hurrah for GoGo :), etc. Do I? Hell no! An aircraft autopilot has no clue what other aircraft are doing. TCAS might see another nearby aircraft, maybe it won't. I keep my hands on or near the controls, I look out the window, and I scan the instruments - all the time. Which is pretty much what I do in the Tesla. The big difference is that the Tesla actually does a pretty decent job of reacting to other cars. Odd lane markings and construction zones do freak it out from time to time. I have had the Tesla alert me to an unsafe traffic or road condition and tell me to take over - in a flurry of beeps and on-screen alerts. Freaks me out every time. I wish the autopilot in the airplane would do that - instead it just shuts off, throws a warning light if I'm lucky, and the plane wanders off somewhere in the sky until I pull head of my my ass. I probably hand-fly the airplane more than I hand-drive the Tesla - on cross country trips. Taxiing around on the ground is a bit like driving a Tesla to the grocery store - an annoying fact of life to tolerate only until I get where I belong - out on the road, or up in the air, where the massively automated systems not only make my life easier, they make it safer as well.
You people bitching about how dangerous the Tesla autopilot is are just spoiled, bitchy little meat bags of self-loading cargo. You have no concept of automation, risk, and capability, you see the autopilot and cry that it's not perfect. You all need to fly from LA to NYC in a Ford Trimotor, or drive between them in a model T. Keep a spare set of points and a condenser in the glovebox. The magneto on the Trimotor's radial engines probably uses the same points as the Model T. Make sure you can change the points and gap them in the middle of nowhere, because that's where they'll fail. You'll be flying for about 20 hours, and you'll make about 8 stops for fuel and maintenance. The Model T will take a wee bit longer, at least 60 hours, with modern roads, unless you have to stop and fix the engine. A model 3 can make that drive in 50 hours, and you won't have to change the points once.