Slashdot Mirror
Dashcam Video Shows Tesla Steering Toward Lane Divider - Again (arstechnica.com)
AmiMoJo shares a report from Ars Technica: The afternoon commute of Reddit user Beastpilot takes him past a stretch of Seattle-area freeway with a carpool lane exit on the left. Last year, in early April, the Tesla driver noticed that Autopilot on his Model X would sometimes pull to the left as the car approached the lane divider -- seemingly treating the space between the diverging lanes as a lane of its own. This was particularly alarming, because just days earlier, Tesla owner Walter Huang had died in a fiery crash after Autopilot steered his Model X into a concrete lane divider in a very similar junction in Mountain View, California.
Beastpilot made several attempts to notify Tesla of the problem but says he never got a response. Weeks later, Tesla pushed out an update that seemed to fix the problem. Then in October, it happened again. Weeks later, the problem resolved itself. This week, he posted dashcam footage showing the same thing happening a third time -- this time with a recently acquired Model 3. "The behavior of the system changes dramatically between software updates," Beastpilot told Ars. "Human nature is, 'if something's worked 100 times before, it's gonna work the 101st time.'" That can lull people into a false sense of security, with potentially deadly consequences.
Beastpilot made several attempts to notify Tesla of the problem but says he never got a response. Weeks later, Tesla pushed out an update that seemed to fix the problem. Then in October, it happened again. Weeks later, the problem resolved itself. This week, he posted dashcam footage showing the same thing happening a third time -- this time with a recently acquired Model 3. "The behavior of the system changes dramatically between software updates," Beastpilot told Ars. "Human nature is, 'if something's worked 100 times before, it's gonna work the 101st time.'" That can lull people into a false sense of security, with potentially deadly consequences.
He isn't replicating the situation consistently and it's never been fixed.
We're going to have fully autonomous cars by the end of the year!
Are they sharing the same autopilot dev team?
Tesla's autopilot automatically takes aim at anything the camera doesn't recognize and the Boeing 737-Max autopilot automatically takes a 90 degrees plunge to the ground the moment something abnormal happens.
There are parallels here..
Keep using the flawed thing that killed other people. Hope springs eternal.
It's clear in the video the the Telsa is trying to take the left lane that has that strange signage showing it is closed. When the driver steers back to the right at that point it is heading towards the divider, but the car is trying to take that lane that goes to the left of the barrier. That's different than "the car is trying to steer into the lane divider".
In my 30+ years of driving I have never seen that kind of signage or markers that are apparently used to dynamically close lanes at certain times. I would wonder what I was seeing myself the first time I encountered that.
It looks like two things are going on:
1) The visual system of the Tesla does not understand that signage meaning a lane / offramp has been closed.
2) The GPS routing shows that is a viable route when it is somehow only intermittently open.
Better known as 318230.
This must be the autopilot in the Boeing 737 Max 8!
Laws are rules for the court, but merely a bottom bar to hit for life. Think beyond laws in your actions always.
Not funny. Not in my garage.
Autopilot can do a great job crossing Texas, Kansas, Nebraska, Nevada, the Dakotas, and many more of those mind-numbing straight roads, especially if it doesn't nag you every few seconds like some systems do.
....that autonomous driving is going to work? I mean, have you actually used software? Anything moderately complex has tons of bugs on it. And autonomous driving is extremely complex.
No kidding. It's like Adobe Flash updates of the past. Except now the bug fixes introduce new people killing issues.
list of places least likely to have Tesla owners
Hey, If people are OK putting their lifes in hands of Tesla engineers, traveling at speeds 60 mph+, all more power to them.
I ordered my Model 3 with no autopilot and no self driving. Also, I refused the 30-day trial of autopilot that was being offered.
its AI is just wanting to ends it misery sooner than later.. phoning home...HAL, Joshua, D.a.r.r.y.l.... no one answers.
"Tesla has built Autopilot by training complex neural networks. This makes it difficult to test because no one fully understands how these networks work or can predict how their behavior might change."
Not understanding how these things work when lives are on the line is top notch fuck-headery. Seriously. WTF?
Tesla and just about everyone else in the "autonomous" driving game is using an Expert System. This isn't AI, it is something dug up from the 1970s that sort of modeled what AI could do. Someday.
Well, someday isn't quite here yet. There is no underlying intelligence to these things. It is all based on rules and if you get to the bottom of the list of rules, the car has no idea what to do. This is freaking dangerous.
A true "AI" would have some default precepts, like "don't crash" and "don't hit people". These precepts or basic concepts do not exist in these vehicles. So when something unexpected comes up, there is no rule and no action. So when the roadway presents a situation that isn't recognized, the car will do something unpredictable - or it will do nothing and just keep on going.
Why can't the cars simply be programmed to brake gently to a stop if there is a condition they do not understand? That would require defining what "understand" means. An Expert System has no real intelligence, it is just a series of rules that say when you recognize the conditions of X then do Y. So a basic requirement of this is to load up the computer with every possible scenario that can happen on the road. Anyone that has driven a car for more than a week will tell you that is impossible well, maybe possible but it would take a huge amount of work.
Google had a car that drove into the side of a bus. The response was to add 3600 new rules to the package. Does that illustrate the scope of the problem?
Having a fleet of autonomous vehicles, such as what Uber is thinking of, is foolish and dangerous. Driving on the road with one of these present will present and unlimited capacity for chaos because if something unexpected (or unprogrammed) happens, the car will do something unexpected. And that could be dangerous to everyone around.
Back in 94/95 a friend and I went to a Progfest in LA. My navigator was poring over his paper maps trying to figure out where I should get off. I was in the middle lane of a 5-6 lane freeway, ready to go in either direction at a moment's notice. Keep in mind this is at 70 MPH, surrounded by other cars doing 70. And Ken was a pretty good navigator.
About the time he said "shit!", I said "shi!t" as the freeway split into 2. 2-3 lanes going left, 2-3 lanes going right, and I was on an offramp straight down the middle. Turned out to be the offramp I wanted.
I remember the year(s) because I bought a brand spankin new car in '94, and still considered it brand new for a year after.
AI will be the end of us.
They also can't drive into sunlight, in snow, in rain, in fog, in construction areas, in places with potholes, in places with faded reflective paint, in places where other drivers are idiots, can't discern basic optical illusions, can't figure out what road heat mirages are, can't read text on some road signs...basically self-driving cars are one giant lie and only work under extremely controlled conditions and the technology to drive in even 90% of worldwide driving conditions won't be available for 50 years.
The video clearly shows that the Tesla was in the Ravenna section of Seattle, which is reasonably nice. It was simply trying to avoid heading further south into the lower-class area known as the University District.
#DeleteChrome
The Tesla seems to be a relatively impressive electric car.
The Tesla is not even close to a self driving car in any capacity. Look at the amount of sensors, software on the Waymo vehicles and they're still not finished.
The Tesla is a 'toy' automated vehicle. Using this feature is dangerous and foolish. Leave it as an electric vehicle, not an autonomous vehicle in any capacity. I'm shocked more people aren't dead due to this.
I know. I for one will only buy cars where I have a lower chance of survival and less safety features. None of this guinea pig stuff.
I know. This YouTube video explains everything about Musk and how he affects people: https://www.youtube.com/watch?...
Many of the people who buy Teslas are just tech nerds who got lucky in IT and made some money and blow it on a $60,000+ car. They like tech toys. Musk figured it out and now he has worshippers throwing money at him (until the recession hits).
As they "fix" the problem for situation A, they break it for B. And then when they fix B again, A breaks. And since they have no Q/A from what I can tell, it never will get fixed as their sensor set/processing capability is not up to both A & B.
While substantial reward$s fathomable if able to iffy autonomous transport , it is monumental undertaking. There are simpler incremental safety and efficiency tech solutions that could help in near term such as drive recorders , smart roads that can share road/traffic conditions , monitor dangerous drivers etc... Smarter roads can help autonomous driving. But since financial risk / rewards dispersed less investment. Still transportation getting better. The ride hailing app investors are subsidizing a transition to Sharing idle assets. Car sharing/pooling will get better as cost of ownership management improves. Meanwhile pumping venture money into Engineering still helpful for economy, since they will spin off into other areas that might seem more practical.
Tesla's poorly named Autopilot is not an autonomous driving system, it is a driving assist! Some people don't seem to understand that. Too many people probably don't take the time to learn to properly use Autopilot, or learn when it is best NOT to use it. My only complaint is that Autopilot lets you take your hands off the wheel for a couple of minutes before warning you...it should be alarming the instant that no hands are on the wheel! And after 10 seconds of no hands on the wheel, the car should start looking for a safe place to pull over and park itself, while notifying law enforcement!
"Human nature is, 'if something's worked 100 times before, it's gonna work the 101st time.'" That can lull people into a false sense of security, with potentially deadly consequences.
You got that right.
When you are dealing with AI, and it gets retrained, it MUST be retested fully.
And it appears that this edge-case is not being tested.
I prefer the "u" in honour as it seems to be missing these days.
If that was an attempt at sarcasm, you failed miserably. Get your head out of Musk’s ass and breathe the fresh air. The statistics you’re referring to are very misleading and hand-picked
So that's why you always lie.
Unfortunately for you, people caught on after the 100 times, but you still think you can get away with it.
Did you see Elon Musk "demonstrating" Autopilot on 60 Minutes? He sure wasn't acting like it was driver assist with both hands off the wheel ignoring the road. Tesla fans are in for a very hard fall.
1. It doesn’t have to handle every situation, it just has to know when to give up and ask for help.
2. It doesn’t have to be perfectly safe, it just has to be demonstrably safer than humans.
3. Every time any Tesla encounters an exceptional situation, the SW gets altered to deal with that, and then *every* Tesla gets better. That’s exponential improvement.
Time for
Ralph Nader to write an Unsafe at Any Speed 2 auto ride of death.
There are hundreds of millions of cars on the road with no sensors at all other than two human eyes. I’m not sure why a biological neural net can drive on two human eyes but a digital neural net needs 75 times of radar.
When you are dealing with AI, and it gets retrained, it MUST be retested fully.
Not quite right. You are assuming that the machine learning technique involved suffers from Catastrophic Forgetting upon re-training. This was a problem back in the early days of machine learning, but any modern AI engineer and researcher knows of this problem and is or will be implementing solutions.
When a human learns to fly a Cessna, we get a pilot's license. When we get a type certificate to fly an Airbus after learning to fly the Cessna, we don't forget how to fly the Cessna and need re-training in the Cessna.... unless we don't fly a Cessna for a long time. Humans are engineered to forget, which is fundamentally important for being human, but not at all important for a control system.
It's very clear signage, and I don't believe you've never seen it before. It's also completely irreverent to the accident. It's driving into a concrete pillar.
The car is clearly treating the white line (which would become the *right* edge of the closed off road) as the *left* edge of its lane, leading it to drive straight into the lane divider. It isn't seeing the lane divider at all.
It also wouldn't correctly enter the left lane even if it was open. It would drive into the concrete divider.
The marking makes no difference, if a piece of concrete fell into road, I'd expect a driver to stop, regardless of markings. So should the car.
I'm a pilot. I fly a plane with an autopilot. I also drive a Tesla with their "autopilot".
The very expensive aircraft autopilot flies great. I can be hands-off the controls for extended periods of time, read a book, browse Facebook (hurrah for GoGo :), etc. Do I? Hell no! An aircraft autopilot has no clue what other aircraft are doing. TCAS might see another nearby aircraft, maybe it won't. I keep my hands on or near the controls, I look out the window, and I scan the instruments - all the time. Which is pretty much what I do in the Tesla. The big difference is that the Tesla actually does a pretty decent job of reacting to other cars. Odd lane markings and construction zones do freak it out from time to time. I have had the Tesla alert me to an unsafe traffic or road condition and tell me to take over - in a flurry of beeps and on-screen alerts. Freaks me out every time. I wish the autopilot in the airplane would do that - instead it just shuts off, throws a warning light if I'm lucky, and the plane wanders off somewhere in the sky until I pull head of my my ass. I probably hand-fly the airplane more than I hand-drive the Tesla - on cross country trips. Taxiing around on the ground is a bit like driving a Tesla to the grocery store - an annoying fact of life to tolerate only until I get where I belong - out on the road, or up in the air, where the massively automated systems not only make my life easier, they make it safer as well.
You people bitching about how dangerous the Tesla autopilot is are just spoiled, bitchy little meat bags of self-loading cargo. You have no concept of automation, risk, and capability, you see the autopilot and cry that it's not perfect. You all need to fly from LA to NYC in a Ford Trimotor, or drive between them in a model T. Keep a spare set of points and a condenser in the glovebox. The magneto on the Trimotor's radial engines probably uses the same points as the Model T. Make sure you can change the points and gap them in the middle of nowhere, because that's where they'll fail. You'll be flying for about 20 hours, and you'll make about 8 stops for fuel and maintenance. The Model T will take a wee bit longer, at least 60 hours, with modern roads, unless you have to stop and fix the engine. A model 3 can make that drive in 50 hours, and you won't have to change the points once.
For almost 200 years, railroads have allowed vehicles to avoid obstacles by:
a) confining them to pre-fixed paths
b) clearing thoses paths of obstacles
I think this would be the solution for autonomous cars too!
You do not have to run them on steel rails, but could confine them to preapproved lanes on Highways.
The easiest way to achieve that would be to paint "middle of the lane" demarcations in the form of continuous barcodes on highways cleared for automated driving. Every couple of yards, these barcodes would repeat a global 48 bit "lane identifier" and a current position on that lane.
In order to allow the car to drive on that lane, it has to establish a radio link to a central databank to inquire which lanes on the highway the car is driving on are currently cleared for traffic and not closed down for construction or reported to be blocked by an obstacle by previous cars passing the spot, or even an automated car that broke down.
This would eliminate 99% of the problems, and leave just the much more manageable task of scanning the lane ahead with lidar, radar and cameras for obstacles like slow or standing cars not yet known to the system.
If any such new obstacle is detected, the car hands over to the driver or slows down, just as it would if the barcode is not legible.
If Snow, dirt or heavy rain obscures the barcode, the no automated driving today!
This would be small step toards the final solution, but with a comparatively small cost of implementation and a fairly low complexity.
Japan actually banned them from doing tests on customers. Tesla cars in Japan have old versions of the software because the regulator realized it was incredibly dumb to do constant over-the-air updates that alter the behaviour of the car and which have not been certified or properly tested.
const int one = 65536; (Silvermoon, Texture.cs)
SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
Its incredibly dumb to try to play a game (especially with your life) where someone continously changes the rules. Period.
Especially when the (changing of the) rules are not communicated at all.
Would you play a poker game like that ? I certainly wouldn't. Not as long as I have a choice in it that is.
There is an old joke: "If cars where build like Computers than {something bad, happening regulary}". It looks like the joke is on us now. :-(
....pay me to buy these kind of cars.
A system that produces an audible warning if the driver drifts away from the middle of the lane makes some degree of sense. I think if you need that, the correct response is to find an exit and take a break; so I guess these have a purpose as a tired driver alert system.
What is the purpose of automatically staying in the lane? The driver is still obliged to pay attention. There doesn't seem to be any more cognitive load to actually turning the steering wheel. All this does is remove that warning that you might be too tired.
This is the reality of autonomous cars. The fact is that these systems will have issues that will get people killed. However, I think that the total number of people killed over a time period will be much lower. The question is do you trust your life to statistics, or to take your life in your own hands. You may be a near-perfect driver but are still at risk for terrible drivers killing you. Do you give up your perhaps better than average chance of living due to your excellent driving, or give up control so that the terrible drivers also have to give up control?