Waymo Built a Fake City In California To Test Self-Driving Cars (arstechnica.com)

An anonymous reader quotes a report from Ars Technica: In May, Waymo revealed key details of its latest self-driving car design to Bloomberg as part of the rollout of a new program that ferries ordinary passengers around in Phoenix. Now The Atlantic's Alexis Madrigal has a new piece revealing important details about Waymo's extensive infrastructure for testing self-driving cars. Madrigal reports on two Waymo projects that haven't been previously made public. One is an extensive virtual city in California, 100 miles east of Silicon Valley. Named Castle after the former Castle Air Force Base, the facility hosts a network of private roads for testing self-driving vehicles. It's a proprietary cousin of Mcity, the open vehicle testing facility we visited in 2015. At the Castle facility, Waymo builds replicas of real intersections -- like a two-lane roundabout in Texas -- that have given Waymo cars trouble.

Having their own extensive network of private streets allows Waymo engineers to perform repetitive tests to observe how Waymo's software reacts in carefully controlled situations. In one series of tests seen by Madrigal, another car cuts off a Waymo car at a variety of speeds and angles. The tests were designed to help engineers calibrate how hard cars brake in these kinds of situations. Brake too slowly and there's a risk of a crash. Brake too hard and passengers will get whiplash. The Castle team has amassed an extensive collection of props -- traffic cones, tricycles, fake plants, dummies -- that help simulate a wide variety of road situations.

Comments

good for them...test and test again...data

[turkeydance]

how do they simulate snow?

Re:good for them...test and test again...data

[TWX]

Well, automakers simulate ice at the various desert proving grounds by wetting-down roads they've covered in ceramic tiles. Throw a layer of something foamed on top and you've probably got something that behaves like snow over black ice.

Re:good for them...test and test again...data

[Knuckles]

I don't think ice on the ground is the problem, but falling snow

Self driving cars will be hacked

[craXORjack]

Someday terrorists will upload a zero day worm that spreads from car to car turning our quiet city streets into Death Race 2000. (David Carradine RIP)

Re:Self driving cars will be hacked

[slew]

Someday terrorists will upload a zero day worm that spreads from car to car turning our quiet city streets into Death Race 2000. (David Carradine RIP)

I was thinking Maximum Overdrive myself...

Re:Self driving cars will be hacked

[epyT-R]

Planes and their communications equipment are not easily accessible (assuming they are even allowed be flown remotely) from 30000 feet, and their ground time is under guard. In addition, planes still have two pilots watching the boards ready to take control when necessary. Would be self driving cars and their network transmissions are a different matter. Also, autopiloting an aircraft that essentially just has to hold a straight (well parabolic) course is a much easier problem to solve than something that would navigate a busy city.

Nice

[Anonymous Coward]

This disclosure gives me some comfort that they at least have a non-production test environment :) and I like that they built specific scenes replicating places and incidents where they had reported problems... good approach.

Maybe build a catalog of unique features and road incidents and let all manufacturers test against it (for a fee) so we can build confidence in everyone's models.

Cars still need work

[TWX]

Saw a funny one a couple of weeks ago. A Waymo car (technically a Pacifica minivan) was essentially stuck. The street had a median and a maintenance crew had closed the left lane with cones, and they had a wood chipper trailer and truck, and the various workers were cutting apart trees that had tipped over and were dragging pieces over to the chipper's hopper, which faced oncoming traffic.

The Waymo car was up at the start of the cones, and it couldn't interpret what the workers were doing and couldn't figure out how to merge-right. I think it was assuming the workers were going to run out into the street, so it would begin to move right but would stop as soon as a worker moved, and the rest of the vehicle traffic that was moving-right was just thick enough that it couldn't manage to get moving in those few seconds when workers were not being interpreted as an obstruction.

It was pretty funny to watch, and the employee in the car gave it a good ten minutes before giving up and manually taking control.

Re:Cars still need work

[diesalesmandie]

Saw a funny one a couple of weeks ago. A Waymo car (technically a Pacifica minivan) was essentially stuck. The street had a median and a maintenance crew had closed the left lane with cones, and they had a wood chipper trailer and truck, and the various workers were cutting apart trees that had tipped over and were dragging pieces over to the chipper's hopper, which faced oncoming traffic.

The Waymo car was up at the start of the cones, and it couldn't interpret what the workers were doing and couldn't figure out how to merge-right. I think it was assuming the workers were going to run out into the street, so it would begin to move right but would stop as soon as a worker moved, and the rest of the vehicle traffic that was moving-right was just thick enough that it couldn't manage to get moving in those few seconds when workers were not being interpreted as an obstruction.

It was pretty funny to watch, and the employee in the car gave it a good ten minutes before giving up and manually taking control.

This is a good example of where current machine learning (what people incorrectly call AI) doesn't work. Sure, in a discrete solution space like the go game (a LOT of different solutions, but still a discrete solution space) machine learning can beat the best human, but, in a semi-discrete, semi-continuous solution space like traffic (semi-discrete in the sense that the vehicle is bounded by the road, semi-continuous in the sense that a hazard/obstruction/malfunction/misinterpretation could be anything a.k.a. real life) it's just not good enough yet.

I don't claim to be a expert, i'm just giving my (not recently researched) opinion. Feel free to mod me into oblivion, if you think i'm talking through my ass.

Re:Cars still need work

[TWX]

I don't think that you're talking through your ass in this case. Human drivers, and arguably human beings in general both learn how to predict what will happen, and when to take risks if their predictions are less than certain. This doesn't mean that humans are immune to making errors as the auto accident statistics strongly demonstrate, but in this case a human driver probably would have seen the lane restriction and the reason for it from further out, and even if not (like following a large vehicle that chose to merge at the last moment) the human driver would recognize what the people in the median are actually doing and judge it reasonably safe that they're not going to walk out in front of traffic without warning.

In short, the human driver is going to call on his life experiences while driving in order to attempt to predict the behavior of his surroundings. Even a novice fifteen year old with a learner's permit has fifteen years of learning about some forms of human behavior, and would probably recognize a worker with a hardhat at a wood chipper as someone unlikely to cross into the right-of-way. A computer system doesn't learn in that fashion.

Re:Cars still need work

[kackle]

The Waymo car was up at the start of the cones, and it couldn't interpret what the workers were doing and couldn't figure out how to merge-right.

Ad infinitum...

The delirious, ignorant enthusiasm for autonomous vehicles ("Look mom, no steering wheel!") has been rolling over common sense, particularly on this website. There is an INFINITE number of "unusual" situations that a driver will encounter, situations that a human can easily comprehend. In fact, I started writing these down about 2 years ago just to see; I experienced about one abnormal situation per month during my short commute of several miles.

There was another thread on Slashdot discussing whether an autonomous vehicle (AV) would drive over a tree branch in the road. This is a great point because it is SO subjective: How large is the tree branch? Can my car clear it? Is it the kind of branch that will up-end under my car and get stuck? Can the AV even estimate the tree branches' density due to the leaves? A human can make a good guess. An AV will be baffled by such a decision, no matter how much programming. Want a real-life example? There was a big storm that rolled through my town; afterward, there were branches down all over the place. On one street, where there are no trees near the road, what do you think this branch I saw really was? A downed power line - I turned the car around.

As a firmware developer, I predict that these will be forced onto the road/us before they're ready, because I think they can never be ready. I predict this will slow all but highway traffic as the software engineers and car manufacturers use our daily commutes, and lives, has their own personal test beds.

Re:Cars still need work

[diesalesmandie]

I don't think that you're talking through your ass in this case. Human drivers, and arguably human beings in general both learn how to predict what will happen, and when to take risks if their predictions are less than certain. This doesn't mean that humans are immune to making errors as the auto accident statistics strongly demonstrate, but in this case a human driver probably would have seen the lane restriction and the reason for it from further out, and even if not (like following a large vehicle that chose to merge at the last moment) the human driver would recognize what the people in the median are actually doing and judge it reasonably safe that they're not going to walk out in front of traffic without warning.

In short, the human driver is going to call on his life experiences while driving in order to attempt to predict the behavior of his surroundings. Even a novice fifteen year old with a learner's permit has fifteen years of learning about some forms of human behavior, and would probably recognize a worker with a hardhat at a wood chipper as someone unlikely to cross into the right-of-way. A computer system doesn't learn in that fashion.

Agree

Re:Cars still need work

[apoc.famine]

Except for one thing: If one human runs across this one non-standard case, that one human figures out how to navigate it. If another human comes across the same or another similar example, they too need to figure out how to navigate it. With machine learning, we can feed this into every single car, and they will all understand how to navigate it.
 
I'm really interested in seeing what a million Tesla M3s will be able to do in this realm. It's a massive amount of data collection, and if they can push lessons learned to all the cars, that's going to lead to a rapid increase in capability. Humans are definitely going to be better at navigating unique situations, but with machine learning and networking, the machine learning is not going to run into very many unique situations as time goes on.

Re:Cars still need work

[mschwanke97402]

You are quite right. New rech seems to get a pass every time. Everyone pooh-poohed rational concerns about the security of IOT devices. Now we have billions of them, compromised, being used in massive DDOS attacks.

People will die. I guaranty it. If you can count on anything it is that as the technology is commercialized, on a wide scale basis, corners will be cut in pursuit of profits. While the Googles, Teslas and other well heeled pioneers in the field will do years of testing, you know that won’t last, as 2nd and 3rd tier companies enter the field, or should I say hit the streets.

Re:Cars still need work

[TWX]

I expect that for self-driving cars to really work, some compromises in how traffic redirection for special cases is handled. I would not be surprised if cones and other barriers are required to have some technology, even something as simple as a pattern that the car can interpret, that describes something of the conditions and the expected behavior from the car. That would allow cars to at least handle construction zones and other "men working" zones better.

Still doesn't address things like power lines and road debris though.

Why cities and towns?

[Berkyjay]

Why not focus on the real and more fixable issue of highway commutes? Am I way off base in thinking that it would be far easier and safer to introduce fully autonomous cars onto highways first? Infrastructure wise you could first establish autonomous only lanes then gradually make all highways autonomous once the technology has been vetted and has become ubiquitous.

Re:Why cities and towns?

[Tenebrousedge]

If we lived in China and could arbitrarily decide to change our infrastructure on a large scale, yes, that might be a better idea. Otherwise, no, it would not be safer, and it makes little or no sense to implement this technology before it can deal with complicated hazards. If today you want an autonomous vehicle with a dedicated lane, build light rail. Preferably on the model of Portland, not SF.

I'm thinking you must be a software engineer.

Re:Why cities and towns?

[Berkyjay]

I'm thinking mainly of who would want to adopt this first. People in cities and towns? Or people who have to struggle with highway traffic everyday? I also feel that tackling the hazards on a highway as far easier than those on city and town roads. There are a lot less variables on a highway than on a city street. I also don't feel that infrastructure would be a major hurdle. We already have HOV lanes on most major highways. I just feel like the focus has been too much on the wrong problem.

And no, not an engineer. Just a humble developer.

Re:Why cities and towns?

[Tenebrousedge]

I also feel that tackling the hazards on a highway as far easier than those on city and town roads. There are a lot less variables on a highway than on a city street.

No, the incidence of the variable elements is lower. If I may be allowed to characterize, I think it's kind of a software mindset to try to reduce the problem to a simpler one, and to pay more attention to the error rate than the failure mode. At the risk of being flippant, we have a tolerance for crashes (I'm also a dev). Given that the energy of any collision rises with the square of velocity, the inherent risk of these collisions also rises proportionally with velocity. Immature technology in this kind of unforgiving environment is likely to result in serious loss of life. We can't guarantee that there will not be road hazards on the highway, so as long as there's a relatively large chance that the car will make a serious error in identifying or responding to a hazard, it's best if we test things out in the environments where hazards are both higher-incidence and lower-risk.

As for the infrastructure, we have HOV lanes because they improve throughput. There may be a point in the future where there are sufficient autonomous vehicles on the road that this will be true for them as well, but until then most cities are not going to be able to carve off extra lanes for this traffic.

Re:Why cities and towns?

[mschwanke97402]

If we lived in China and could arbitrarily decide to change our infrastructure on a large scale, yes, that might be a better idea. Otherwise, no, it would not be safer, and it makes little or no sense to implement this technology before it can deal with complicated hazards. If today you want an autonomous vehicle with a dedicated lane, build light rail. Preferably on the model of Portland, not SF.

I'm thinking you must be a software engineer.

Yeah, we certainly couldn’t do a large scale change in infrastructure. Nothing like switching from analog SD to HDTV or building out a cellular telephone system. What’s this Internet thingy I’ve been hearing about? Carpool lanes all across the country come to mind.

We’ve done a ton of things over the years with a combination of government subsidy and regulation and private commercial investment.

Re:Why cities and towns?

[Tenebrousedge]

Your analogies are inapplicable; see my other response.

The sarcasm was quite cute.

Re:Roundabouts"??

[BasilBrush]

No, they only give Americans trouble. In Europe they are everywhere and very routine. And I'm staggered that two lane roundabouts were considered a weird thing. They are very, very common too. Outside of America.

Re: Roundabouts"??

[jabuzz]

In addition to begin common place outside 3rd world countries, they are fantastically safer too. Repeated studies have shown that the introduction of a roundabout at a junction will given around a 90% reduction in fatal accidents. A major part of the reason that road deaths are much lower in Europe compared to the USA is that our roads are simply "structurally" safer. So while Americans are obsessed with getting T boned at a junction with an ever increasing race to drive bigger cars as a result, Europeans look on with bemusement and wonder why they don't put in more roundabouts.

Re:Roundabouts"??

[apoc.famine]

Yeah, those are fucking hard! You have to look left, and if nobody is coming, you go. And then if someone is coming you have to...I don't know...not go or something? JESUS THAT'S COMPLICATED!!!
 
And then once you're in them, it's SCARY AS FUCK! You have to figure out if you're taking the FIRST RIGHT, the SECOND RIGHT, OR THE THIRD RIGHT!!! And if you miss one, I think you die or something. It's not like you can just go around again.