Another Step Towards the Driverless Car

jtogel writes "At Essex, we have for some time been working on automatically learning how to race cars in simulation. It turns out that a combination of evolutionary algorithms and neural networks can learn how to beat all humans in racing games, and also come up with some quite interesting, novel behaviours, which might one day make their way into commercial racing games. While this is simulation, the race is now on for the real thing — we are setting up a competition for AI developers, where the goal is to win a race between model cars on real tracks. As the cars will be around half a meter long, the cost of participating will be a fraction of that for the famous DARPA Grand Challenge, whereas the challenges will be similar in terms of computer vision and AI."

Comments

The underlying research

[jtogel]

If anyone actually is interested in reading the papers discussing the experiments we did (many more than you see in the videos!), most of them are available on my website.

Some of them are of course better than others. I can recommend this one, about evolving general and specific driving skills, this one about co-evolution, this one about different learning techniques, and this one about modelling human driving and evolving tracks. There are several new ones, including one on physical cars, which are not on the website yet - mail me if you want a preprint!

All this assuming that anyone actually reads academic papers... sometimes it seems that not even the guy who writes the paper actually reads it. (Not true in my case, of course!)

Only works on wide roads

[Animats]

Notice that in all the examples, the road is much wider than the car. That's not by accident.

Driving using reactive behaviors is easy if you have plenty of room. On narrow roads, though, those approaches fail. You have to look ahead. In fact, to drive in the real world, you need a controller that plots at least an S-curve ahead. Otherwise, you'll end up in a tight spot pointed in a direction that won't get you through.

You don't necessarily have to "plan", in the AI sense, but you need a fairly good dynamics prediction capability, after which you can run a reactive controller on the prediction.

We went through this with our DARPA Grand Challenge vehicle. We started out with a reactive planner, but it just couldn't deal with tight spots. Most of the other teams ended up with S-curve planners, too. The reason you need S-curves is that you need to be able to achieve both a desired position and direction at a point ahead of the vehicle. So you need a curve with at least two degrees of freedom.

The predictor needs to know enough about the vehicle dynamics to make reasonable predictions. For example, predicted S-curves have to be built knowing how fast you can change the steering angle and how tightly you can turn given the current speed and ground bank.

If you need to do this stuff, read up on adaptive model-based feedforward control. The idea is that you have a system that learns how the system behaves as the inputs change and builds a model. Inverting the model gives you a predictor. Given a predictor, you can control.

A useful feature of that approach is that, while you're using one predictor, you can be training a better one safely. Predictors are trained by watching; they don't have to be in control. So you can start out with some dumb controller and work your way up to better ones, without crashing. This is probably how mammals learn motor skills.

Re:Only works on wide roads

[jtogel]

This is an interesting argument, and yes, we are experimenting with predictive control and internal models as well (experiments running as we speak). But I think that the real power of predictive models only show when you have multiple agents involved. People often underestimate the power of reactive control just because they assume that humans can't tune them well enough. But evolution is not limited in that way, and can sometimes do things with reactive controllers you just wouldn't think possible. (Stefano Nolfi has done some thinking and experimenting on this, see for example his paper "Power and limits of reactive agents".)

In the case of our (admittedly simple) model, we have a limited line of sight, and I think a good reactive controller can perform optimally (however you define that, often optimality is just another buzzword) given the limited sensor data. We did try evolving reactive control on much narrower tracks with good results - see for example our papers on track evolution. What the controller learns is often just to slow down when coming up to a narrow passage.