Grand Challenge 1, Competitors 0

Ivan writes "According to the DARPA Grand Challenge Status Board, 2 bots were withdrawn before the race started and the remaining 13 were all disabled. Red Team and SciAutonics II tied at 7 miles, a bit short of the 142 miles required." CNN has coverage and interviews.

Comments

Congratualations to those that tried.

[onyxruby]

This was not a failure just because no one got further than seven miles. Contrary to a failure, this has been a grand success. DARPA spent around 13 million to host it, and got a lot of great minds in the public at large to start thinking of ways to solve very complex technological difficulties. In terms of sheer dollar value, the amount of technological research by private individuals easily surpassed the 13 million the government invested. Already companies are being created, and the wheels of commerce spun.

This benefits the public from the technology that is being created that otherwise lacked an impetus. It benefits industry by showing a host of new ideas that otherwise would have never come through the regular channels. It certainly benefits DARPA for sheer investment and public relations value. It can benefit future soldiers by reducing their risk to dangerous jobs. This also benefits the defense contractors that just got a small reminder that someone from out of nowhere could become a player - think of it as lighting a fire in their belly.

All told this was a challenge, and was never intended to be easily winnable. It certainly was advertised as being unlikely to be won this year. All told I think DARPA should hold more contests like this for other areas that have grown stagnant. For a historical perspective consider that Lindbergh crossed that Atlantic on just such a contest. A contest that inspired the X-Prize. Perhaps we should see DARPA become involved in future X-Prizes as well?

Just remember not to name the project skynet.

Re:Congratualations to those that tried.

[KingJoshi]

Just because we can learn from failure and make the most of it doesn't mean it's not failure. I'm not saying the challenge itself was a failure but we can't always lower standards after the fact to suit our egos. I'm sure many of the participants and DARPA officials were expecting (not just hoping) for much better.

Re:Congratualations to those that tried.

[fermion]

I will go a bit further and say the thing to look at here is the process. What has been learned, what has been contributed to the nation, and how has it benefited the world. In this light the fact that there was no 'winner' or the race was not 'finished' may of little or no significance. This is why many research projects have a series of goals in which the 'answer' is only one of the many achievement that are pursued.

If you tru to do something significant that no one has done before, that is a success in itself. We hear all the time about people doing trivial things, or something that has been done 100 times before, and fawn over those achievements simply because a finish line was crossed. We too often forget about the process that went to make those things happen, and that many things are much easier today than even a year ago because the process was refined by people who perhaps never bother worried about crossing a finish line.

Re:Congratualations to those that tried.

[gaijin99]

True, but there will be pleanty of benefit in seeing what stopped each robot, what went wrong with its pathfinding algorithms, etc. I'll bet you that if they hold the event again next year they'd get to at least 14 miles, possibly even have one finish.

While I agree that in general we shouldn't define "success" to mean "learning something", don't forget that this whole project was for research. The whole object of research is to learn things, so I can see the granparent's point.

Re:Congratualations to those that tried.

[tftp]

On a battlefield, sometimes even half a mile is suicidal. If you have your men cut off from supplies by such a stretch of, say, bombarded road, or snipers, what do you do?

In World War II a soldier was sent to drive a truck, and if he fails... then another, and another. Today we can send such a robot. It is safe from snipers, and if it gets hit with a shell it will be simply replaced.

Machines like these can -already- be used to patrol large territories; with improvement, they will be really good at that.

Rough terrain's a bitch

[Anonymous Coward]

A guy on a mule has been evading the might and majesty of the United States Army in the Afghan mountains for over two years.

Re:Rough terrain's a bitch

[monster811]

And somehow he is dragging a dialysis machine with him...

Re:Rough terrain's a bitch

[Cruciform]

Osama bin Laden captured by Aibo. News at 11. :)

Is there still a chance....

[Dr Reducto]

Has the "privateer" race been done? It would be nice to see a privateer complete the challenge.

I think that even though they only got 7 miles into the course, thats still damn good engineering. Maybe next year they will have worked out what has gone wrong and figured out a way to flesh out an autonomous robot (Or hide a midget navigator somewhere!).

Processing power

[Ephboy]

The whole thing makes you conisder just how much processing power we use to control our speed around curves and avoid potholes when we're driving. We can integrate a hell of a lot of information, process the relavent signals and adjust our behavior in milliseconds. And that's not adding the additional struggle of trying to get your iPod to play through the stereo system....

This is *great* news!

[myowntrueself]

It means that autonomous fighting machines are still some way off.

I suspect that the first industrialised nation that develops autonomous fighting machines will take over the world (or at least have a damn good go).

Re:This is *great* news!

[Jodka]

" I suspect that the first industrialised nation that develops autonomous fighting machines will take over the world"

I predict the opposite. Any industrialised nation sufficinatly advanced to create an autonomous fighting machine would have little to gain from taking over the world. With adequate robot labor, you would have no need to exploit the world. At that point, added territory is no longer a source of useful resources but only an administrative burden. Primitive peoples are difficult to civilize and govern. Sure, we might use our robot warriors to down a particulalry bloodthirsty dictator from time to time and seed a self-governing democracy, just as we have used human soldiers to do with Milosovic in ex-Yugoslavia and Hussein in Iraq. But the goal in both places is to install a democracy and get the hell out ASAP. Fighting wars with robots will not change the underlying economic calculus of occupuation. It won't make ruling over the conquered any less of a pain in the ass, or any more profitable a proposition than today.

The more technologically advanced we become, the more we substitute common substances for exoctic mineral resource imported from abroad. Why conquer Brazil for copper mines when you get zillion times the bandwidth of copper from silicon glass fiber which is make from sand ? Power lines ? Use a superconductor strands. Conquer Africa for daimonds ? Bah !We can grow them more pure, large and cheaper in a vacuum deposition chamber in a New Jersey shopping mall. Once we find an adequate subsitite for fossil fuels, or choose to rely more heavily on those which we already have such as fission, that will be one less thing which we need from the outside.

The danger of autonomous fighting machines is not that the nations which develop them would use them to take over the world. The danger is that those weapons would fall into the hands of hostile and primitie societes which do have that goal, the same theat we face today. The technologically advanced nations which invented chemical, biological, and nuclear weapons could use them to to enslave the world, but we don't do that. The expense of occupation is too high and the payoffs for us are too low. The real danger of such technology is that is falls into the hands of primitive societies in which a primal warmongering mindset dominates.

They tried and failed?

[albeit unknown]

They tried and died.

Re:Really pathetic showing?

[Ephboy]

I don't think you've considered the immense complexity of simply adjusting your speed/direction to avoid a rock or pothole. Turn too fast, you flip (as at least one vehicle did). Next time you get in your car pay attention to just how many tiny speed/direction adjustments you make even on straight paved roads. Now add stuff you have to avoid and the process is incredibly complicated!

Re:Really pathetic showing?

[irokitt]

There were engineered obstacles, such as barbed wire and the like. And the deserts out here are not like the Sahara. They have washes and rock formations and various natural obstacles. Finally, a lot of the problems were mechanical or technical-brakes locking up and such. So it isn't as simple as it sounds.

Re:Really pathetic showing?

[Juggle]

Hate to break it to you but the desert is far from a "completely barren plan, with very few obstacles".

Even with a previously traveled path and prepared track it's not uncommon for VERY well financed race teams to fail to finish in a desert race. Most desert racers consider it a win just to make it to the finish line and that's with a driver!

Look into the SCORE side of this challenge a bit more and you'll find a LOT of info about just how challenging desert racing is with drivers - let alone trying to do it autonomously.

What's with all the mechanical failures?

[Anonymous Coward]

What's up with all the mechanical failures? Yeah, it's rough terrain, but we've been building human-powered vehicles that can handle it for decades! I'd think that keeping your engine going or your brakes from locking up would be the least "grand" part of the challenge.

Re:Really pathetic showing?

[Doomdark]

142 miles across a completely barren plain, with very few obstacles

You might to read the facts about challenge; SciAm for example had nice article.

But basically, it's not "just 142 miles in the middle of nowhere", but 142 miles with rather tight time limit (ie. they have to race almost as fast as human drivers would drive normally); exact route they HAD to follow (with some max. deviation allowed) was only disclosed few hours before start, and definitely wasn't just a straight line, and terrain was not just barren, it's pretty rough (meaning that staying on the road or path or whaver is a must) no matter how you look at it.

Still, might have been better to start small

[btempleton]

Yes, this was a grand challenge. But it would be nice if teams could solve part of the problem at first, get some recognition and minor prize money for that, and then move on.

So perhaps step one should have been just doing a long ordinary road course, minimal obstical avoidance, just handling roads, turns, potholes, ramps and even traffic lights (where you are told they are).

That contest would provide useful civilian tech and also useful military tech in terms of a autonomous vehicles to carry cargo in a controlled area with intact roads.

Or you could also imagine autonomous vehicles which handle roads, but then get to a rough patch they can't handle. At the rough patches you station soldiers who drive/remote control the vehicles over the rough patch, but you need far fewer because they stay in one place and only do the rough patch. Let humans do what they can do and computers do the boring long-haul road drive.

Next, hold a contest for a shorter rough course with obstacles.

Finally, combine the two.

Not _all_ that impressive

[Imperator]

To be fair, they were looking for him in the region of Afghanistan known as Iraq.

So considering no vehicle made it past 7 miles...

[ikewillis]

...how insane does this make the team that entered a motorcycle?

in WWI

[deathcloset]

The first tanks could only go a thousand yards before breaking down, and they had a 7 man crew.

it didn't take long for things to change.

Lessons?

[Quixote]

I submitted the following as a story for some discussion, but it got rejected; so I'll just post this for discussion.

Just like 1000s of geeks worldwide, I watched this with great interest. But the whole organization thing left me with a funny taste in the mouth.

It seemed as if the group that could throw the most money at the "problem" would win. Take the CMU team for example: they paid for a high-res survey of the area; had undergrads map out each and every obstacle in all of the possible paths; etc. Now, if the goal of this "grand challenge" was to unleash the entrepreneurial spirit, then it failed. Money != Entrepreneurial spirit.

Taking lessons from the RoboCup people, I would have preferred that DARPA organize it as follows:

1. Create a simulator for the sensors, and design a small (virtual) course for this simulator. Let people develop algorithms using this simulator, and have a competition in this virtual simulator to select a set of (say) 30 teams.
2. Provide each of these teams a platform: a humvee equipped with the sensors (actual ones from (1) above). Of course, if a team wants, it can add its own sensors.
3. After some time, hold a "grand challenge".
4. Analyse the approaches taken by the various teams, and (important) share the code among the teams. If a team designs a new sensor that is useful, get copies made and share with the teams for the next iteration.
5. Go back to step (1) above, and repeat.

Preference could be given to schools or efforts involving students, as not only is this a great learning experience, but also it will be a great motivator.

Just look at the technology gap between CMU and the rest of the entrants. It is quite an achievement that someone was able to equal CMU in performance.

There are a lot of smart hackers out there who would love to take a crack at this problem, but the lack of hardware is a serious hurdle.

Re:Lessons?

[EvilTwinSkippy]

Bull. Simulators allow you to cheaply rule out possibilities. Nothing that doesn't pass a simulation will work in the real world. And a simulation doesn't have to be a digital simulation. A 1/32 scale model running through a sandbox is a simulation. As is a wind tunnel.

It is true that passing a simulated test is no measure of success in the real world. But it will certainly be more prepared, and in a faster time and with less expense than an "all up" design method.

Look at the space shuttle if you want an example of "all up" gone bad. I'm not talking about the end product, I'm talking about billions that were squandered during development. The waste of time and money during the engine testing was extraordinary.

Another example is the Mark XIV torpedo. Google around, but the long and the short of it is the navy deployed a torpedo without testing it. A series of design flaws kept them from working, and their failure cost us dearly during the early parts of the war.

The Navy refused to believe there was a problem. The weapon worked 50 percent of the time for the 2 shots that were fired before the war. When they tested the torpedos properly they found numerous problems with the design of the guidance system and the detonators.

try this at home

[Kappelmeister]

I was thinking about that a few months ago when I was working on a film shoot. I had to run some supplies between a remote location and the base camp. The terrain was really rough-- this was a forest, and it had a lot of trees, roots, rocks, crevices and the like. And I absolutely, positively had to be at the base camp ASAP.

I found myself running as fast as I could, but my mind set up an interesting pipeline. I was always looking five to eight feet in front of me and my brain feverishly tried to parse out what was a rock, what was a branch, what was a big root, what was sloped ground, what was even ground, etc. Then, that information got passed to a route-choosing part of my brain that decided where the best place was to put each foot: left, right, left, right. That information, in turn, went to my brain's motor department, which was actually in charge of balance, weight distribution, and muscle movement to actually put the feet where they were supposed to be and keep my momentum without falling.

I call this a pipeline because my eyes never left that five-to-eight foot range. When I was selecting any bit of route, I was already looking at the next bit of route and stepping on the last bit of route. I never looked at my feet, but somehow always put them where they needed to be.

I wouldn't make such an analogy anywhere other than slashdot, but I could feel that the load average on my brain was as high as it could be. I didn't have any free cycles to think about my day, or have a song in my head, or think of my next joke, as I usually do. Every ounce of my concentration was going to these automatic, practically sub-conscious processes. I know was processing as fast as I could -- any faster, and my brain would tell me, "I can't parse the terrain that fast," or "I can't decide on a route that quick."

Don't give me any credit for it, because it has nothing to do with knowledge or intelligence, but I was solving an extraordinarly tough problem very quickly. In short, if I could bring my brain to the edge, I can see how tough this is for the DARPA contestants!

Re:try this at home

[Graff]

I found myself running as fast as I could, but my mind set up an interesting pipeline. I was always looking five to eight feet in front of me and my brain feverishly tried to parse out what was a rock, what was a branch, what was a big root, what was sloped ground, what was even ground, etc.

This is why new drivers and people who are disorientated or distracted often have trouble driving.

For new drivers there are 2 factors working here. First is the lack of experience of WHERE to look. New drivers often keep their vision trained too close in front of the vehicle. This works for very very low speeds but once you try to go road speeds you just don't have enough time to react. Experience teaches us to lift our eyes higher and seek ahead further in order to drive effectively.

The second factor for new drivers is how to handle this new kind of input. Stuff that you don't worry about when jogging or running becomes a big problem when driving, like corners and wet roads. The increased distance also means that you have to have a different sort of thought process in order to handle the increased amount of information.

With disorientated or distracted drivers they may have the knowledge of how to handle the processing of driving stimuli but since they are at diminished capacity they are not able to do so fast enough. Drunk drivers, for example, often start slewing back and forth because their reactions are lagging behind what their senses are telling them. They turn, overturn, correct, overcorrect, and so on.

kalman filtration

[rebelcool]

There is an algorithm out there called the Kalman filter which does this. It's very complicated and rooted in probability theory, but it basically takes several sensor inputs, smooths out their response based on previous values (and known noise characteristics, such as the typical standard deviation from the truth) and makes a good assumption about where the sensors will be in the near-future.

It is very accurate, if you tune it properly (thats the tricky part)

This is very important for real time things because you need to begin to smoothly react to situations before they happen (ie, driving into an obstacle at high speeds).

Re:kalman filtration

[groomed]

What the Kalman filter does is predict the future state of a model based on previous estimates and measurements. It takes into account the expected measurement error and the expected modeling error, and dynamically adjusts it's "confidence" in both model and measurements based on the estimation errors. The Kalman filter is an optimal filter, in that it can be shown to minimize the estimation error.

The beautiful thing about Kalman is that it works with partial data, that is, it can be applied recursively, "as the data are coming in". This is what makes it so suitable for realtime applications, as well as the fact that it is very robust in the face of temporary sensor failure.

Kalman is frequently used in tracking and control applications. Interestingly, Kalman filtering was also recently applied to the problem of task scheduling in the Linux kernel in the Entitlement Based Scheduler. There's lots of info about Kalman filtering on the web, use Google if you want to know more.

Re:need better collision avoidance

[cybermace5]

Whoa, look at this: with three minutes of typing and thinking, this Slashdotter just solved the problem that experienced engineers and computer scientists worked on nonstop for a year, at the cost of millions of dollars!

TROLL EXPOSED: COPIED FROM ALASKAN ENTRY

[cybermace5]

http://dgc.gi.alaska.edu/Overview_2_1_04.htm Section 2.4.1

Don't mod this cut-n-paste troll up anymore.

Re:Brick on the Accelerator

[ljavelin]

One might think a brick jammed on the accelerator of a jeep or H2 might have a chance to make it 7 miles.

Not without refueling, of course.

midget drivers, eh?

[squidgyhead]

You know, that gives me an idea. Would it count if you were to train some animal to drive this thing? Would "monkey-controlled" be close enough to "autonomous"? I bet you could get a monkey behind the wheel of a 4x4, train him, hook him up to enough drugs and electrodes to get him to drive the thing.

After all, you've got to remember that the world's fastest computers, the really, really big iron out there, still have about as much computational power as your average cockroach.

Not that I would condone such a thing, but, hey, if you're designing delivery systems for the US military, I think you've already lost the moral high ground.

what can we learn?

[snarkh]

Absolutely right.

In fact, it is not even clear what we can learn from failure like that, which we could not learn otherwise.

Flashy things like this race do not necessarily tell us anything more about deep problems of AI. One can spend millions and millions and not get any closer to the goal.

A: VERY INSANE Re:So considering no vehicle made

[feelyoda]

...how insane does this make the team that entered a motorcycle?

I suppose you weren't watching the live satellite feed when the motorcycle was demonstrated via remote control. It couldn't enter the race, but they just wanted to show it off.

It fell to the ground in literally 1 second.

Why they tried to solve a stabilization problem instead of an autonomy problem is beyond me. As I've said before, they engineered their own failing. This is different than the Red Team, where the basic hard problem of obstacle detection killed them.

First person

[Kallahar]

I was lucky enough to have a press pass. We had to get to Barstow at 5:00am (driving up from LA) which was rough. Driving in the dark along the highway leading to the start, we topped a rise and in the valley below portable lights had been strung up creating an oasis of light. As we got closer we could see the satelite uplink trucks, grandstands, police, and military.

After registering and getting our fancy orange safety vests, we went to the starting line and were able to get some great pictures as the bots prepared to start the race. Red Team (from Carnegie Mellon) had the best qualifying time so they were first out of the gates. Their 3 million dollar hummer started off fast and was quickly out of sight. The four helicopters filming from the air were flying dangerously close to each other trying to get a good shot so it was easy to track where the vehicle was.

SciAutonics II was the next bot out of the gate, the also had a good start and proceeded out of view. About this time we heard the good news, Team Red was seven miles down the course. Soon after we got the bad news, Red Team's vehicle had died. The odds on favorite was eliminated by a mechanical failure after only 7 miles.

Several of the rest of the bots failed to get past the first turn, and the rest either withdrew or failed within a few miles. The six wheeled ENSCO lost control, ran up the embankment, and overturned. Of the 100+ teams who built bots, 25 were invited to qualify, 15 of those were allowed to race, and only 7 of those made it more than a mile.

All in all it was an excellent experience. My suggestions for next year (or for the openchallenge, etc) would be to do the race in segments like the WRC does. Divide the 200 mile race into 10 mile segments, teams get points based on their performance for the stage. If you fail a stage you're not eliminated, you just fall in the rankings. Teams are allowed an hour of maintenance between stages to fix any problems they think they can fix. This would make it both a lot more interesting, and a lot more satisfying.

Here's what is on the course.

[forii]

142 miles across a completely barren plain, with very few obstacles.

Here's a very good .pdf slideshow that shows what type of terrain is on the course. I've driven off-road through the California desert many times, and it's pretty rugged stuff, lots of ravines, gullies, brush, and sand. Although one time I managed 40mph in an '83 Honda Accord. Not to mention that I had been up all night, was half-stoned, and had a car full of people tripping on LSD. Ah, those good old college days.

Re:The trouble spot

[throwaway18]

The preliminary results have been posted on the day 6 media gallery. Since it is a word document I'l post it in full.

Preliminary Data from DARPA Grand Challenge
As of 11:00 a.m. PST, March 13, 2004

Vehicle 22 - Red Team - At mile 7.4. Vehicle went off course, got caught on an obstacle and rubber on the front wheels caught fire, which was quickly extinguished. Vehicle has been disabled.

Vehicle 21- SciAutonicsII - At mile 6.7. Vehicle went into an embankment and became stuck. Vehicle has been disabled, and the team is recovering it.

Vehicle 5 - Team Caltech - At mile 1.3. Vehicle went through a fence, and couldn't come back through. Vehicle has been disabled, and the team is recovering it.

Vehicle 7 - Digital Auto Drive - At mile 6.0. Vehicle was paused to allow a wrecker to get through, and, upon restarting, sensors were not able to determine the proper route. After sensors tried unsuccessfully for three hours, vehicle was disabled.

Vehicle 25 - Virginia Tech - Vehicle brakes locked up in the start area. Vehicle has been removed from the course.

Vehicle 23 - Axion Racing - Vehicle circled the wrong way in the start area. Vehicle has been removed from the course.

Vehicle 2 - Team CajunBot - Vehicle brushed a wall on its way out of the chute. Vehicle has been removed from the course.

Vehicle 13 - Team ENSCO - Vehicle flipped in the start area, experienced a fuel leak, and the team needed to shut off the fuel. Vehicle has been removed from the course.

Vehicle 4 - Team CIMAR - At mile 0.45. Vehicle ran into some wire and got totally wrapped up in it. Vehicle has been disabled.

Vehicle 10 - Palos Verdes High School Road Warriors - Vehicle has been removed from the course - it hit a wall in the start area.

Vehicle 17 - SciAutonics I - At mile 0.75. Vehicle went off the route. After sensors tried unsuccessfully for 90 minutes to reacquire the route, without any movement, vehicle was disabled.

Vehicle 20 - Team TerraMax - Got to mile 1.2. Vehicle then started backing up and after .5 miles, the vehicle was disabled.

Vehicle 15 - Team TerraHawk - Withdrew prior to start.

Vehicle 9 - The Golem Group - At mile 5.2. Vehicle stopped. Vehicle had a throttle problem while going up a hill. After trying for 50 minutes, the vehicle was disabled.

Vehicle 16 - The Blue Team - Withdrew prior to start.