Slashdot Mirror

A better LIDAR sensor is needed. Something more like the Advanced Scientific Concepts flash LIDAR. Right now, it costs too much (about $100K) but that's because they're hand-made in Santa Barbara for DoD and space applications. It has custom ICs made in volumes of tens. Volume production would bring that way down. You don't get the full circle field of view of the Velodyne, so it may take multiple sensors.

To deal with rain and snow, you need "first and last" return data. This is used in air to ground sensing to sense both the top of tree cover and the ground underneath. With that, and a good frame rate, you'll be able to distinguish rain and snow noise from solid objects. You'll lose range in heavy rain and snow, and will have to slow down. That's OK; humans can't see through it either.

Radars can. What's Google doing on the radar front? Off the shelf automotive radars are getting pretty good. Modern millimeter radars can see pedestrians. The older units from the Grand Challenge days could only see car-sized obstacles, maybe motorcycles.

Having worked on self-driving cars (2005 Grand Challenge), a few points:

The comment about minimizing "near-collision states" is significant. A near-collision state is one where a reasonable variance of the behavior of another vehicle could cause a collision. It's about predicting other-vehicle behavior. That's an important area to study. Aviation people put a lot of effort into minimizing near-misses, and it pays off.

Incidentally, Tesla's announcement that they're starting work on an "autopilot" is them playing catch-up. Audi, BMW, Cadillac, and Ford are already demoing automatic driving systems. It looks like Cadillac will be the first to ship hands-off highway driving, in 2015. All these early systems are highway driving only, although Cadillac includes stop-and-go driving in traffic jams. That's likely to be a very popular feature.

On the sensor side, more progress is needed, and it's coming. That rotating LIDAR contraption on top of Google's self-driving cars is from Velodyne. It's 64 LIDAR units on a spinning turntable. That's a research device, not a production one. There are better ways to do LIDAR, but the cost needs to come down. The approaches used in the Kinect and the XBox One will not work outdoors in bright sunlight. Outdoor LIDAR systems work fine, but they're pulsed, not continuous. For a nanosecond, at one frequency (color) they far outshine the sun. But the total energy per pulse is low, so they're eye-safe. Currently, such devices are very expensive, but that's not for any good reason. It's because some exotic ICs have to be made in tiny quantities.

Radars are getting better, too. A decade ago, in the Grand Challenge, we had to use Eaton VORAD radars, which operate at 24GHz. These could reliably range cars, trucks, and larger bicycles, but not people at long range, or signposts. (Such radars return range, azimuth, and range rate; this isn't a speed gun. I used to have one of these looking out my window at at an intersection, with a display plotting the traffic.) Today's automotive radars are running at 77GHz, with plans to move to 79GHz. There's an effort to standardize on 79GHz internationally. Tripling the frequency, plus applying more compute power to the processing, means that most objects a car might hit are detectable. These radars are getting cheap and small, so a car will have enough of them to provide full-circle data. Long range is needed mostly in front; on the side and in back, much lower power can be used.

A key issue is a high viewpoint. This isn't just about obstacle detection. You also need to profile the road. This was a big deal for the off-road DARPA Grand Challenge, but even on paved roads you need to be able to detect junk on the pavement and potholes. Google has their sensor on top of the roof. This will probably be unacceptable in a production car. I'd go for flash LIDARs at the top corners of the front windows. One possibility is a narrow strip just above the windshield, to contain all the sensors. This is one way to combine vehicle aesthetics and field of view.

Cameras are useful, but computer vision is still kind of dumb. Distance from stereo only works at short ranges, and range rate info from cameras is poor. Digital cameras are so cheap now, so it's tempting to think they can do the whole job. Not yet. Computer vision isn't good enough. Tesla is probably putting too much hope into camera processing. You need cameras to recognize signs, traffic lights, and such. Also, you need multiple sensors because not all objects are visible on all sensors. Radars can't see insulators. Cameras can't see objects with little contrast against the background. LIDARs can't see some materials, such as the charcoal fabric used on many office chairs. Sensor fusion is essential.

Enough for now. This looks quite do-able.

"The problem with Google's current approach is that the sensor system is too expensive" - Musk

He's referring to the expensive Velodyne rotating array of 64 LIDARs found on top of Google's cars. It's a useful device, but it's a research tool, not something that belongs on top of production vehicles.

What's needed is a compact solid-state 3D LIDAR for outdoor use. Advanced Scientific Concepts makes such things, but they're sold to DoD for about $100K each. Typical performance is 300 meter range, 128x128 pixels, 30 FPS. There's no fundamental reason the technology needs to be that expensive; it's just that the things are hand-made at a lab in Santa Barbara, CA. (I visited them a decade ago when we were doing a DARPA Grand Challenge vehicle. Back then, they had the technology working on an optical bench, but didn't have usable hardware yet.) This technology needs to be turned into a mass market product. The current generation Kinect, (which is a true LIDAR, not a trianguation sensor like the previous model) does roughly the same thing, but with a less sensitive sensor and a weaker laser. Eventually somebody will put enough money behind this to get it right.

As someone who's actually done this stuff, LIDAR gives solid data, but is range-limited. Cameras have more ambiguous results. Cameras are most useful when things are going well, as on a highway under good conditions. That was Stanford's approach in the Grand Challenge. All their vision system really did was answer the question "is the near section of road (within LIDAR range) like the far section of road"? If the LIDAR said the near section was OK to drive on and the vision system said the far section was like the near section, then the vehicle could speed up and out-drive the LIDAR range. That sped up travel on good sections of road.

Google is using Velodyne LIDAR units, which are effective but an expensive mechanical kludge. A better approach is from Advanced Scientific Concepts, which has an eye-safe flash LIDAR. No moving parts.

ASC's units cost about $100K each, but that's because they're hand-made for DoD. The technology isn't inherently expensive if made in volume. It uses custom imaging ICs, and because they're made by tens, not millions, they cost far too much. If the cost can be brought down, the vehicle can have multiple LIDAR units around the car to get full coverage, rather than one big spinning thing up on the roof.

Millimeter radar is also useful. It's good to have a Dopper anticollision radar as a backup system. It provides an unambiguous "rapidly approaching big solid object" signal. We had one of those on our DARPA Grand Challenge vehicle as a backup to the fancier LIDAR system.

the structured light fields from one will interfere with another

Indeed, I think a better solution is a time-of-flight sensor such as pulsed IR phase or flash LIDAR.
 

this is bad advertisement. And timothy ought not have posted it.

Right. It's 3D from stereoscopy, not a depth camera. The baseline between imagers is small, so it won't have much depth resolution for distant objects. Note that while the video shows outdoor scenes, it doesn't show depth information for them.

Now the Advanced Scientific Concepts flash LIDAR is a real time of flight depth camera, a 128 x 128 pixel LIDAR. With a real time of flight device, depth resolution stays constant with distance, as with radar. This device tends to be equipped for a narrow field of view and long range, because it's sold to the military. But that's not inherent in the technology. A similar device, but with mechanical scanning, is the Velodyne laser scanner, which almost everybody in the 2006 Grand Challenge used.

I've met the people behind both systems. The ASC device is potentially mass-produceable at moderate cost, but the company is focused on DoD applications and hasn't pursued that. It requires custom IC fabrication, which is cheap when you make millions and extremely expensive when you make hundreds. The Velodyne thing is a better version of the impressive but fragile Team Dad laser scanner from the 2005 Grand Challenge. It's a spinning array of little LIDAR units. Both cost around $100,000, due to the tiny market.

http://www.advancedscientificconcepts.com/images/GUI.jpg
An image sensor picks up color data. LIDAR picks up luminosity and distance.

You're right....not phone size yet.
http://www.advancedscientificconcepts.com/

Except that your 'slightly bigger copter' is 'not even fractionally as capable'

It exists. That's what an autonomous helicopter with onboard sensors can do as of a year or two ago.

Environment sensing is coming along. Check out Advanced Scientific Concepts' flash LIDAR. (Still too expensive, but it's a tenth the size of what it was five years ago. I saw the optical bench prototype in 2003, when it was the size of a desk.) Simultaneous Localization and Mapping finally works.

All that fixed motion tracking gear is a debug environment.

Advanced scientific concepts has 3d cameras.

http://www.advancedscientificconcepts.com/
Read the press release...NASA is mentioned.

In my humble opinion, the Darpa Grand Challenge, by offering a market to LIDAR makers, made vision-based SLAM a thing of the past and the under-budgeted.

That's what many of us with Grand Challenge entries once thought. Even Sebastian Thrun once thought that. But, in fact, the winning 2005 Stanford "Stanley" vehicle was running mostly on vision. Above 25MPH it was out-driving its LIDAR range. The vision system wasn't doing SLAM, though. It was comparing the road further ahead with the near road. If they "looked the same" (the machine learning system for making that judgment was the breakthrough) and the LIDARs profiled the near road as flat, then the vehicle could drive faster than it could stop within the LIDAR range.

For the Urban Challenge, LIDAR units were more useful, because the speeds were slower and the environment more cluttered. But see the current issue of IEEE Trans. on Robotics, the special issue on SLAM, to see how much progress has been made. It's useful to use a camera and a limited LIDAR together with a SLAM algorithm; the vision system brings in more data and the LIDAR has less ambiguity.

The Velodyne thing (which is a better-built version of the Team Dad spinning LIDARs of 2005) is a good device, but too big, too expensive, and has too much rotating machinery for a production product. I've met its designers and seen the thing. The next step will probably involve either flash LIDAR or MEMS mirrors. Eye-safe flash LIDAR is a reality, and if produced in volume, it wouldn't be that expensive. It's expensive now only because it needs custom ICs.

An affordable little non-scanning 3D LIDAR for indoor use would be useful. There's the Swiss Ranger, the first device that qualifies. This is a true 3D time of flight sensor with no moving parts and 176x144 pixels. It's been around for about five years as a custom research item, but it's now being sold as a product by Acroname for $7500. The price needs to drop by an order of magnitude or two, which is quite possible.

This is a big step forward. I know this technology. Back in 2004, when we were putting our DARPA Grand Challenge vehicle together, I went down to Advanced Scientific Concepts in Santa Barbara to see the thing. Back then, they had a prototype that worked, but it was on an optical bench (one of those big plates with screw holes to which you attach optical components), nowhere near ready to go on a vehicle. It was just too early. We had to go with SICK rotating-mirror line scanners, like everybody else. But I was convinced it was the right direction to go, and I dragged a venture capitalist who had some underperforming photonics companies down there to see the thing. He didn't want to fund them, because they were too far from a consumer product; the near term market was DoD-only.

ASC kept working, and by 2006 they had working portable prototypes. By 2007, you could buy a LIDAR about the size of a large-format camera for about $100,000. Now they've downsized it further.

Unlike the laser scanners with spinning mirrors or sensors, which is what everyone else uses, this technology has no moving parts. The system has two main components - a pulse laser with diffusing optics, and a detection and timing IC with one LIDAR receiver per pixel. Neither of these is inherently expensive in quantity. It may take a while to get this down to webcam prices, but $1000 is a reasonable near-term target.

It's amazing that this can be done in an eye-safe way, since this approach is subject to the radar equation - returned power decreases as the fourth power of the distance. But the detectors can be made good enough. Some of their more sensitive detectors use a photomultiplier tube technology, like a night vision system. Night vision systems use a photoelectric detector plate - when a photon hits it, an electron pops out. Electric fields are used to accelerate the electron, which then hits one of the electron detectors on a specially designed IC. Photomultipliers have been around for decades, and can detect single photons. The neat thing about the photoelectric effect is that it's at the atomic level, and happens in picoseconds. So it can be used as a light amplifier for a time-of-flight LIDAR.

The current generation of compact sensor is 128x128 pixels at 30Hz. The sensors are currently smaller than the lasers, but for smaller robots where you need only 10m of range or so, a smaller laser can be used.

This is the sensor that will make automatic driving commercially feasible.