Slashdot Mirror

← Back to Stories (view on slashdot.org)

Nvidia GPU-Powered Autonomous Car Teaches Itself To See And Steer (networkworld.com)

Posted by BeauHD on from the homeschooling dept.

An anonymous reader quotes a report from Network World discussing Nvidia's project called DAVE2, where their engineering team built a self-driving car with one camera, one Drive-PX embedded computer and only 72 hours of training data: Neural networks and image recognition applications such as self-driving cars have exploded recently for two reasons. First, Graphical Processing Units (GPU) used to render graphics in mobile phones became powerful and inexpensive. GPUs densely packed onto board-level supercomputers are very good at solving massively parallel neural network problems and are inexpensive enough for every AI researcher and software developer to buy. Second, large, labeled image datasets have become available to train massively parallel neural networks implemented on GPUs to see and perceive the world of objects captured by cameras. The Nvidia team trained a convolutional neural network (CNN) to map raw pixels from a single front-facing camera directly to steering commands. Nvidia's breakthrough is the autonomous vehicle automatically taught itself by watching how a human drove, the internal representations of the processing steps of seeing the road ahead and steering the autonomous vehicle without explicitly training it to detect features such as roads and lanes.

36 of 54 comments (clear)

  1. Inherent shortcomings by fyngyrz · · Score: 2

    In situations that do not resemble the training data, the network's response is essentially undefined, as well as unknown (it's all unknown... an NN results in behaviors that are not deterministic in the sense that anyone planned them out -- they are what they are, that's all.)

    Nice experiment, though. :)

    --
    I've fallen off your lawn, and I can't get up.
    1. Re:Inherent shortcomings by OzPeter · · Score: 1

      In situations that do not resemble the training data, the network's response is essentially undefined, as well as unknown (it's all unknown... an NN results in behaviors that are not deterministic in the sense that anyone planned them out -- they are what they are, that's all.)

      Nice experiment, though. :)

      While I tend to agree with you on the un trained aspect (and I hope they have their liability insurance in place for driving an "unknown" on a public road) it would be interesting to know if you can combine NN datasets and thus massively parallelize the learning process.

      --
      I am Slashdot. Are you Slashdot as well?
    2. Re:Inherent shortcomings by fyngyrz · · Score: 4, Interesting

      No, not like people. An NN of the type described doesn't learn further; won't generalize from other experiences (FI, this thing won't know what to do with a pedestrian unless it's observed the human driver stopping for some number of them. Otherwise it has no training that places value on a pedestrian. Same for cat, chunks of some trucks blown-off retread, potholes, etc.) A person will know from their far, far deeper experience that it's inherently a bad idea to run down the three year old that walked in front of the car, will usually at least try not to run down little Susie's pet cat, and will choose to go around the pothole if it is reasonable to do so.

      An NN is a low-dimensional approach to solving problems. But most non-trivial problems -- and driving a vehicle is an entirely representative proxy for such problems -- tend not to be uniformly low-dimensional. Many aspects come into play suddenly and unpredictably. Some might not be seen for years, or ever. But then again, they might. In order to deal with such things, more than low-dimensional problem solving is required. NN's can't do it. They're inherently limited.

      It's going to require very high level software. Not intelligent, but damned well informed and replete with a huge rulespace that can solve all of the types of reasonably solvable problems in the space.

      That's all IMHO, of course. But I'm right, so there's that. :)

      --
      I've fallen off your lawn, and I can't get up.
    3. Re:Inherent shortcomings by fyngyrz · · Score: 1

      That may well be a very good approach in terms of covering all the bases in a distributed fashion. However, now the problem arises of how to combine all of that training into one coherent, functional space so your vehicle knows everything it needs to know. While it seems obvious that a lot of hardware spread wide will gain lots of experience, and all taken together might consist of a very well covered solution space, as each NN system will be different, even in how it solves the same problems, how to winnow that down to a full, but minimal, solution set that would fit in your own vehicle, or talk to it in a timely manner to control a task that has millisecond-resolution response requirements... that's just plain gnarly.

      --
      I've fallen off your lawn, and I can't get up.
    4. Re:Inherent shortcomings by laing · · Score: 2

      "Otherwise it has no training that places value on a pedestrian." /p It's been years since I played Death Race 2000 (the video game, not the movie). How many points is a pedestrian worth these days?

    5. Re:Inherent shortcomings by fyngyrz · · Score: 1

      How many points is a pedestrian worth these days?

      um.... well, is it a mime? Also, African, or European?

      --
      I've fallen off your lawn, and I can't get up.
    6. Re:Inherent shortcomings by lorinc · · Score: 4, Informative

      An NN is a low-dimensional approach to solving problems. But most non-trivial problems -- and driving a vehicle is an entirely representative proxy for such problems -- tend not to be uniformly low-dimensional. Many aspects come into play suddenly and unpredictably. Some might not be seen for years, or ever. But then again, they might. In order to deal with such things, more than low-dimensional problem solving is required. NN's can't do it. They're inherently limited.

      No, they're not. The lack of proper response to an unseen before event (which is called bad generalization) can come from at least 2 things: a bad sampling for the training set (no pedestrian in the set), and the absence of transfer learning (use of another system trained on pedestrian to improve the first one). The first one is really easy to solve but time consuming, a lot of very bright people are working on the second.

      Honestly, automatic driving by visual cues is not longer a challenging computer vision problem. It is still a challenging engineering problem, but all the scientific tools needed to solve it are already there, and they are being actively used to solve it.

      There are a lot of computer vision tasks that are now to be considered as solved from the research point of view. Very few people seem to acknowledge the enormous progress that have been made in the field in the last 10 years. Even people working in the field. But it's there, and now the engineers have to use it.

      --
      Video of some good progressive thrash music
    7. Re:Inherent shortcomings by fyngyrz · · Score: 1

      Alluding to them, anyway.

      --
      I've fallen off your lawn, and I can't get up.
    8. Re:Inherent shortcomings by fyngyrz · · Score: 1

      Yes, actually, they are -- because the things you list that would defuse that aren't achievable with a wide problem; only with a deep one.

      We have LDNLS pretty well solved. The rest is, at best, research-level stuff that just isn't ready for prime-time, and probably will never be until something more sophisticated is employed. Providing a training set won't ever reach competence. You have to provide the ability ti learn independently, and well. No one has that on the table. Yet.

      --
      I've fallen off your lawn, and I can't get up.
  2. Why a single camera? by fustakrakich · · Score: 1

    Ever driven with one eye shut? The drunks needn't answer that one. But with two or more cameras and various other sensors, it seems that the "learning" process would go much smoother.

    --
    “He’s not deformed, he’s just drunk!”
    1. Re:Why a single camera? by OzPeter · · Score: 1

      Ever driven with one eye shut? The drunks needn't answer that one. But with two or more cameras and various other sensors, it seems that the "learning" process would go much smoother.

      If you read TFA you'd see that the learning process is done with 3 cameras. Only the actual driving is done with a single camera.

      But yes I drive with "one eye shut" all the time courtesy of being blind in one eye. However I prefer my automated systems to have some degree of redundancy - and a single camera failure would cripple this system.

      --
      I am Slashdot. Are you Slashdot as well?
    2. Re:Why a single camera? by npslider · · Score: 1

      A human's two eyes gives depth perception by seeing the same object from two slightly different angles, the brain composites this and creates the perception of depth. Is there technology to allow a computer to understand the information 2 cameras provide like a human can?

  3. Now there's a Rick and Morty line by the_skywise · · Score: 3, Funny

    "What is my purpose?"
    "You drive me places." ... "Oh God!"
    "Welcome to the club, pal!"

    https://www.youtube.com/watch?...

  4. yeesh, I need a stiff drink or three by Thud457 · · Score: 1

    To be fair, if I was driving on the highway at night and a deer jumped out in front of the car I was driving,
    I too, would reboot myself.

    Once from the shock of the deer jumping in front of the car,
    once from the collision
    once from appraising the mess the collision created
    once from clearing the mess off the road (assuming my car was still drivable)
    once from the mental anguish of gruesomely killing a large charismatic animal.

    --

    the preceding comment is my own and in no way reflects the opinion of the Joint Chiefs of Staff

  5. Re:Nvidia by scheveningen · · Score: 1

    Marketing. They are showcasing their hardware.

  6. Re:Nvidia by Anonymous Coward · · Score: 3, Insightful

    Last I checked NVIDIA's market cap was 20.3B$. You'd think they have the ability to do more than one thing at a time, especially if it returns a profit on investment in the future.

    This sort of AI usage is only going to increase, it would be dumb of NVIDIA not to at least divert a miniscule amount of manpower and resources to try and get their feet wet in this field. Anything they can patent or turn into a product will outweigh the paycheck for the engineers on this "worthless" venture.

  7. in an ideal autonomous vehicle development world.. by sittingnut · · Score: 1

    ideally,
    computing hardware running AI should be developed separately ( by many companies ),
    so many deferent 'AI's can be created by many companies/groups separately, to run on those hardarews to interact with generic prototype vehicles/interfaces,
    then many different vehicle producers can create many actual models to sell, enhancing those generics with added customized features that can be marketed.

    doing all of the creating from start to selling, or at least most of that, by a single company, while other similar companies also do most of that separately, is a bad way to go.

    many groups creating each separate part, and competing with each other to get those lower down to accept their creation, will benefit all.

  8. Computers don't learn by Sam36 · · Score: 1

    Sorry, there is no algorithm that makes algorithms..

    1. Re:Computers don't learn by slew · · Score: 4, Interesting

      Sorry, there is no algorithm that makes algorithms..

      Although there might not be an algorithm that makes algorithms, there are algorithms to configure a meta-algorithm implementations. And example meta-algorithm implementation would be a deep neural net, or a human brain. I don't think it is a stretch to call the algorithm used to configure meta-algorithm implementation "learning" (although commonly this is called training)...

      But this is merely a semantic point.

    2. Re:Computers don't learn by wonkey_monkey · · Score: 1

      Sorry, but-

      No, wait, I'm not sorry. Why would I be sorry? Why are you sorry?

      Anyway, yes there is.

      --
      systemd is Roko's Basilisk.
    3. Re:Computers don't learn by lorinc · · Score: 1

      Definition of learning: The acquisition of knowledge or skills through experience, study, or by being taught

      The computer is acquiring the new skill driving, therefore the computer is learning. End of the story.

      By the way, the learning algorithm that optimizes the parameters of the neural net is making an algorithm. The neural net itself is an algorithm that takes a flow of images as inputs and outputs a steering decision. Therefore, the training/learning/optimization procedure that produces such neural net is an algorithm that makes algorithms.

      --
      Video of some good progressive thrash music
    4. Re:Computers don't learn by Anonymous Coward · · Score: 1

      I wrote a fairly lengthy reply to the above chain but realised that every point I was going to make was made better by Douglas Hofstadter in Godel, Escher, Bach: an Eternal Golden Braid. It really should be essential pre-reading for any slashdotter engaged in a discussion about AI. If you haven't read it, go and do so now.

  9. A Scary Thought by npslider · · Score: 1

    "Automatically taught itself by watching how a human drove..."

    Oh my... and just what kind of driver was used as the role model?

    1. Re:A Scary Thought by mspohr · · Score: 1

      Teenage boys... they (think) they're the best drivers.

      Or... maybe you could customize your driving:
      - teenage girl - texting and talking
      - old geezer - very slow reflexes
      - little old lady going to church on Sunday
      - soccer mom with 8 kids in the car running around after school to get everybody delivered
      - middle age guy with road rage
      - new driver with an "L" on the car
      The possibilities are endless

      --
      I don't read your sig. Why are you reading mine?
  10. Re:Hasn't this been done before? by npslider · · Score: 1

    As this technology continues to improve I'm sure we will hear more and more about this until we take it for granted as the new normal.

  11. Re:Nvidia by npslider · · Score: 1

    It's nice to see how technology can be used in real world (ie on the road), not just on a store shelf.

  12. Re:Nvidia by npslider · · Score: 1

    A special chip for adaptive learning... heck yes, show that off!

  13. Re:in an ideal autonomous vehicle development worl by mspohr · · Score: 1

    This might work:

    Neural net on a stick.
    http://techcrunch.com/2016/04/...

    --
    I don't read your sig. Why are you reading mine?
  14. Heh by MobileTatsu-NJG · · Score: 3, Funny

    We all know this car's running Windows cos Linux ain't got no good nVidia drivers!

    --

    "I like to lick butts!" by MobileTatsu-NJG (#32700246) (Score:5, Informative)

    1. Re:Heh by Trogre · · Score: 2

      Funny, but wrong.

      nVidia hardware performs at least as good under Linux as Windows, including CUDA processing.

      --
      "Nine times out of ten, starting a fire is not the best way to solve the problem." - my wife
  15. "Directly" by wonkey_monkey · · Score: 1

    The Nvidia team trained a convolutional neural network (CNN) to map raw pixels from a single front-facing camera directly to steering commands.

    That's about as "direct" as... well, as direct as the connection between the photons that enter my eye and the hand movements I make to steer my car, I suppose. Not very direct at all.

    --
    systemd is Roko's Basilisk.
  16. DARPA's CPU by Macdude · · Score: 1

    Isn't this exactly the type of problem that DARPAs new CPU that's bad at math should be really good for?

    --
    "Grab them by the pussy" -- President of the United States of America
  17. Beemers by Nethead · · Score: 1

    I love the comment when there is a merging lane, "I just have to watch out for BMWs."

    --
    -- I have a private email server in my basement.
  18. Not a Luddite fear, but a valid fear by GPS+Pilot · · Score: 2

    Here's an excellent article by Ashlee Vance about a self-driving neural net-based system created by one man, George Hotz: http://www.bloomberg.com/featu...

    It contains one passage that sums up my fears:

    Hotz hadn't programmed any of these behaviors into the vehicle. He can't really explain all the reasons it does what it does. It's started making decisions on its own.

    This will come back to bite us. More than once. Systems can exhibit unexpected behavior even when the inventor has an excellent understanding of the invention; here, a very bright inventor seems to have no hope of fully understanding things.

    Unexpected behavior from the control system of an object that has a lot of kinetic energy is usually bad. And a car is not the most dangerous thing you can put a neural net in charge of.

    --
    That that is is that that that that is not is not.
  19. Who said Knight Rider? by paolo.redaelli · · Score: 1

    "KITT, drive me home!" I can't believe no one has't already cited Michael Knight and KITT of "Knight Rider" the TV series that launched David Hasselhoff

  20. Starman by DriveDog · · Score: 1

    learned by observing, especially to accelerate hard to get through yellow lights.