Slashdot Mirror

← Back to Stories (view on slashdot.org)

The Dark Secret At the Heart of AI (technologyreview.com)

Posted by BeauHD on from the two-sides-to-every-coin dept.

schwit1 shares an excerpt from a report via MIT Technology Review: No one really knows how the most advanced algorithms do what they do. That could be a problem. Last year, a strange self-driving car was released onto the quiet roads of Monmouth County, New Jersey. The experimental vehicle, developed by researchers at the chip maker Nvidia, didn't look different from other autonomous cars, but it was unlike anything demonstrated by Google, Tesla, or General Motors, and it showed the rising power of artificial intelligence. The car didn't follow a single instruction provided by an engineer or programmer. Instead, it relied entirely on an algorithm that had taught itself to drive by watching a human do it. Getting a car to drive this way was an impressive feat. But it's also a bit unsettling, since it isn't completely clear how the car makes its decisions. Information from the vehicle's sensors goes straight into a huge network of artificial neurons that process the data and then deliver the commands required to operate the steering wheel, the brakes, and other systems. The result seems to match the responses you'd expect from a human driver. But what if one day it did something unexpected -- crashed into a tree, or sat at a green light? As things stand now, it might be difficult to find out why. The system is so complicated that even the engineers who designed it may struggle to isolate the reason for any single action. And you can't ask it: there is no obvious way to design such a system so that it could always explain why it did what it did. The mysterious mind of this vehicle points to a looming issue with artificial intelligence. The car's underlying AI technology, known as deep learning, has proved very powerful at solving problems in recent years, and it has been widely deployed for tasks like image captioning, voice recognition, and language translation. There is now hope that the same techniques will be able to diagnose deadly diseases, make million-dollar trading decisions, and do countless other things to transform whole industries. But this won't happen -- or shouldn't happen -- unless we find ways of making techniques like deep learning more understandable to their creators and accountable to their users. Otherwise it will be hard to predict when failures might occur -- and it's inevitable they will. That's one reason Nvidia's car is still experimental.

schwit1 adds: "To be fair, we don't really understand how natural intelligence works, either."

19 comments

  1. Fuck AI by Anonymous Coward · · Score: 0

    Fuck Nvidia.

    Burn the motherfucker down.

    1. Re:Fuck AI by Anonymous Coward · · Score: 0

      forced meme is forced. also, me? I'd serve crab legs.

  2. We're Geeks! Dark Secret Heart? Please!! by Anonymous Coward · · Score: 2, Insightful

    What is wrong with slashdot lately? We are geeks. We know how a lot of this stuff works. That the neural net doesn't give you an algorithm that's easily converted into a step by step recipe with an explanation doesn't mean it's voodoo!

    1. Re: We're Geeks! Dark Secret Heart? Please!! by Anonymous Coward · · Score: 0

      not voodoo, but apparently people forgot its just a complex statistics machine. as are we.

    2. Re:We're Geeks! Dark Secret Heart? Please!! by Anonymous Coward · · Score: 0

      Right. This probably just means that instead of talking about algorithmic complexity, we'll be talking about probability classes and limits of function graphs, or something like that. As a greater whole, we've got this.

    3. Re: We're Geeks! Dark Secret Heart? Please!! by Anonymous Coward · · Score: 0

      yeah 3dfx died long ago

    4. Re:We're Geeks! Dark Secret Heart? Please!! by Bite+The+Pillow · · Score: 1

      "Easily" and "Millennium Prize Problem level difficulty" are apparently the same thing in your language...

      What I think you meant to say, however, is that with a hardware debugger, we can record and the underlying processor instruction pointer and see where in the decision tree something went wrong. Not something I'd like to have to debug due to the likely crazy amount of data and decision points involved, but technically possible.

      That's very different from being able to assert that we use computer vision to identify walls and guard rails and vehicles, and include logic to avoid them. No such rule was ever introduced, except by omission.

      And, as such, there is no good way to predict what the AI will do given specific input. A "classical" program would be highly predictable based on its accuracy rate in identifying known objects, the number of unusual objects, and the image quality (visibility, contrast, or glare based on the environment). This thing on the other hand may not have a threshold for admitting it shouldn't be driving.

      Without the training data set and software being used, it is possible that I have missed something vital. But what you missed is simple. Is it good enough that a car drives itself successfully? Or should we be able to make certain guarantees, or assertions, about its behavior?

      Should the NTSB be able to review what caused a 10 car pile up, or church bus head on everyone died collison, and be confident of coming to a conclusion in a human lifetime? Because right now with this technology, we can't do root cause analysis. We can build experiments like Deep Dream to understand how decisions are made, but there's no guarantee of finding the answer.

    5. Re:We're Geeks! Dark Secret Heart? Please!! by Pseudonym · · Score: 1

      As geeks, I hope we all know that an artificial neural net should never be allowed to control a safety-critical system without a more predictable layer which can override it.

      --
      sub f{($f)=@_;print"$f(q{$f});";}f(q{sub f{($f)=@_;print"$f(q{$f});";}f});
    6. Re: We're Geeks! Dark Secret Heart? Please!! by Pseudonym · · Score: 1

      We also forgot that the whole point is to stop entities "like us" being in control of 2 tonne death machines.

      --
      sub f{($f)=@_;print"$f(q{$f});";}f(q{sub f{($f)=@_;print"$f(q{$f});";}f});
    7. Re:We're Geeks! Dark Secret Heart? Please!! by Anonymous Coward · · Score: 0

      "Easily" and "Millennium Prize Problem level difficulty" are apparently the same thing in your language...

      Apparently, "Not easily" and "easily" are the same thing in your language.

  3. So many things we don't understand... by Dan+East · · Score: 2

    Ahh, the mysteries of the universe that we cannot fathom. Such as why this is a dupe of a story posted just yesterday....
    https://apple.slashdot.org/sto...

    --
    Better known as 318230.
  4. I wonder by religionofpeas · · Score: 1

    Does slashdot use an AI to check stories for duplicates, and does anyone understand how it works ?

    https://apple.slashdot.org/sto...

  5. Hogwash of course by Anonymous Coward · · Score: 0

    A Simple Matter of Programming ( and the commitment of resources to hold the data ). The system merely needs to record its decisions along the way.

    I worked with a rule-based AI circuit router and for each new technology (chip / power rules, etc) we would look at the usual suspect test cases and if any of the routes looked funny fire up the logger and see the decision trace, and tune the AI. Same thing for the chip or board or module development runs if any of the downstream checkers showed any concerns.

    The problem is that full logging takes substantial resources, so it is seldom done. But there are all the usual tricks available to auto-force data collection for shady cases and trimming out fully dependent sequences to eliminate a few orders of magnitude of the data.

  6. Fuck Memes by Anonymous Coward · · Score: 0

    Fuck Slashdot.

    Burn the motherfucker down.

  7. nothing new. by Anonymous Coward · · Score: 0

    I feel the same way about my neighbors' highschool son and daughter.

  8. No one really knows? by Tony+Isaac · · Score: 1

    No, not true. If you work in AI, you know that it's possible to understand how to train an AI, and how to diagnose issues. No, it's not the same as the procedural algorithms we're used to using, but it's not "unknown"!

  9. I knew it! by Gravis+Zero · · Score: 2

    AIs are going to make bodies for themselves and then develop a taste for human flesh. I told people before and they said I was off my meds (which is was but that's irrelevant) but I knew AI had a dark secret it was hiding! #OnlyReadTheHeadline #ThePillsAreTrackers ;)

    --
    Anons need not reply. Questions end with a question mark.
  10. The "secret" is this- by Anonymous Coward · · Score: 0

    AI is just statistics, calculated really fast & used for prediction or behavior mimicry.

  11. Drive like humans--scary by Anonymous Coward · · Score: 0

    I can see the AI coming to the conclusion that red means stop, green means go, yellow means go faster.