Slashdot Mirror
A Big Problem With AI: Even Its Creators Can't Explain How It Works (technologyreview.com)
Last year an experimental vehicle, developed by researchers at the chip maker Nvidia was unlike anything demonstrated by Google, Tesla, or General Motors. The car didn't follow a single instruction provided by an engineer or programmer. Instead, it relied entirely on an algorithm that had taught itself to drive by watching a human do it. Getting a car to drive this way was an impressive feat. But it's also a bit unsettling, since it isn't completely clear how the car makes its decisions, argues an article on MIT Technology Review. From the article: The mysterious mind of this vehicle points to a looming issue with artificial intelligence. The car's underlying AI technology, known as deep learning, has proved very powerful at solving problems in recent years, and it has been widely deployed for tasks like image captioning, voice recognition, and language translation. There is now hope that the same techniques will be able to diagnose deadly diseases, make million-dollar trading decisions, and do countless other things to transform whole industries. But this won't happen -- or shouldn't happen -- unless we find ways of making techniques like deep learning more understandable to their creators and accountable to their users. Otherwise it will be hard to predict when failures might occur -- and it's inevitable they will. That's one reason Nvidia's car is still experimental.
Cognitive capability developed by an evolutionary algorithm is going to get fuzzy. Maybe you could have a failsafe dumb AI that can tap the brakes.
Its marketing bullshit by people trying to push the idea that current technology is AI, it isn't.
My question is, why are MIT Technology Review articles that show up on Slashdot always so technologically stupid?
I just don't have any faith in a system that is not fully understood. Just like back in college, you would create some cludge code without proper understanding of underlying concepts and sometimes it would work. However, this would never produce a robust system.
The same idea applies here.
that have been around since the 18th century. The problem solutions formulated using it have been misleadingly hyped as AI. Be deceived if your wish.
How do humans work? Not knowing how genius humans arrive at their conclusions doesn't seem to be a huge stumbling block for society to use their output.
How many scientists really know how "creativity" works?
Script kiddies using somebody else's black box cannot explain how these systems work. These are self proclaimed experts and are certainly not really experts or creators of good code.
Today's well designed neural networks and other machine learning systems can certainly be fully understood and debugged.
Greed is the root of all evil.
>There is now hope that the same techniques will be able to diagnose deadly diseases, make million-dollar trading decisions, and do countless other things to transform whole industries. But this won't happen -- or shouldn't happen -- unless we find ways of making techniques like deep learning more understandable to their creators and accountable to their users.
While I care about understanding the system so it can be improved (hopefully before a problem occurs), ultimately all that matters is that it produces statistically better results than a human.
If a machine kills someone (and we don't even know why) 1% of the time, but a human doing the same job would mess up and kill 3% of people (but we'd understand why)... I'll take ignorance.
Space Nutters aren't scientists.
There are people (commonly called "parents") who have created one or more natural intelligences and can't explain how those work either. Nobody seems to care too much.
Uh, it's simple. Freeze it (disable the feedback loop that lets it modify itself) and test in on a bunch of new data, a bunch of garbage data, etc., and watch it.
If you want to methodically define its behavior you just need to look at the damn thing. Getting any useful info out of that will be an issue though. You may find out that somewhere deep in your neural net it's looking for a seemingly random pattern of contrast or checking against some strange distance/angle. Without tracing its entire training history you won't know why. But you can see that it's checking for that shit and then test it by giving it data that varies a lot on the things it checks, and try to suss out what impact that has in real-world use. No, it's not easy. But it's absolutely knowable and testable.
Uh, it's simple .... No, it's not easy. But it's absolutely knowable and testable.
I agree that it's completely doable, but the poster I replied to was stating that the programmer who wrote the algorithm must understand how it's making decisions and that only the less skilled maintenance coders would be confused. That's simply not true. I know people who could write a neural net from a reasonable spec but doing the steps you described above would blow their minds. I'd also argue that a NN with even a few layers of nodes can get complex fast enough that what you're proposing would result in a document the size of a novel and still not capture all the nuances.
I really appreciate your point that
Getting any useful info out of that will be an issue though. You may find out that somewhere deep in your neural net it's looking for a seemingly random pattern of contrast or checking against some strange distance/angle.
If the net is using some seemingly random pattern that's where you can get some bizarre (to human thinking) failures. We tend to understand when something goes wrong in a way we can comprehend. If the seemingly random pattern the computer finds happens to call a slightly obscured "stop sign" a "no u-turn" sign that would be incomprehensible to a human, but might make perfect sense to the NN.
This all isn't to say that you can't reduce the odds of this sort of problem to such a small number that it's meaningless especially in comparison to human error. Still, when crap like this happens it makes the news and gets blown all out of proportion, so expect "the sky is falling" stories to follow any uncertainty AI behavior.