Slashdot Mirror
A Big Problem With AI: Even Its Creators Can't Explain How It Works (technologyreview.com)
Last year an experimental vehicle, developed by researchers at the chip maker Nvidia was unlike anything demonstrated by Google, Tesla, or General Motors. The car didn't follow a single instruction provided by an engineer or programmer. Instead, it relied entirely on an algorithm that had taught itself to drive by watching a human do it. Getting a car to drive this way was an impressive feat. But it's also a bit unsettling, since it isn't completely clear how the car makes its decisions, argues an article on MIT Technology Review. From the article: The mysterious mind of this vehicle points to a looming issue with artificial intelligence. The car's underlying AI technology, known as deep learning, has proved very powerful at solving problems in recent years, and it has been widely deployed for tasks like image captioning, voice recognition, and language translation. There is now hope that the same techniques will be able to diagnose deadly diseases, make million-dollar trading decisions, and do countless other things to transform whole industries. But this won't happen -- or shouldn't happen -- unless we find ways of making techniques like deep learning more understandable to their creators and accountable to their users. Otherwise it will be hard to predict when failures might occur -- and it's inevitable they will. That's one reason Nvidia's car is still experimental.
Please come to this thread and explain the need for Robopsychology!
I am Slashdot. Are you Slashdot as well?
http://rocknrollnerd.github.io... - I recommend.
It's really hard to predict what the deep learning is in fact learning. It may be often useful over the training, this very much does not mean that it's going to do the expected when faced with the unexpected, and not for example decide that it should go over an intersection because the person next to it is wearing a green hat that looks more like a green light than the red light looks like a red light.
Cognitive capability developed by an evolutionary algorithm is going to get fuzzy. Maybe you could have a failsafe dumb AI that can tap the brakes.
Its marketing bullshit by people trying to push the idea that current technology is AI, it isn't.
My question is, why are MIT Technology Review articles that show up on Slashdot always so technologically stupid?
I'll tell you what's experimental: msmash's use of "English" - two blatant fuckups in the first goddamn sentence.
How do humans work? Not knowing how genius humans arrive at their conclusions doesn't seem to be a huge stumbling block for society to use their output.
How many scientists really know how "creativity" works?
I've tried to learn some AI techniques, but I run into the following issues: 1. I never took linear algebra in school.
2. I never took advanced statistics in school
3. Everything I have read on the topic of AI requires a fluent knowledge of 1 and 2. I know basic statistics, I can do differential equations (with some difficulty). However, you have to completely think in terms of linear algebra and advanced statistics to have a basic understanding of what's going on. Very few people are taught those subjects.
One of our competitors trademarked the term "hypothesis". From now on, we will call them "boneheaded ideas".
I just don't have any faith in a system that is not fully understood.
But intelligence and consciousness are not fully understood, and may not even be understandable. And I say that not to invoke some kind of mysticism, but because our decision making processes are lots of overlapping heuristics that are selected by yet other fuzzy heuristics. We have this expectation from sci-fi that a general purpose AI is going to be just like us except way faster and always right, but an awful lot of our intelligent behavior relies on making the best guess at the time with incomplete information. Rustling in bushes -> maybe a tiger -> run -> oh it was just a rabbit. Heuristics work until they don't.
It may be that an AI must be fallible, because to err is (to think like a) human. But forgiveness only extends to humans. When the human account representative at your bank mishears you you politely repeat yourself. When the automated system mishears you you curse all machines and demand to speak to a "real person." The real person may not be much better but it doesn't make you as angry when they mishear you. With automobile pilots we tolerate faulty humans whose decision-making processes we absolutely don't understand such that car crashes don't even make the news, but every car AI pilot fender bender will "raise deep questions about the suitability of robots to drive cars."
We don't have a state-run media we have a media-run state.
Based on this statement I'm guessing you've never worked with statistically based machine learning. Take a "simple" artificial neural network trained to do classification. The person who wrote the algorithm knows how samples from the training set are presented to the network, i.e. what features hit the first layer. The author also knows how data propagates through the network (i.e. a value is propagated to the next layer along the edges connected to a previous layer's node) and even how the weighting on different edges connecting the nodes are updated based on classification failures.
Once that network is trained it may spit out correct answers time and time again, but the author who knows the algorithms inside and out doesn't know exactly how the network decides that it's looking at a lunar crater and not a volcano. Not knowing those details means that it is incredibly hard to define how the trained AI will fail when faced with an unexpected input.
There's the problem: if you have a trained AI and not some sort of expert system based on a collection of human knowledge it's nearly impossible to say how it will handle the unexpected near-garbage input.
>There is now hope that the same techniques will be able to diagnose deadly diseases, make million-dollar trading decisions, and do countless other things to transform whole industries. But this won't happen -- or shouldn't happen -- unless we find ways of making techniques like deep learning more understandable to their creators and accountable to their users.
While I care about understanding the system so it can be improved (hopefully before a problem occurs), ultimately all that matters is that it produces statistically better results than a human.
If a machine kills someone (and we don't even know why) 1% of the time, but a human doing the same job would mess up and kill 3% of people (but we'd understand why)... I'll take ignorance.
There are people (commonly called "parents") who have created one or more natural intelligences and can't explain how those work either. Nobody seems to care too much.
Uh, it's simple. Freeze it (disable the feedback loop that lets it modify itself) and test in on a bunch of new data, a bunch of garbage data, etc., and watch it.
If you want to methodically define its behavior you just need to look at the damn thing. Getting any useful info out of that will be an issue though. You may find out that somewhere deep in your neural net it's looking for a seemingly random pattern of contrast or checking against some strange distance/angle. Without tracing its entire training history you won't know why. But you can see that it's checking for that shit and then test it by giving it data that varies a lot on the things it checks, and try to suss out what impact that has in real-world use. No, it's not easy. But it's absolutely knowable and testable.
Uh, it's simple .... No, it's not easy. But it's absolutely knowable and testable.
I agree that it's completely doable, but the poster I replied to was stating that the programmer who wrote the algorithm must understand how it's making decisions and that only the less skilled maintenance coders would be confused. That's simply not true. I know people who could write a neural net from a reasonable spec but doing the steps you described above would blow their minds. I'd also argue that a NN with even a few layers of nodes can get complex fast enough that what you're proposing would result in a document the size of a novel and still not capture all the nuances.
I really appreciate your point that
Getting any useful info out of that will be an issue though. You may find out that somewhere deep in your neural net it's looking for a seemingly random pattern of contrast or checking against some strange distance/angle.
If the net is using some seemingly random pattern that's where you can get some bizarre (to human thinking) failures. We tend to understand when something goes wrong in a way we can comprehend. If the seemingly random pattern the computer finds happens to call a slightly obscured "stop sign" a "no u-turn" sign that would be incomprehensible to a human, but might make perfect sense to the NN.
This all isn't to say that you can't reduce the odds of this sort of problem to such a small number that it's meaningless especially in comparison to human error. Still, when crap like this happens it makes the news and gets blown all out of proportion, so expect "the sky is falling" stories to follow any uncertainty AI behavior.
When an AI can explain how AI works, then maybe I'll believe that it's an AI. Until then ...
Since a human brain can't explain how a brain works, that seems like a silly criteria.