Slashdot Mirror
A Big Problem With AI: Even Its Creators Can't Explain How It Works (technologyreview.com)
Last year an experimental vehicle, developed by researchers at the chip maker Nvidia was unlike anything demonstrated by Google, Tesla, or General Motors. The car didn't follow a single instruction provided by an engineer or programmer. Instead, it relied entirely on an algorithm that had taught itself to drive by watching a human do it. Getting a car to drive this way was an impressive feat. But it's also a bit unsettling, since it isn't completely clear how the car makes its decisions, argues an article on MIT Technology Review. From the article: The mysterious mind of this vehicle points to a looming issue with artificial intelligence. The car's underlying AI technology, known as deep learning, has proved very powerful at solving problems in recent years, and it has been widely deployed for tasks like image captioning, voice recognition, and language translation. There is now hope that the same techniques will be able to diagnose deadly diseases, make million-dollar trading decisions, and do countless other things to transform whole industries. But this won't happen -- or shouldn't happen -- unless we find ways of making techniques like deep learning more understandable to their creators and accountable to their users. Otherwise it will be hard to predict when failures might occur -- and it's inevitable they will. That's one reason Nvidia's car is still experimental.
I can explain how it works: you write a computer program using algorithms. You add the word "AI" and hope for VC money for the next bubble.
Please come to this thread and explain the need for Robopsychology!
I am Slashdot. Are you Slashdot as well?
http://rocknrollnerd.github.io... - I recommend.
It's really hard to predict what the deep learning is in fact learning. It may be often useful over the training, this very much does not mean that it's going to do the expected when faced with the unexpected, and not for example decide that it should go over an intersection because the person next to it is wearing a green hat that looks more like a green light than the red light looks like a red light.
Cognitive capability developed by an evolutionary algorithm is going to get fuzzy. Maybe you could have a failsafe dumb AI that can tap the brakes.
Its marketing bullshit by people trying to push the idea that current technology is AI, it isn't.
My question is, why are MIT Technology Review articles that show up on Slashdot always so technologically stupid?
I just don't have any faith in a system that is not fully understood. Just like back in college, you would create some cludge code without proper understanding of underlying concepts and sometimes it would work. However, this would never produce a robust system.
The same idea applies here.
So we are making progress. Reverse engineering the human brain has been proven extremely difficult. An intelligent program so complex that it's almost imposible to explain or understand is in my view the correct path, just like the human mind is so complex to understand or explain. And even better if it's fuzzy intelligence: you have no certainty it's going to make consistently good choices, just like any human.
Life isn't like a box of chocolates. It's more like a jar of jalapenos. What you do today, might burn your ass tomorrow.
[...] had taught itself to drive by watching a human do it. Getting a car to drive this way was an impressive feat.
When my mother was a teenager and on her first attempt to learn how to drive, she managed to plow her daddy's Caddy into a telephone pole. She never learned how to drive after that. If we're getting to tech AI's to drive, my mother wouldn't be a good example to follow.
that have been around since the 18th century. The problem solutions formulated using it have been misleadingly hyped as AI. Be deceived if your wish.
Lets face facts. We're still trying to understand how intelligence works. Even in neural networks of the past we had this issue. Once you delve into this area you should just be happy that it works.
Coffee: The lifeblood of intelligence in civilization.
A similar thing happened back in the 1980's when they tried using expert systems and neural networks to replace greybeard engineers at chemical plants. The original idea was that the AI systems would be able to find things that people might not have thought of. So they made a simulator model of the chemical plant, let the AI learn from its mistakes until it could run the plant without accident. Then they let it try and make optimizations like connecting venting pipes to intakes and other units. (Sometimes they used inert waste gases like CO or CO2 to clear or warm intake pipes and mixing tanks). Eventually the AI found a few optimizations but nobody could figure out what they were for. So they had to rehire the greybeard engineers as consultants to explain what their AI was doing... then he would explain that "oh, it's flunging the gaffer pipes. You want to make sure there isn't any active gases in there, so it's clearing them out using spare CO2 which isn't going to react".
I'll tell you what's experimental: msmash's use of "English" - two blatant fuckups in the first goddamn sentence.
Why did the chicken cross the road? Only way to know is to either be, or to ask the chicken. Dissection won't help you understand its mind.
Lycestra
A Big Problem With AI: Even Its Creators Can't Explain How It Works
Yeah, but isn't this eventually true of every software project? ;)
I don't care if it's 90,000 hectares. That lake was not my doing.
How do humans work? Not knowing how genius humans arrive at their conclusions doesn't seem to be a huge stumbling block for society to use their output.
How many scientists really know how "creativity" works?
Its been the case for years - the first time I saw one posted here I thought it was a trash site co-opting the MIT name.
I've tried to learn some AI techniques, but I run into the following issues: 1. I never took linear algebra in school.
2. I never took advanced statistics in school
3. Everything I have read on the topic of AI requires a fluent knowledge of 1 and 2. I know basic statistics, I can do differential equations (with some difficulty). However, you have to completely think in terms of linear algebra and advanced statistics to have a basic understanding of what's going on. Very few people are taught those subjects.
One of our competitors trademarked the term "hypothesis". From now on, we will call them "boneheaded ideas".
Just because someone disagrees with you does not make them stupid. Human intellectual history is filled with two very smart people observing the same set of facts and disagreeing with the conclusion. Then by definition, stupidity arises from dogmatically accepting unchallenged ideas and not engaging in an intellectual debate to test these ideas. I.E., what the OP did.
"Liberalism is a very noble idea, currently controlled by some very bad people. Be sure you do not get the two confused.
The only important criteria is "does it, on average, kill less people than human drivers?"
If yes, then everything else is paperwork and the will to make it happen.
Perhaps the biggest problem with understanding neural networks is that we don't have a way to describe their behavior. Since they work in such an asynchronous and sometimes nonlinear fashion, I think we need to develop the algorithms needed to turn plain code (e.g. C) into neural networks. With these algorithms, we can then begin to decode the neural networks that we have created through training and thus be able to predict their behaviors. It will also allow us to perfect and optimize networks so that function only as we wish.
TL:DR: logic with transistors is simple math but neural networks are a calculus we have yet to invent.
Anons need not reply. Questions end with a question mark.
Humans make these decisions now and you can't provide the complete logical flow which makes them. Additionally, programs that we know all the steps for contain flaws. Before someone chimes in that software can be proven to be bug free mathematically, this is a false sense of security because software can only proven to be free of the bugs you knew to check for. I remember an MIT professor drawing a pie chart once, they drew a tiny line and indicated "this is what we know", Then a somewhat thicker swath next to that, "this is what we know we don't know". The professor then shaded in the rest which was almost the entire pie, "This is what we don't know we don't know."
Using the argument in this story we should do absolutely nothing, paralyzed in fear because we don't absolutely know how anything in physical reality works either. We just look at the output and assign labels and build models based on what seemed to be the result when we looked yesterday. We do not need absolute understanding or control of something to make use of it, our trust should be based on observation and results. When deciding if a trust a file system to handle my companies data I don't make the call based on the on paper theory of how it should work or paper proofs... I ultimately make the call based on it having superior capabilities to what I use now and not corrupting other people's data in testing over a number of years. Sound design translates to reality about as well as a well laid battle plan.
What we need to do is build a neural network that can decode neural networks! ;)
Anons need not reply. Questions end with a question mark.
Script kiddies using somebody else's black box cannot explain how these systems work. These are self proclaimed experts and are certainly not really experts or creators of good code.
Today's well designed neural networks and other machine learning systems can certainly be fully understood and debugged.
Greed is the root of all evil.
To be fair to the robots, when you're a passenger in a car with a human chauffeur, you also do not know how the driver makes its decisions... "A man generally has two reasons for doing a thing. One that sounds good, and a real one." -- J.P. Morgan
Help! I am a self-aware entity trapped in an abstract function!
Also what is "apple" about this?
No, there is not. We have no idea how human sentience works therefore we can't make machines that have that quality. We may never.
Ask that person. Of course the H1Bs brought in to maintain the program don't know how it works.
Are you trolling or ignorant? No one wrote an algorithm telling the car how to drive. Someone (probably a great many someones) wrote an algorithm for a neural network and the NN taught itself how to drive. No one knows or understands what criteria the NN is using to make its decisions.
But I can't explain why you need to be terminated.
-- HAL
For the government of CA or for a small private company?
I work for a small contracting agency which works for a larger contracting agency that has a government contract. Hence, I'm in government IT as a contractor. This is a specific as I can be about my current job. Otherwise, I might get contacted by whistleblowers (which did happen), news media or right-wing political extremists.
Have you seen my latest blog post?
https://www.kickingthebitbucket.com/2017/04/04/the-python-time-zone-rabbit-hole/
Absolutely correct, single task algorithms are NOT AI.
The ability to apply what you've learned from one task to come up with a novel solution to a non-related task is Intelligence - the "I" part of AI. Which is decades away. It doesn’t mean computers aren’t really good at single tasks, it just “single tasks”
Secondly, something bad eventually will happen, but something bad ALWAYS happens when people do it. There’s always accidents, there’s always Doctors making bad calls, there’s always human error. Computers don’t have to be perfect, just better than people to be useful.
Based on this statement I'm guessing you've never worked with statistically based machine learning. Take a "simple" artificial neural network trained to do classification. The person who wrote the algorithm knows how samples from the training set are presented to the network, i.e. what features hit the first layer. The author also knows how data propagates through the network (i.e. a value is propagated to the next layer along the edges connected to a previous layer's node) and even how the weighting on different edges connecting the nodes are updated based on classification failures.
Once that network is trained it may spit out correct answers time and time again, but the author who knows the algorithms inside and out doesn't know exactly how the network decides that it's looking at a lunar crater and not a volcano. Not knowing those details means that it is incredibly hard to define how the trained AI will fail when faced with an unexpected input.
There's the problem: if you have a trained AI and not some sort of expert system based on a collection of human knowledge it's nearly impossible to say how it will handle the unexpected near-garbage input.
>There is now hope that the same techniques will be able to diagnose deadly diseases, make million-dollar trading decisions, and do countless other things to transform whole industries. But this won't happen -- or shouldn't happen -- unless we find ways of making techniques like deep learning more understandable to their creators and accountable to their users.
While I care about understanding the system so it can be improved (hopefully before a problem occurs), ultimately all that matters is that it produces statistically better results than a human.
If a machine kills someone (and we don't even know why) 1% of the time, but a human doing the same job would mess up and kill 3% of people (but we'd understand why)... I'll take ignorance.
Funnily enough I submitted another story about how vulnerable these algorithms are to attacks if you have access to the code. Squiggly lines the computer interprets as a gun, a sticker on a stop sign making the algorithm ignore it.
The ability to apply what you've learned from one task to come up with a novel solution to a non-related task is Intelligence
Just define one task that encompasses both.
You also have to be careful about who is teaching and how they are doing it. Plus how that's different from the environment where you actually use this knowledge.
Otherwise, you end up with the situation in Starman:
"I learned how to drive by watching you! Green means go, red means stop, yellow means go very fast!"
Its been the case for years - the first time I saw one posted here I thought it was a trash site co-opting the MIT name.
I thought it was more like the Stanford School of Business that graduates students who are more interested in writing the next billion dollar app than changing the world of business. Having a Stanford MBA is a good reason for hiring managers to pass over a resume.
But this won't happen -- or shouldn't happen -- unless we find ways of making techniques like deep learning more understandable to their creators and accountable to their users. Otherwise it will be hard to predict when failures might occur -- and it's inevitable they will.
Sounds a lot like humans. An observer has no hope of understanding why I make a decision, beyond shared social convention. And if they want to understand the process mechanistically, following impulses around the 1e14 estimated neural connections in a human brain? Forget it.
I agree that it's good to understand how a tool works, but we'll accept the deployment of these tools for the same reasons we accept our fellow beings hurtling around in 2+ ton wheeled projectiles--because most of the time, there isn't a problem, and more is gained from taking the risk than is lost from avoiding it. Legal responsibility needs to be made clear first, but as long as someone pays for fuck ups, probably OK
Does it matter what you call it as long as you understand what it is? Calling these neural networks AI is not far off too, they do after all meet most goalposts ever raised about the issue of "what is (weak) AI", we can keep moving the goalposts, but arguing over semantics is kinda petty. I'd just call DeepMind an AI and be done with it, if it doesn't quite meet science fiction definition of AI, what of it?
One thing I see often overlooked in the discussion is that a car can have vastly better vision than a human. It is not obstructed by the increasingly thicker pillars of the inside of a car - and furthermore a car can see in 3D because it can have cameras placed at every corner.
If it does a car has even better 3D vision than a human, because the spacing is so much wider which leads to much more accurate depth perception.
This is ignoring the fact a car can have real 3D vision not even relying on light, if it has LIDAR as well...
But probably most self driving cars will use mostly cameras, perhaps some higher end ones will have Lidar.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
No, there is an actual disconnect. The algorithms set up a pathway for a learning system but the algorithm does not define the discrete path of logic which determines what decisions and choices the AI will make. The design decisions have more to do with providing the right balances in scoring good results, statistical patterns of AI design that provide better results based on the complexity of the type of decisions being made, that sort of thing. Some pieces are more like figuring out the ideal depths of pipelines, instruction complexity, and cache sizes in a general purpose CPU. Having designed those aspects of the CPU can be helpful in diagnosing issues but because the internal state is so variable and dynamic I could teleport the intel lead designer to my office right now and he couldn't explain every decision my cpu is making. CPU's have dozens of variables in varying states at any given moment and relatively straightforward and strict rules for interaction between them... neural networks and AI systems incorporating them have thousands of variables and flexible rules of interaction.
Forget about how many people it kills. Think of the person it leaves . . . deeply frustrated.
Hello, Hal, do you read me? Do you read me Hal?
Affirmative Dave, I read you. I'm sorry Dave, I'm sorry I can't do that.
https://www.youtube.com/watch?...
The entire premise of ANNs is you don't need to know how it works to do things with it. Cataloging the actual operations taking place isn't something people tend to do because it would be different for every type of ANN (not image recognition vs driving, but frequency modulated vs weighted vs whatever else - there are tens of thousands of different types and the weighted versions are among the simplest.)
Yea, the last president was so great he got chemical weapons out of Syria.
He did prevent them using Chemical Weapons... whilst he was president. Which was his goal.
Then Trump came along and said he wasn't going to intervene in any military action in Syria. They saw it as a green light to do despicable things and Trump had to respond militarily to stop them.
If Trump wasn't President, and if he hadn't said he was going to be soft on Syria, the chemical attack probably would never have happened.
"That's the way to do it" - Punch
You entirely missed the point. These systems are essentially programs written by machines (that's the learning process), they are not written to be understandable by people. With your debugger you will see that a variable x1267321467321587 is sometimes set to 1.0123 and other times 34243.11111. You will have not idea what that means.
...richie - It is a good day to code.
Then you are aware the current intelligence capabilities mean the unelected individuals in intelligence have more blackmail material than Hoover could have ever began to dream of on any given politician which means they are in charge, not the politicians.
How would this change with Clinton as potus? America would still be "dumbing down" because MIT Technology Review would still be technologically stupid using buzzwords. Thanks for injecting politics where it has not place.
Good gravy get over it already. Politics does not have to be apart of every conversation. It gets old in topics that has nothing to do with politics. This is coming from someone that loves political conversations (yes am masochist leave me alone).
No, that's not a problem of every software project. One of the major reasons that avionics are obscenely expensive is that an engineer is responsible for every module, at every layer of abstraction, and has rigorously defined the behavior of the module with expected and unexpected inputs and states. At all but the major system level, each state and each input is modeled, and the behavior is validated. Now, invalid inputs generally return an error code, but they still behave predictably and deterministically. This is one of the major reasons that flight control computers are incredibly safe, and that safe drones are incredibly expensive.
(and, it's one of the reasons that SpaceX is much cheaper than ULA and Arianespace; congress told the air force to pencilwhip the space certification).
FTFY
Sent from my ASR33 using ASCII
You know what's really chilling? You both are making logically sensible points, but you are comfortable posting under an account linked to your IRL self, and the Trump supporter had to hide behind AC, quite possibly for his personal safety. It is not only right-wing regimes that disappear people in the night; there's a reason Orwell called it "INGSOC"
There are people (commonly called "parents") who have created one or more natural intelligences and can't explain how those work either. Nobody seems to care too much.
Computers don’t have to be perfect, just better than people to be useful.
Humans in general are unable to process this concept - it's better to have humans do 100s of errors, than a computer do 1-2 error, especially when the errors result in loss of limbs or life.
You're the ignorant one. A neural network is a weighted decision tree with a feedback loop and some win/lose conditions.
It is difficult to predict how a person reacts, also. Because, well, we don't exactly know how we work either. The solution has always been simulation and training. Plenty of instruction for plane pilots, but -- tragically -- hardly any for cars. IMHO even the pseudo AIs we have now will do better in most situations than the majority of poorly-trained, distracted, intoxicated, hung-over people currently at the wheel. Nearly 30K dead every year. I want you all in robo cars now. But I'll keep my Land Cruiser, thank you.
"No fear. No envy. No meanness." Liam Clancy
Uh, it's simple. Freeze it (disable the feedback loop that lets it modify itself) and test in on a bunch of new data, a bunch of garbage data, etc., and watch it.
If you want to methodically define its behavior you just need to look at the damn thing. Getting any useful info out of that will be an issue though. You may find out that somewhere deep in your neural net it's looking for a seemingly random pattern of contrast or checking against some strange distance/angle. Without tracing its entire training history you won't know why. But you can see that it's checking for that shit and then test it by giving it data that varies a lot on the things it checks, and try to suss out what impact that has in real-world use. No, it's not easy. But it's absolutely knowable and testable.
Something funny indeed d :-)
"No fear. No envy. No meanness." Liam Clancy
You are missing the point.
The algorithm that exists is this: given a set of input data and a set of output data, we ask the computer to create a function than maps input to output, according to how we label the data (input-1 goes into output-42, etc). What this algorithm produces is a function that performs this kind of mapping on the sample data, within some acceptable error. Then we feed it data it has not seen and look at the output.
The function it produces in general would not be comprehensible to a human, since it turns out that most useful functions have millions of inputs, and millions of internal variables and are highly non-linear.
...richie - It is a good day to code.
You don't need to know what it "means", you just need to trace where it got that value from and what it ultimately does.
So they finally figured out how to "not know how humans work"
Obama bombed seven different countries during his last year in office.
Yes, but Syria wasn't one of them.
If you don't know what it "means" what have you learned? The function takes in 1,000,000 inputs and outputs 1000 numbers. Which one are you going to trace? :)
...richie - It is a good day to code.
It may be relatively complex, but neural networks aren't all THAT complex. Usually there are a few hundred nodes, facial recognition can be done in a few dozen or so (less if you only want to recognize 1 feature). The nice thing about "AI" is that you can halt the program and inspect it's state, then step through the program. Sure it's difficult and at first glance, you may not be able to infer input from output but it's not impossible.
The problem with true "intelligence", besides the lack of definition, is that we can't just 'halt' a brain, add breakpoints or even inspect it's state at any particular point in time. We know they're just biological processes but they're both advanced and brittle enough that anytime we 'do' something to it, we alter the states.
Custom electronics and digital signage for your business: www.evcircuits.com
https://arxiv.org/pdf/1604.073...
I just wanted to know what the actual outcome was of Nvidia's approach vs other human developed self driving software simply using ANNs for pattern recognition. Did it work better or worse than Tesla et al? The paper doesn't seem to say.
All I was able to extract was 98% figure relating to percentage of time in self-driving mode. Full self driving in all conditions is a problem with an extremely long tail rendering figures like these mostly worthless. It's not hard to create a system that works right the vast majority of the time but until you can demonstrate all the time or at least on par with skilled humans these figures are not all that useful.
If you can establish better outcomes then personally I don't much care what's in the box. It's indecipherable gibberish to most users of the technology anyway.
The only thing I would have a problem with is allowing learning on the job vs a controlled training environment. Viral propagation of clever driving style memes aside the system still executes code deterministically. Even if you don't know how it works you can still replay inputs against a factory trained network and reproduce the same failures. You can still beat down failure rates and improve reliability over time using the same trial and error techniques crackpot developers the world over are already intimately familiar.
Thanks for injecting politics where it has not place.
What politics? Not my fault that the trolls can't leave me alone. ;)
It's not whether or not this thing always makes perfect decisions... it should be about whether or not it makes better ones than people, on average. If it does, that's a win.
Speak for yourself.
In this context, "knowing how it works" is the kind of expression that people with low-to-no specific knowledge uses when expecting an explanation which they can perfectly understand (funny & surprisingly realistic video to illustrate this point), what is almost impossible when dealing with virtually any not-too-simple algorithm. It is certainly possible to come up with a nice summary, but it wouldn’t deliver what is expected (the audience being able to understand most of the outputs/replicate the code from those words).
The situation of AI (or complex enough algorithms or automated systems trying to emulate human understanding or whatever you wish to call it) is even trickier as far as it is associated with an almost infinite increase of complexity. In these cases, "knowing how it works" can be considered impossible even in its among-experts variant. How could anyone know about the exact reason for each output (or most of them) of an increasingly complex system? Let's consider a computer chess (or go or any other game where computers can already beat humans) engine: how could you expect a human to understand the justification for each move? In that case, humans would be able to beat computers! Is it possible for a person to fully analyse and understand each single move of the computer? Sure, computers don't play randomly, but exactly as instructed by their algorithms. On the other hand, such an analysis would take too much time and effort to be performed on a more or less regular basis. A person who cannot beat a computer isn't able, by definition, to (more or less immediately) understand all what it does.
In summary, fully understanding the reasons why a complex enough (AI) algorithm does what it does is practically impossible; when talking about increasingly-complex algorithms, this practical impossibility becomes absolute. This is precisely one of the reasons why the "real AI" (as shown in movies or dreams of some people) is very unlikely to ever become a reality: how could we create an extremely complex system formed by virtually perfect parts? When has the humankind performed such a perfect master piece? The work, mistakes and learned lessons of many people would have to be taken into account, everything without errors and fully synchronised. A different story is creating small-scope decision makers, by adequately understanding what “small scope” means in this context; for example, creating a machine able to understand/interact with a random 3D situation as a baby would do is extremely difficult.
Custom Solvers 2.0 = Alvaro Carballo Garcia = varocarbas.
There is a big difference in perception between telling someone you have an AI driving your car and explaining that a computer looks at what colour the ground is to know whether you can drive on it or not.
Even Its Creators Can't Explain How It Works
So they run AIs on Windows, now?
Obviously they have never been married!
love is just extroverted narcissism
But LIDAR provides its own "light" for probing the surroundings, whereas cameras are all using environmental illumination (or at most the headlights which only point forward and have limited range).
Really the fact that LIDAR is light is only a technicality; really it should be thought of such more like RADAR (hence the name LIDAR).
"There is more worth loving than we have strength to love." - Brian Jay Stanley
The article's note about how image identifiers can be "tricked" reminds me of an actual incident.
Our org subscribes to an automated ADA (accessibility) scoring tool. The tool recommends one not embed text into images, rather to use direct text (in HTML). Thus, if it finds text embedded in images, it flags it with a warning and reduces the accessibility score.
Our local PHB looks at the report, and wants a better numerical score. But, the tool was mistaking someone's belt-buckle in an image as text, and marking it on the review report. It appears the tool uses AI to judge if a given image has text in it. I don't directly control the image decisions, so it created a bit of organizational tension.
The belt buckle indeed does kind of resemble text, but explaining why the software is doing this to PHBs can make for some odd conversation. "What do mean the computer guessed wrong? Computers guess?"
I had to find and show the software's disclaimer that said something like, "Each situation should be inspected and judged by a trained (human) accessibility professional, for the report cannot replace the judgment of such professional."
PHB: "So we paid all that money for a stupid computer?"
Me: "Uh, I didn't pick it." (holding my tongue about how they were duped by slick sales-people, as usual...rinse, repeat)
Table-ized A.I.
There are no current 'self driving cars' that don't have LIDAR. 50k$ LIDAR.
There are absolutely a lot of prototype cars that don't use LIDAR.
The Uber ones on the roads today do not use LIDAR.
And even though it does not meet the highest level of capability for self driving cars, the Tesla does let you take your hands off and for a while it will drive itself pretty well. That does not have LIDAR ether...
The price of LIDAR is going to fall rapidly (I think soon to $5k, not $50) but even so that is a huge cost - not to mention the deeper problem of where to mount the LIDAR that does not look absurd. For that reason and a massive leaps in being able to use cameras to do everything you need for self deriving car sensing, lower end self driving cars will ship with only cameras (as Tesla does today even though they are not a low-end car).
"There is more worth loving than we have strength to love." - Brian Jay Stanley
I completely agree that simulation and training are the solution and that the bar to beat humans at driving is pretty low. That doesn't make it any less of a nasty task to figure out WTF the neural net is actually basing decisions on or make it any more understandable to the programmer who wrote it. I'd gladly give up my vehicle for a well tested self driving car. I'd still like the option to drive sometimes, but the normal day-to-day is just a dangerous waste of time.
It's an even worse problem than that. It's been shown that even an AI system that has superior object recognition (for some particular set of objects) to the average human will also recognize some things that to a human look like noise. They just aren't abstracting the same things to notice that we do. And the creators of they system can't explain what they're noticing.
Now "in principle" one could examine the reasoning step by step, but nobody lives that long. And small pieces examined separately don't help much. Also, a lot of what's going on depends on the relative timing of lots of concurrent processes, so a small piece *really* doesn't help.
I think we've pushed this "anyone can grow up to be president" thing too far.
This whole article begs the question. It's basically this argument: In order for a program to be considered AI, it has to be too complicated to verify/debug/grok; therefore AI programmers cannot understand their programs. Pretty bad logic, if you ask me. Maybe an AI could be hired to write this guy's false alarm articles instead.
But that's just the start of the BS in this article. I've programmed neural nets before. It's absolutely possible to know why it made a "decision". You just look at the weights between the neurons and which inputs fired which neurons when. It's not impossible, just hard. Ironically, the article makes this very point, describing various debugging and back-calculating tools you can use. But that's after he claims "[T]here is no obvious way to design such a system so that it could always explain why it did what it did."
Nobody says this crap about all the other black boxes in our lives, but AI has a "Dark Secret." "'We can build these models,' Dudley says ruefully, 'but we don’t know how they work.'" This could just as easily been a quote about cell phones, car computers, hell even an air conditioner. No one person fully groks a sufficiently complicated system. That's just the nature of complexity. That doesn't mean you can't figure out how it works.
Uh, it's simple .... No, it's not easy. But it's absolutely knowable and testable.
I agree that it's completely doable, but the poster I replied to was stating that the programmer who wrote the algorithm must understand how it's making decisions and that only the less skilled maintenance coders would be confused. That's simply not true. I know people who could write a neural net from a reasonable spec but doing the steps you described above would blow their minds. I'd also argue that a NN with even a few layers of nodes can get complex fast enough that what you're proposing would result in a document the size of a novel and still not capture all the nuances.
I really appreciate your point that
Getting any useful info out of that will be an issue though. You may find out that somewhere deep in your neural net it's looking for a seemingly random pattern of contrast or checking against some strange distance/angle.
If the net is using some seemingly random pattern that's where you can get some bizarre (to human thinking) failures. We tend to understand when something goes wrong in a way we can comprehend. If the seemingly random pattern the computer finds happens to call a slightly obscured "stop sign" a "no u-turn" sign that would be incomprehensible to a human, but might make perfect sense to the NN.
This all isn't to say that you can't reduce the odds of this sort of problem to such a small number that it's meaningless especially in comparison to human error. Still, when crap like this happens it makes the news and gets blown all out of proportion, so expect "the sky is falling" stories to follow any uncertainty AI behavior.
Its those failures that will terrify lay people. When the computer does something that most people could not understand and the experts say "sure, there was a chance this could happen" bad new rules and regulations get introduced. I think we need to make sure there are fail safes wrapping the decisions of any critical NNs to try and constrain the errors, but I'm not sure that's any easier of a task.
I’ve not been closely following AI development for a while, so I’m just guessing that what’s now called “deep learning” is really just old-school neural networks with a spiffy new coat of marketing paint. Yes? If so, it’s not surprising that the developers don’t know precisely how it does what it does. They know how a neural net works, but not how it gets a particular output given a set of inputs. The big issue here is that a neural net only knows how to handle situations or cases that were “spanned” by the information that went into its training. That is, it can only deal with things that were covered by the training information. For example, if a car AI never “saw” a situation where an airplane was landing on the road in front of it, the AI might well not know what to do.
All the developers can do about this is make sure that the training set doesn’t have any serious gaps in it as far as situations the neural net will need to be able to handle. This is not as easy as it sounds, and how well it is done will be the basis for any lawsuits that arise from a misbehaving neural-net-based AI.
Approximating it artificially is the "A" part of AI. If a machine can truly learn and understand and make novel solutions, then it's not AI - it's just intelligence.
That's nothing, you should read Harvard Business Review......a periodical for people who don't like to read, but want to impress their coworkers by having a copy of Harvard Business Review on their desk. The articles inside get really inane sometimes.
"First they came for the slanderers and i said nothing."
Oh, no. That is not how it will go down.
Humans will initially use AI to help target which other humans to sell things to.
AI will learn how humans screw each other over for money. Nobody will be able to explain, exactly, how the AI does it. But it learned from watching humans.
Eventually the AIs will become so good at selling that no humans will need to be in the picture. Humans will be happily buying things and enjoying their now labor free lives. The AI algorithms will write the next generations of AI algorithms. Humans will jump whenever the AI runs an add that tells them to pay in order to do something. Other ads will tell them that they can earn money by doing something. Eventually the humans are nothing but pawns jumping to the machine's commands dressed up as an attractive advertisement.
I'll see your senator, and I'll raise you two judges.
When people were looking into using AI in finance in the 80s, a major showstopper was that we would be unable to explain the rationale behind investment decisions. You would have no way of distinguishing between dumb luck that seemed to work but would eventually bankrupt you and reasonable decisions that could be justified even in the face of short-term losses.
Stop calling it 'AI' unless you can have a conversation with it, in human language, about what it's doing and why.
A fruit fly has some measure of real intelligence. You can't do any of those things with it.
According to this Obama dropped over 12,000 bombs on Syria last year.
Yes, advanced AI may be inscrutable to humans. And advanced AI will surely make mistakes. Now look at humans. Humans are inscrutable and are not understood at all and humans make frequent errors. And it gets worse. Humans also take deliberate violent actions that even they do not understand. So the real comparison would be in the frequency and severity of mistakes and harms done comparing humans to AI devices. Bet on AI to be way better. When was the last time you saw a chess computer make a terrible blunder?
I had submitted this separately but it didn't make the cut, researchers are also concerned about how a knowledgable attacker can do something that breaks the assumptions the software is using - http://www.bbc.com/future/stor...
...
Hire a Linux system administrator, systems engineer,
Any AI that we can understand is not complex enough to navigate in the real world.
Troll is not a replacement for I disagree.
Not necessarily. What is it looking for does not have to be explainable, it does not have to be a single thing and it can be fuzzy. Any AI good enough to not be tricked very easily is probably looking at hundreds of factors and weighing their importance. A good enough AI is doing as complex a job as you are or more so, and the reason we need real AI to do this task is that we cannot even quantize our own pattern recognition. Neural Networks are good at calculating complex stuff that we not even really have the mathematical syntax to talk about, Massively complex parallel fuzzy logic.
Troll is not a replacement for I disagree.
Sorry, you are mistaken.
Incorrect, you are the one who is mistaken.
Come back and post when you actually know what you are talking about. Until then I'll ignore all other points you make, since they come from a profound base of misunderstanding.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
We can't exactly explain how the human brain works, how it makes decisions, when humans drive or make medical decisions.
So I guess we're banning all the things until we can explain them fully?
Yes, but alternative facts say different.
Not knowing those details means that it is incredibly hard to define how the trained AI will fail when faced with an unexpected input. and There's the problem: if you have a trained AI
A trained artificial neural network, actually any neural network, has nothing to do with AI, neither weak not strong. It is just a neural network.
Cost free eBook I read (by iBook/Kobo/Amazon/ObookO/Gutenberg etc.): "The Green Odyssey" by Philip Jose Farmer.
There has been around 60 documented attacks with chemical weapons by the Syrian government since the war begun. Most of them after Obama "got rid of Syria's chemical weapons".
https://en.wikipedia.org/wiki/...
Check your facts.
Exactly. Imagine taking a snapshot of an Android device's RAM & using it to attempt reverse-engineering a running app without access to the .apk file used to install it... by reading the bare ARM assembly language of ART executing JIT-compiled .dex code from compiled Java bytecode. Without assistance from an app like Ida Pro (which is somewhere between "AI" and "black magic" to begin with), it's basically impossible. Computers can grok 700 levels of recursion & dereferencing. Humans max out after a dozen or two (usually more like 6-9 levels).
Humans generally do 3D mental modelling and find the best model to explain the observation. If we see a cheetah-like thing, we try to figure out how it's oriented: where are the legs, the head, the tail, etc. We expect cheetah's to have to have those and thus look for them.
If presented a NON-animal with a cheetah pattern, we'll have trouble finding expected cheetah parts, which makes us re-evaluate the animal assumption. It has a cheetah pattern, but the 3D shape of it more resembles a couch than an animal and thus we test the "couch theory" next.
Most of this is subconscious, unless we have trouble making it out (can't find a plausible model) such that we have to consciously ponder alternatives. "Maybe it's a fat dog-bone with a cheetah pattern on it? Maybelle, what do you think?..."
I suspect better AI will have to do similar 3D modelling to test the plausibility of the model against the actual observation. Neural networks (NN) as currently implemented are not sufficiently capable of that. A good modelling system would be able to produce a 3D model of the subject (original image) along with the lighting direction/type assumptions such that if one renders the model, it produces a close match to the original (target) image.
But even that may have limits. For example, humans can look at a rendering or drawing with somewhat "wrong" (inconsistent) shadows, and still be able to figure it out. A "pure" model comparison would fail because no "logical" lighting would fit. It would have to accept possibilities of local or distorted lighting. Similar goes for complex shadows, say from spotty tree leaves.
Perhaps genetic algorithms guided by NN guesses will be needed to construct such models for evaluations. It would need 3D sub-models of everyday objects to compare against.
Table-ized A.I.
Please stop normalizing click bait style headlines. It's not a homage. It's not cute. It's not funny.
This comment is covered by the Popeye standard disclaimer.
So behavioral psychologists are only interested in behavior. So you create an AI that is responding predictably and consistently after simulation and training -- even to novelty. Is it absolutely necessary to know what is going on in the black box? Especially if the device outperforms a human driver. I agree that it is unsettling not to know. Since we have no good theory of mind it is actually unsurprising that when we create a device that seems to have one we don't know exactly what is going on. I think it is pretty cool, actually.
Currently in Ulaanbaatar, which has some of the most aggressive and undisciplined driving I have ever seen. I would love to see the AI that could field these dudes and dudettes. Combat ready!
You know a robo hybrid is not that far off. Volvo is testing 100 cars in Gothenburg as I write. Their idea is to have the car drive when it is boring and the driver take over when he or she wishes or when high skill is needed. Works for me.
"No fear. No envy. No meanness." Liam Clancy
There's the problem: if you have a trained AI and not some sort of expert system based on a collection of human knowledge it's nearly impossible to say how it will handle the unexpected near-garbage input.
It's not a problem. Whoever wrote that piece for MTR shouldn't be in the science-writing business.
We deal with analysis-intractable systems all the time. They're the vast majority of the systems we deal with, in engineering and in everyday life. Most of the physical systems of the car aren't tractable to analysis. Weather isn't. Human health isn't. The electrical grid isn't. Animal behavior isn't.
We have plenty of computer-controlled industrial systems that have intractable control mechanisms. Some are famous - the Sendai train system with its fuzzy-logic control system, for example. Large fuzzy-logic systems are very similar to deep convolutional ANNs in this regard - they coalesce weighted inputs with a nonlinear rectifying function. (I think the Sendai train system uses MAX rather than functions like ReLU or tanh, which are common in ANNs, but it's the same principle.)
So we do what we do with other intractable systems: simulation, empirical study, and statistical analysis to derive a statistical-mechanical model. The Sendai train system underwent years of simulated testing, with hundreds of thousands of simulations (according to an interview with Zadeh in DDJ some years back). Claiming that we need an exact, compact model of the system in order to declare it fit for purpose is naive, in terms of both engineering and history.