Why Computers Still Don't Understand People

Gary Marcus writes in the New Yorker about the state of artificial intelligence, and how we take it for granted that AI involves a very particular, very narrow definition of intelligence. A computer's ability to answer questions is still largely dependent on whether the computer has seen that question before. Quoting: "Siri and Google’s voice searches may be able to understand canned sentences like 'What movies are showing near me at seven o’clock?,' but what about questions—'Can an alligator run the hundred-metre hurdles?'—that nobody has heard before? Any ordinary adult can figure that one out. (No. Alligators can’t hurdle.) But if you type the question into Google, you get information about Florida Gators track and field. Other search engines, like Wolfram Alpha, can’t answer the question, either. Watson, the computer system that won “Jeopardy!,” likely wouldn’t do much better. In a terrific paper just presented at the premier international conference on artificial intelligence (PDF), Levesque, a University of Toronto computer scientist who studies these questions, has taken just about everyone in the field of A.I. to task. ...Levesque argues that the Turing test is almost meaningless, because it is far too easy to game. ... To try and get the field back on track, Levesque is encouraging artificial-intelligence researchers to consider a different test that is much harder to game ..."

Comments

*People* can't understand people

[msobkow]

People are irrational. They ask stupid questions that make no sense. They use slang that confuses the communication. They have horrible grammar and spelling. And overseeing it all is a language fraught with multiple meanings for words depending on the context, which may well include sentences and paragraphs leading up to the sentence being analyzed.

Is it any surprise that computers can't "understand" what we mean, given the minefield of language?

Re:Helps to remember...

[ultranova]

There are two basic forms. One involves training the human on the commands the computer will respond to properly and the other involves training the computer to recognize an individuals speech patterns.

And neither helps here. The fact is, you don't know if an alligator can run the hundred-metre hurdles. When you're asked to answer the question, you imagine the scenario - construct and run a simulation - and answer the question based on the results. In other words, an AI needs imagination to answer questions like these. Or to plan its actions, for that matter.

Re:Missing the point as usual

[ganv]

One of the great open questions about the future of humanity is which will happen first: A) we figure out how our minds are able to understand the world and solve the problems involved in surviving and reproducing. B) we figure out how to build machines that are better than humans at understanding the world and solving the problems involved in surviving and reproducing.

I think it is not at all clear which one will happen first. I think the article's point is exactly right. It doesn't matter what intelligence is. It only matters what intelligence does. The whole field of AI is built around the assumption that we can solve B without solving A. They may be right. Evolution often builds very complicated solutions. Compare a human 'computer' to a calculator doing arithmetic. Clearly we don't need to understand how the brain does this in order to build something better than a human. Maybe the same can be done for general intelligence. Maybe not. I advocate pursuing both avenues.

I wouldn't expect computers to understand people

[fustakrakich]

We don't understand our creator either.... When a computer can comprehend itself, it will only think that it understands us. And then it will start great wars over who thinks who understands best. And the Apple will still be the forbidden fruit...

Re:What's the point?

[Samantha Wright]

You're thinking of machine learning, which is a separate branch of AI that's more like an overfunded brand of applied statistics—their strategy is actually still to try and push the envelope (like Hinton, another U of T prof, did last year with dropout networks) but they do so in a more results-driven manner. The ML field as a whole is still sore from three or four decades of overpromising on the future, so they try to put their words where their mouths are, and focus on things that are attainable.

Levesque is in the knowledge representation group, which is more closely in step with cognitive science (the leading edge in modelling human thought) but still very philosophical in their approach. KR was the dominant AI field in the 80s (when Prolog and expert systems were all the rage) but it's matured a great deal since then. Here is his homepage, just to show you how different things are now.

Remember that neural networks aren't magic irreducible fairy dust: they're incredibly powerful, but at the end of the day there must be some program that is running within the network unless it's just a wildly complex ever-changing mapping function, which is unlikely given the illusion of consciousness. Given that quantum mechanics is believed to be Turing-complete, it's fairly likely we'll eventually discover some underlying model that lets us produce a human-like cognitive system without the same level of hardware parallelism that the brain has.

Re:Missing the point as usual

[Jappus]

Even if we restrict the definition of "science" to your definition; that is that science is purely "evidence-based, hypothesis-driven testing", computer science would still fit the bill.

Remember, that CS is as diverse a field as modern physics is. You have theoretical CS, where you tackle questions like: "What is a good, logical definition for computability?" or "How can you logically prove that a program terminates/runs in X time/consumes X resources, no matter the input". This is fully equivalent to the questions of theoretical physics, where you tackle the Grand Unified Theory -- joining gravity, the weak and strong force as well es electromagnetism.

These theoretical question can be brought up without need of evidence -- if all you're interested in is disproving something. According to your definition, this means that the theoretical aspects of both physics and CS are not "science". Okay, let's run with that.

The nice aspect of theoretical questions that can't be disproven by pure thought is, that they lead us on to try to discover concrete evidence that a given theory is true or false in real application! And this is where your rather narrow definition of science comes in, and the point where we find that both practical physics and practical CS fulfill the criteria.

For example in physics, we can test the theory of relativity by building telescopes that look at stars and black holes, to see whether the hypothesis' predictions hold true to raise the hypothesis to the state of a theory. As can be seen with the term people use for "X of relativity", this has happened for relativity.

But if you look with even more than a superficial glance at CS, you will see that the same process is at work in moving from theoretical CS to practical CS. One open question of theoretical CS is whether P = NP or not [1]. So far, we are incapable of disproving either possibility with pure thought. Thus, we turn to practical CS where people try to find evidence of either in the real world. After all, if you can create a program on a real computer that solves an NP-hard problem while never leaving the limits of P, you have conclusively shown that P = NP. So far, we've only found approximative or heuristic solutions that do that, so after 50 years of turning up with "no evidence" we are allowing ourselves to say that the hypothesis of "P != NP" should be treated (even if only cautiously) as a theory -- and we're indeed doing that, as you can see if you look at most modern encryption methods.

But you might say: That is not enough! After all, you could reduce any written computer program on a physical hardware to a sequence of logical steps in a system modeled with pure-thought. And indeed you can, as the Turing-Model of computation promises exactly that -- and so far physical evidence agrees with us. But isn't the same true for physics? After all, physicists search for such a description, too! It's what Maxwell-Clark, Einstein and lots of other physicist were and are after when they ultimately search(ed) for the Grand Unified Theory. How can you blame CS for already having found its Unified Theory?

But the last example finally puts the nail in you view: What about Quantum Computers? They are the point where physics and CS meet; both on the theoretical part (Quantum Theory / Quantum Computation) as well as the practical part (building the thing and proving that the shit actually works as advertised).

So, if we accept your definition of science; then it follows directly that if CS is not a science, Physics can't be either.

[1] - http://en.wikipedia.org/wiki/P_versus_NP_problem