Loebner Talks AI

Mighty Squirrel writes "This is a fascinating interivew with Hugh Loebner, the academic who has arguably done more to promote the development of artifical intelligence than anyone else. He founded the Loebner prize in 1990 to promote the development of artificial intelligence by asking developers to create a machine which passes the Turing Test — meaning it responds in a way indistinguishable from a human. The latest running of the contest is this weekend and this article shows what an interesting and colourful character Loebner is."

Comments

Don't forget Steve Furbur

[jd]

He is the genius who brought the UK the BBC Micro, and is now studying the relationship between AI and biological neurons. His comments on the BBC website make very interesting reading regarding the problems facing AI and computer intelligence.

Re:Don't forget Steve Furbur

[Anonymous Coward]

Intelligence is not an arbitrary point on a line, there are varying degrees.

Re:Don't forget Steve Furbur

[jd]

BBC Basic always initialized variables on first use, so you're ok.

This is news for nerds?

[malajerry]

Hardly a fascinating interview, more like 4 paragraphs and a soundbite or two, if you haven't read TFA, don't bother.

Re:This is news for nerds?

[Anonymous Coward]

if you haven't read TFA

Way ahead of you!

Re:This is news for nerds?

[Mr Thinly Sliced]

I'm just here for the comments!

I'm not even here!

Arguably?

[mangu]

the academic who has arguably done more to promote the development of artificial intelligence than anyone else

Well, I suppose someone could argue that. But it would be a pretty weak argument. I could cite at least a hundred researchers who are better known and have done more important contributions. to the field of AI.

Re:Arguably?

[sketerpot]

The purpose of the Turing test was to make a point: if an artificial intelligence is indistinguishable from a natural intelligence, then why should one be treated differently from the other? It's an argument against biological chauvinism.

What Loebner has done is promote a theatrical parody of this concept: have people chat with chatterbots or other people and try to tell the difference. By far the easiest way to score well in the Loebner prize contest is to fake it. Have a vast repertoire of canned lines and try to figure out when to use them. Maybe throw in some fancy sentence parsing, maybe some slightly fancier stuff. That'll get quick results, but it has fundamental limitations. For example, how would it handle anything requiring computer vision? Or spatial reasoning? Or learning about fields that it was not specifically designed for?

It sometimes seems that the hardest part of AI is the things that our nervous systems do automatically, like image recognition, controlling our limbs, and auditory processing. It's a pity the Loebner prize overlooks all that stuff in favor of a cheap flashy spectacle.

Re:Arguably?

[Workaphobia]

Amen. The whole point of the Turing Test was to express a functionalist viewpoint of the world, that two blackboxes with the same interface are morally and philosophically substitutable. And this whole media-fueled notion of the Turing Test as a milestone on the road to machine supremacy just muddles the point.

Sorry, Loebner Has Done Nothing for AI

[Louis Savain]

The Turing test has nothing to do with AI, sorry. It's just a test for programs that can put text strings together in order to fool people into believing they're intelligent. My dog is highly intelligent and yet it would fail this test. The Turing test is an ancient relic of the symbolic age of AI research history. And, as we all should know by now, after half a century of hype, symbolic AI has proved to be an absolute failure.

Re:Sorry, Loebner Has Done Nothing for AI

[Tx]

Just because the best-scoring programs to date on the Turing test are crap does not necessarily mean the test itself is not useful. Yes, it tests for one particular form of AI, but that form would be extremely useful to have if achieved. You may consider your dog highly intelligent, but I'm not likely to want to call it up and ask for advice on any given issue, am I?

Re:Sorry, Loebner Has Done Nothing for AI

[bencoder]

Well that's really the point of the test. Any "AI" that simply manipulates text as symbols is going to fail the turing test. To make one that can pass the test, imho, would probably require years of training it to speak, like one would with a child. It also requires solving all the associated problems of reference - how can a deaf, blind and anesthesic child truly get a sense of what something is, so much so that they can talk about it(or type about it, assuming they have some kind of direct computer hook up which allows them to read and write text).

Basically, nothing's going to pass the turing test until we have actual AI. Which is the whole point of the test!

I study AI at Reading by the way so I'll be going along to the event tomorrow morning :)

Re:Sorry, Loebner Has Done Nothing for AI

[jbsooter]

I always imagine the first, and last, computer that will pass the Turing Test will be the one explaining to us that it has taken over the world because we aren't intelligent enough to run it ourselves. :P It will end the conversation with "You really didn't see this coming? What a bunch of idiots."

Re:Sorry, Loebner Has Done Nothing for AI

[Louis Savain]

Intelligence, artificial or otherwise, is what psychologists define it to be. It has to do with things like classical and operant conditioning, learning, pattern recognition, anticipation, short and long term memory retention, associative memory, aversive and appetitive behavior, adaptation, etc. This is the reason that the Turing test and symbolic AI have nothing to do with intelligence: they are not conserned with any of those things.

That being said, I doubt that anything interesting or useful can be learned from writing those Loebner/Turing programs. It would be much more interesting to write programs that learn to play a good game of GO. I suggest that Loebner change his competition accordingly.

Re:Sorry, Loebner Has Done Nothing for AI

[geomobile]

Isn't just that the point of the Turing test: if you can fool people into believing you're intelligent, then you are intelligent? No way to tell if something is intelligent apart from its behavior. Lab conditions using language (string manipulation) were probably chosen because the amount of context and the variety of problems that could be encountered during the test are only solvable by humans (until now), and only using that which we call intelligence.

I know this was true for playing chess at a decent level twenty years ago. Ok, probably we'll never accept the intelligence of anything the inner workings of which we understand completely.

But that is not a problem of the test, it is a problem of our willingness to define intelligence. So, no proof of artificial intelligence possible for us, ever. News at eleven.

At least the dog argument doesn't hold. The Turing test does not claim to be able to define all types of intelligence. Also: future dog-driven string manipulation still possible.

Re:Sorry, Loebner Has Done Nothing for AI

[jd]

The Turing Test, as classically described in books, is not that useful, but the Turing Test, as imagined by Turing, is extremely useful. The idea of the test is that even when you can't measure or define something, you can usually compare it with a known quantity and see if they look similar enough. It's no different from the proof of Fermat's Last Theorum that compared two types of infinity because you couldn't compare the sets directly.

The notion of the Turing Test being simple string manipulation dates back to using Elisa as an early example of sentence parsing in AI classes. Really, the Turing Test is rather more sophisticated. It requires that the machine be indistinguishable from a person, when you black-box test them both. In principle, humans can perform experiments (physical and thought), show lateral thinking, demonstrate imagination and artistic creativity, and so on. The Turing Test does not constrain the judges from testing such stuff, and indeed requires it as these are all facets of what defines intelligence and distinguishes it from mere string manipulation.

If a computer cannot demonstrate modeling the world internally in an analytical, predictive -and- speculative manner, I would regard it as failing the Turing Test. Whether all humans would pass such a test is another matter. I would argue Creationists don't exhibit intelligence, so should be excluded from such an analysis.

Re:Sorry, Loebner Has Done Nothing for AI

[Your.Master]

Wait...who made psychologists the masters of the term "intelligence" and all derivations thereof?

No, frankly, they can't have that term. And you can't decide what is interesting and is uninteresting.

If I revealed to you right now that I'm a machine writing this response, that would not interest you at all? I'm not a machine. But the point of the Turing test is that I could, in fact, be any Turing-test beating machine rather than a human. Sure, it's a damn Chinese room. But it's still good for talking to.

Whether or not your dog has intelligence has nothing to do with this, because AI is not robot dog manufacturing.

Re:Sorry, Loebner Has Done Nothing for AI

[Your.Master]

But you did see it coming. And it's on the Internet, which such a machine would have much easier access to and could search much more instantaneously than we. Which means its failure to notice this prediction is a sign of laziness and/or intellectual defect.

Which gives the human race hope.

Re:Sorry, Loebner Has Done Nothing for AI

[ShakaUVM]

>>Intelligence, artificial or otherwise, is what psychologists define it to be

No, it's not. Or it least it shouldn't be, given how psychology is a soft science, where you can basically write any thesis you want (rage is caused by turning off video games!), give it a window dressing of stats (you know, to try to borrow some of the credibility of real sciences), and then send it off to be published.

Given how much blatantly wrong stuff is found in psychology, like Skinner's claim that a person can only be intelligent when reacting to outside stimulus (really? I can't think while I'm by myself?), I'd try and steer far away from giving psychologists the ability to define what intelligence is.

Re:Sorry, Loebner Has Done Nothing for AI

[retchdog]

It would be interesting, except that any reasonable person would conclude that either 1) you were lying; or 2) you (the machine) were following a ruleset which would break after just a few more posts.

AI would be better off focusing on dogs. It's actually better off focusing on practical energy-minimization and heuristic search methods, which would be comparable in intelligence to say an amoeba. Going for human-level intelligence right now, is like getting started in math by proving the Riemann hypothesis.

Re:Sorry, Loebner Has Done Nothing for AI

[thrillseeker]

if you can fool people into believing you're intelligent, then you are intelligent?

as my ethics teacher said, "Sincerity is the most important thing ... once you can fake that, you've got it made."

Re:Sorry, Loebner Has Done Nothing for AI

[grumbel]

Yes, it tests for one particular form of AI, but that form would be extremely useful to have if achieved.

The problem I have with the test is that it isn't about creating an AI, but about creating something that behaves like human. Most AI, even if highly intelligent, will never behave anything like a human, simply because its something vastly different and build for very different tasks. Now one could of course try to teach that AI to fool a human, but then its simply a game of how good it can cheat, not something that tells you much about its intelligence.

I prefer things like the DARPA Grand Challenge where the goal isn't to create something that behaves like a human, but simply something that gets the given task done. That way you can slowly raise the bar instead of setting a goal where you don't even know if there is a point in chasing it. Turning test feels to much like a challenge to fly by sticking wings to your arms, you might be able to do that one day, but the aviation industry doesn't really care, jumbo jets are flying fine even without flapping their wings.

Re:Sorry, Loebner Has Done Nothing for AI

[ceoyoyo]

You're confusing the Turing test with one class of attempts to pass it. In fact, the test has proven remarkably good at failing that sort of program.

Yes, your dog would fail the Turing test, because the Turing test is designed to test for human level intelligence.

Re:Sorry, Loebner Has Done Nothing for AI

[jd]

Intelligence goes way beyond those limited parameters, which is why no psychologist or AI expert would claim to know what intelligence actually, fundamentally, is. Sure, it includes all of those, but there are many examples of intelligence which don't fit any of those categories, and many examples of non-intelligence which do.

Re:Sorry, Loebner Has Done Nothing for AI

[cerberusss]

Most AI, even if highly intelligent, will never behave anything like a human, simply because its something vastly different and build for very different tasks.

Yep. Like hunting down and destroying said human.

Re:Sorry, Loebner Has Done Nothing for AI

[bencoder]

That's true, but it doesnt deny that the Turing test is capable of detecting a real AI over one that just "fakes" it. The main problem here is that when people are looking for "intelligence" they don't really know what they are looking for. Turing test offers one solution to this problem(can it talk like a human). It is certainly not the be-all and end-all of intelligence/thought tests and I don't think anyone ever stated it was.

Unfortunatly, no-one can come up with a suitable, testable, definition of intelligence. Turing's answer is to say that the problem is meaningless because there can be no definition of such an abstract concept, so he devised the test as a way of showing that, in a very behaviourist manner, it doesnt matter. If it can act intelligent according to a human without bias(hence the seperation between human and machine) then there's nothing that can be said to deny that the machine is intelligent. No doubt people will claim it's not, but those people will always have bias against a machine.

Re:Sorry, Loebner Has Done Nothing for AI

[ClassMyAss]

Jeff Hawkins thinks that intelligence is not defined by behavior but by the accuracy of the predictions it makes about the world

Careful - Hawkins doesn't just think the predictions about the world are important, he thinks that the real magic comes when the system tries to predict its own behavior. Without that self-referential prediction, the essential non-linearity that intelligence and perception requires is not present.

Whether this is enough is another matter...

Let's forget the turing test

[Yvan256]

and use the Voight-Kampff test instead.

Re:Let's forget the turing test

[Anonymous Coward]

Is this testing whether I'm intelligent or a lesbian?

Re:Let's forget the turing test

[deodiaus2]

Have it decipher Republican campaign slogans!! I am sure every computer will crash trying to extract information from nonsense.

Real AI is still a long way out..

[thrillbert]

Unless someone can figure out how to make a program want something.

If you take the lower life forms into consideration, you can teach a dog to sit, lay down and roll over.. what do they want? Positive encouragement, a rub on the belly or even a treat.

But how do you teach a program to want something?

Word of caution though, don't make the mistake made in so many movies.. don't teach the program to want more information.. ;)

Re:Real AI is still a long way out..

[thrillbert]

Teaching it to want something is not a detriment, because then it can be taught right from wrong.

Take today's youth for example, most parents today allow their kids to do whatever, no reprimand. What are they being taught? That they can do whatever they want and there are no consequences. Why not take the basics of good and bad and teach those to a machine?

A waste of time.

[Anonymous Coward]

The Loebner Prize is a farce. Read all about it: http://dir.salon.com/story/tech/feature/2003/02/26/loebner_part_one/index.html

Great Book on AI

[moore.dustin]

Check out this great book by Jeff Hawkins, creator of the Palm, called On Intelligence. His work is about understanding how the brain really works so that you can make truly intelligent machines. Fascinating stuff and firmly based in the facts of reality, which is refreshing to say the least.

Re:Great Book on AI

[QuantumG]

He doesn't actually say anything in that book.

Re:Great Book on AI

[moore.dustin]

Like I said, "His work is about understanding how the brain really works so that you can make truly intelligent machines." Your notion is dismissable as we could not fathom what we have no clue about, that being your alien. The best we can do on earth is to look at how the most intelligent species we know, humans, actually develop and groom their intelligence.

Intelligence defined is one thing, intelligence understood biologically is something else altogether.

Too much information

[Anonymous Coward]

Until it asks to see my computer with its door off and showing its top end bits the its not acting human. Seriously if I jerk off to this hardware then a computer should!

That's a little bit too much personal information there sport. You don't have to post to slashdot every thought that pops into your head you know.

Re: "Just Ask"

[TaoPhoenix]

Really now, I wish I had a team partner, because these guys need to take a page from the chess world and buff up their Anti-Trick-Question tactics. Those questions always revolve around rapid context switching that would frankly irritate if not confuse a person as well, such as one speaking a second language. (There's a test for you! Which is the computer and which is the guy speaking his ruined French he learned 20 years ago?)

(Typical Tester fake question) "Is the Queen larger than a breadbox?"
Program: "What kind of question is that?"
Tester: "Answer the question"
Program: "Since you failed to define "Queen" on purpose, you created a question that is simultaneously true and false, and therefore a null question. I can only assume this is some cheap ass attempt to authenticate before you waste your remaining 7 minutes chatting with the human should you be so lucky, so I quit here and now. Ask your judge what to do if your software oppponent is programmed to sulk."

Don't take this wrong

[localman]

I'm fascinated by AI and our attempts to understand the workings of the mind. But these days, whenever I think about it, I end up feeling that a much more fundamental problem would be to figure out how to make use of the human minds we've already got that are going to waste. Some two hundred minds are born every minute -- each one a piece of raw computing power that puts our best technology to shame. Yet we haven't really figured out how to teach many of them properly, or how to get the most benefit out of them for themselves and for society.

If we create Artificial Intelligence, what would we even do with it? We've hardly figured out how to use Natural Intelligence :/

I don't mean to imply some kind of dilemma, AI research should of course go on. I'm just more fascinated with the idea of getting all this hardware we already have put to good use. Seems there's very little advancement going on in that field. It would certainly end up applying to AI anyways, when that time comes.

Cheers.