Variations On the Classic Turing Test

holy_calamity writes "New Scientist reports on the different flavors of Turing Test being used by AI researchers to judge the human-ness of their creations. While some strive only to meet the 'appearance Turing Test' and build or animate characters that look human, others are investigating how robots can be made to elicit the same brain activity in a person as interacting with a human would."

Comments

Dammit, I took one and failed!

[Anonymous Coward]

Arrgg... I just took the test and failed. Does this mean that I'm ready to run Linux, and when I die I'll be running FreeBSD?

Re:Dammit, I took one and failed!

[RDW]

Eliza: Can you elaborate on that?

Syntax

[mfh]

FORMALISTS' MOTTO: Take care of the syntax and the semantics will take care of itself.

Also, if you are animating a dude, he is thinking about sex. If you are animating anyone else, they are thinking about shopping.

Technically AI is not hard, you just need to lower your mind-mechanics bar and focus on trailer parks, and folk psychology.

Abstracting cognitive response is far off

[postbigbang]

The Turing test is for apparent 'human' intelligence, where robotics adds communications via 'expressiveness'. These are two different vectors: rote intelligence and capacity to communicate (via body language, and the rest of linguistics/heuristics).

The article doesn't abstract the basic cognitive capacity because it entangles it with the communications medium. The Turing Test ought to be done in a confessional, where you don't get to see the device taking the test. It would also provide a feedback loop on the test as well.

Re:Abstracting cognitive response is far off

[vertinox]

The Turing test is for apparent 'human' intelligence, where robotics adds communications via 'expressiveness'. These are two different vectors: rote intelligence and capacity to communicate (via body language, and the rest of linguistics/heuristics).

I don't think the body language is the hard part and that important considering the majority of human communication these days either involves just test or voice without seeing the other person. (That and certain persons can't interpret body language anyways)

The key problem with AI is:

Context
Context
Context

The number one failure that most Turing programs is that they only respond to the sentence you just said without any context to the conversation before hand. A really good AI would be able to keep on topic and understand what has been discussed previously so that they can expand on the topic without simply just responding to the current line.

There are several ways to achieve this, but right now I don't think there is any program out there that at least I know of that does this right. The easiest way to tell if you are talking to a chat bot is to refer to something previous in the conversation and see if they respond appropriately.

Re:Abstracting cognitive response is far off

[postbigbang]

Personality and other heuristics are bound to occur. The tests, however, aren't really based on whether someone can play like Billy Joel or Chopin.

Turing was very aware of asking the right questions to get the right answers of a cognitive, self-aware entity. How that entity is abstracted as a physical entity is the mistake of the article. I can't play piano-- do I fail the test? Through what disciplines do we decide that there are cognitive components that establish a baseline of sentience and intelligence? Some of these, Turing tests for. Others are excellence (or not) in communications transports. That's my truck with the article.

Re:Abstracting cognitive response is far off

[jonaskoelker]

The easiest way to tell if you are talking to a chat bot

Reaction time is a factor in this, so please pay attention.

You're in a desert walking along in the sand when all of the sudden you look down, and you see a tortoise, it's crawling toward you. You reach down, you flip the tortoise over on its back. The tortoise lays on its back, its belly baking in the hot sun, beating its legs trying to turn itself over, but it can't, not without your help. But you're not helping. Why is that?

Re:Abstracting cognitive response is far off

[moore.dustin]

You are only partially correct. The focus on context is misplaced, though you are on the right path. Simply remembering words or topics that have been mentioned earlier in the same discussion does not say anything about intelligence.

The main problem with AI is learning.

Nearly all work in the field now has a misplaced or completely wrong approach to achieving real AI. In order to understand how to make truly intelligent machines, we must first know how our own brains work. Most focus is on creating a machine that can perform in some very specific situation, like the Turing Test. However, these machines are not intelligent, they do not learn. They are not creating, storing and recalling patterns which are the crux of our cognitive abilities.

The first step to true AI is understanding how human intelligence is achieved in our brain.

Re:Abstracting cognitive response is far off

[w0mprat]

"A really good AI would be able to keep on topic and understand what has been discussed previously"

Such an AI posting to slashdot would quickly be revealed.

Re:Slashdot please help me!

[morgan_greywolf]

I dunno, but I get the feeling that Solaris 10 must somehow be involved.

We use turing tests on new hires at my job

[jollyreaper]

We do interviews via IM and if the interviewee cannot convince two out of three of the interviewers they are not a bot, they don't make it to the second round.

Re:I think the real test

[fuzzyfuzzyfungus]

http://en.wikipedia.org/wiki/Voight-Kampff_machine

Wonderful!

[FatalTourist]

The wife and I were looking for ways to spice up the ol' Turing Test.

Re:I think the real test

[PJ1216]

http://xkcd.com/329/

Chatbots

[Demiah]

The best chatbots I've come across are at www.a-i.com
Not quite good enough to pass the turing test yet, but some are quite witty.

Human-ness != Intelligence

[Baldrson]

Why are people so interested in mimicing humans? Isn't intelligence far more interesting than human-ness?

I understand people's fear of machine intelligence exceeding that of humans, but it is actually more dangerous to have machines merely mimicing human-ness than to have machines that are intelligent enough to actually understand what we say better than another human could.

That means more than merely having some mockery of mirror neurons for "empathy". It means genuine understanding: The ability to model.

The reason this is central to our relationship to our machines should be obvious: Friendly AI really boils down to the problem of effectively communicating our value systems to the AIs.

That's why natural language comprehension is the first step to friendly AI.

HENCE:

Insensitive Clod!

[Unique2]

While some strive only to meet the 'appearance Turing Test'

I don't come here to be insulted, you insensitive clod!

Re:Sweet

[vertinox]

Don't get me wrong, I have a great respect for these scientists, I just wonder how these sorts of real robots will fare on the market.

I think the idea is that robots will be used to do things that humans aren't willing to put up with.

Which means if you can't find someone to put up with you, then maybe a robot is for you.

Re:Same test?

[Kozz]

Turn in your geek badge. It's Voight-Kampff: http://en.wikipedia.org/wiki/Voight-Kampff

Re:Turing test == unhelpful target

[FTWinston]

The turing test always struck me as ridiculously anthropomorphic. Clearly the existance of non-humanlike intelligence can be envisaged. But no matter how smart, it would fail this test.

Furthermore, in an in-depth conversation, surely an AI would have to lie (talk about its family, its working life, etc)...
If we continue to enshrine the standard of the Turing test, we're aiming for a generation of inherently untruthful fake-people machines. If it 'knows' that many/most things it tells us are lies, it may well have to assume the same for us. At this point, I suspect its time to drop in a skynet reference or two.

Lastly, its worth pointing out that for a 2 minute conversation, a randomly selected response of "lol" "haha" and "rofl" would match, if not out-score many people on the Turing test.

Turing Test won with Artificial Stupidity

[David Gerard]

Artificial intelligence came a step closer this weekend when a computer came within five percent of passing the Turing Test, which the computer passes if people cannot tell between the computer and a human.

The winning conversation was with competitor LOLBOT:

"Good morning."
"STFU N00B"
"Er, what?"
"U R SO GAY LOLOLOLOL"
"Do you talk like this to everyone?" "NO U"
"Sod this, I'm off for a pint."
"IT'S OVER 9000!!"
"..."
"Fag."

The human tester said he couldn't believe a computer could be so mind-numbingly stupid.

LOLBOT has since been released into the wild to post random abuse, hentai manga and titty shots to 4chan, after having been banned from YouTube for commenting in a perspicacious and on-topic manner.

LOLBOT was also preemptively banned from editing Wikipedia. "We don't consider this sort of thing a suitable use of the encyclopedia," sniffed administrator WikiFiddler451, who said it had nothing to do with his having been one of the human test subjects picked as a computer.

"This is a marvellous achievement, and shows great progress toward goals I've worked for all my life," said Professor Kevin Warwick of the University of Reading, confirming his status as a system failing the Turing test.