How I Failed the Turing Test

chrisjrn writes "I stubled across this article today, detailing a man's experiences of being added to AIM Screen Name lists - one full of "celebrities" and the other full of "Sex Bots" (he was, of course, neither of these). Raises a few questions as to how easy it is to get a hold of your screenname, and also of the effectiveness of the Turing Test for AI, in the online world. Or is it just that people aren't bothered trying to tell the humans apart anymore?" Also, it's funny. Don't try to read anything deep into it.

Comments

Another AI test

[Anonymous Coward]

I'll believe in AI when a robot can tie shoelaces. Mimicking conversation is nice and well, but as far as robotics goes, we've yet to see anything remotely resembling artificial intelligence in action.

How he got listed as a sex bot...

[dotgod]

Google for sex bots and look at the first link. It's an article that he wrote, and his screen name is in it.

Skype Prank

[Noksagt]

There is another link going around about an intentional Skype prank:

A profile is put up with a girl's name and picture, and put in "Skype me" mode. Within minutes some seedy guy will invariably try calling/chatting, and there's a little program I made running the whole time which will partner up people 2 at a time, and send messages from the first person to the second, & vice versa. This way both people think they're talking to a girl, when they find out, well, they're not normally too happy about it... It'll also accept and receive all files sent, and if someone tries to call, it'll accept the call with an answerphone message and log what the person says.

Cute

[NoTheory]

that really is a clever passage.

What people should remember is that the turing test requires that the inquistor is competent. If the inquisitor is not (i.e. random AIM idiots), then the test isn't vaild, cause these people can't tell intelligences apart anyway. Also, the inquisitor is supposed to convince themselves via sufficient interaction w/ the system being tested. AIM chats, particularly short one-off dialogues probably aren't a good staging ground for the turing test.

Also, a lot of naive people don't know the capabilities (and limitations) of Artificial Intelligence, so sadly, i'm not surprised at this guy's - or should i say robot's - results.

The role of the Interviewer

[j.leidner]

A factor not often talked about when discussing the Turing test is the qualification of the interviewer (not the subject).

The value of the Turing test depends a lot on the nature of the questions asked. Anybody can ask difficult questions that fellow humans fail to copy with, but not everybody knows what are difficult questions for computers (which may well be simple for humans). Thus, an Artificial Intelligence researcher should be a more suitable interviewer than a non-expert.

Good Example:
requesting the description of an emotion (recalling an event that typically invokes strong emotions)

Bad Example:
Hard math (34589759847359874389574398+3487928479823749837498 )

Then stop calling it "AI"

[squarooticus]

Stop calling it "artificial intelligence." Call it what it is: heuristics research. Oh, I guess that sounds a lot less impressive, huh? Might not be able to get those open-ended grants anymore?

FWIW, I spent two years at LCS, so I have a reasonable idea of what went on in the AI Lab when I was there. There was very little in the way of research into computer-emulating-human intelligence, which is probably a good thing (read: less of a waste of money) considering how little progress the Minsky crowd has made in the past thirty years.

There *are* people who talk to bots

[ytm]

Some time ago I coupled Perl Eliza module with IM account registered as "Irene17". That module works only for English and my IM network was for non-English users so Irene would welcome anyone with message that she understands English only. I set her status to available for conversation and left it running for a week. That IM network has central directory of users so I was sure that sooner or later someone would find her.

Then I looked at logs of conversations. It turned out that there were people who actually talked with her for quite a while, struggling with English. The scheme was more or less the same. First some usual phrases to start a conversation, then trying to get some information about her and finally realizing that she is unwilling to tell anything about herself :) frustration with "Good bye" or some insults.

So, in a way, she has passed a Turing test, but the knowledge of English was poor on both sides.

Cybernetic Poet and the Turing test

[xPsi]

The article, although basically a joke, says something interesting about:
1) people's (AIMers) lower standards for conversation;
2) and also their open mindedness towards what a computer is capable of producing.

I guess the first point is negative and the second positive. The combination leaves a situation where a computer doesn't have to generate anything sophisticated to be tagged as human.

I once administered an informal Turing test using Ray Kurzweil's Cybernetic Poet. I presented to 6 friends several dozen poems, some of which were computer generated (the poems, not the friends...).

People who were computer savvy tended to overestimate what a computer was capable of doing and did rather poorly. Similarly, people who were artistic but not very techie tended to have a very open mind regarding what constituted human poetry (bad grammar, non sequiturs, etc. were ok in an e.e. cummings sort of way) and also did poorly.

The people who did consistently well were those who were neither computer types nor artists, but rather "pure" academics (language specialists, classicists, etc.). They simply used grammar and puncutation as their guide.