Slashdot Mirror
How I Failed the Turing Test
chrisjrn writes "I stubled across this article today, detailing a man's experiences of being added to AIM Screen Name lists - one full of "celebrities" and the other full of "Sex Bots" (he was, of course, neither of these).
Raises a few questions as to how easy it is to get a hold of your screenname, and also of the effectiveness of the Turing Test for AI, in the online world. Or is it just that people aren't bothered trying to tell the humans apart anymore?" Also, it's funny. Don't try to read anything deep into it.
I'll believe in AI when a robot can tie shoelaces. Mimicking conversation is nice and well, but as far as robotics goes, we've yet to see anything remotely resembling artificial intelligence in action.
Google for sex bots and look at the first link. It's an article that he wrote, and his screen name is in it.
that really is a clever passage.
What people should remember is that the turing test requires that the inquistor is competent. If the inquisitor is not (i.e. random AIM idiots), then the test isn't vaild, cause these people can't tell intelligences apart anyway. Also, the inquisitor is supposed to convince themselves via sufficient interaction w/ the system being tested. AIM chats, particularly short one-off dialogues probably aren't a good staging ground for the turing test.
Also, a lot of naive people don't know the capabilities (and limitations) of Artificial Intelligence, so sadly, i'm not surprised at this guy's - or should i say robot's - results.
There are lives at stake here!
What do you expect on AOL ?
I have the ultimate weapon in AI detection , it's called severe dyslexia .
If I don't spell check and proof read then no bot could hold a conversation with me .
Instant messaging is not a great place to rely on spell checking and proof reading , but it does rely on our minds ability to see past simpel speling/grammer erors (intentional)
The only things certain in war are Propaganda and Death. You can never be sure which is which though
The value of the Turing test depends a lot on the nature of the questions asked. Anybody can ask difficult questions that fellow humans fail to copy with, but not everybody knows what are difficult questions for computers (which may well be simple for humans). Thus, an Artificial Intelligence researcher should be a more suitable interviewer than a non-expert.
Good Example:
requesting the description of an emotion (recalling an event that typically invokes strong emotions)
Bad Example:8 )
Hard math (34589759847359874389574398+348792847982374983749
He is just trying to get his AIM fight score raised.
An internationally recognized test for well load balanced http servers.
Anyway: Philip K Dick in the story Do robots dream of electric sheep, discussess the ways to distinguish androids from humans with some nice tests. They also upgrade the tests all the time. Maybe it is time to upgrade the Turing test too.
My wife's sketchblog Blob[p]: Gastrono-me
Stop calling it "artificial intelligence." Call it what it is: heuristics research. Oh, I guess that sounds a lot less impressive, huh? Might not be able to get those open-ended grants anymore?
FWIW, I spent two years at LCS, so I have a reasonable idea of what went on in the AI Lab when I was there. There was very little in the way of research into computer-emulating-human intelligence, which is probably a good thing (read: less of a waste of money) considering how little progress the Minsky crowd has made in the past thirty years.
[ home ]
Some time ago I coupled Perl Eliza module with IM account registered as "Irene17". That module works only for English and my IM network was for non-English users so Irene would welcome anyone with message that she understands English only. I set her status to available for conversation and left it running for a week. That IM network has central directory of users so I was sure that sooner or later someone would find her.
:) frustration with "Good bye" or some insults.
Then I looked at logs of conversations. It turned out that there were people who actually talked with her for quite a while, struggling with English. The scheme was more or less the same. First some usual phrases to start a conversation, then trying to get some information about her and finally realizing that she is unwilling to tell anything about herself
So, in a way, she has passed a Turing test, but the knowledge of English was poor on both sides.
Basically when we think, we can change our thought rules on the fly.
For example, if someone says to me: "The word dog is the same as a canine" I would learn that, and file it away, and the next time I encounter the word canine, I would be able to associate it with the furry thing with 4 legs whenever.
Until a computer can truly rewrite its own code on the fly, it will not be able to emulate a human brain.
Sure there may be ways of layering scripts in order to give the impression that the computer is changing its code, but the sheer volume, as somebody stated, of the different things we know makes it impractical. If you were to train a computer like a baby, it would take a long time to get it even close to an average human intelligence.
1) people's (AIMers) lower standards for conversation;
2) and also their open mindedness towards what a computer is capable of producing.
I guess the first point is negative and the second positive. The combination leaves a situation where a computer doesn't have to generate anything sophisticated to be tagged as human.
I once administered an informal Turing test using Ray Kurzweil's Cybernetic Poet. I presented to 6 friends several dozen poems, some of which were computer generated (the poems, not the friends...).
People who were computer savvy tended to overestimate what a computer was capable of doing and did rather poorly. Similarly, people who were artistic but not very techie tended to have a very open mind regarding what constituted human poetry (bad grammar, non sequiturs, etc. were ok in an e.e. cummings sort of way) and also did poorly.
The people who did consistently well were those who were neither computer types nor artists, but rather "pure" academics (language specialists, classicists, etc.). They simply used grammar and puncutation as their guide.
i\hbar\dot{\psi}=\hat{H}\psi
First off, anyone who doesn't know how to change the AIM settings to say "Only allow people on my buddy list to add me to their buddy list." shouldn't be allowed to use AIM.
But his experiences are amusing. I would have played with it a bit more. Make the idiot invest that much more time to ultimately find out you're not some commonly known celebrity but the Ruler of the Universe instead.
I recall when the ALICE IRC bot was first getting rolled out. Everyone was so amazed and then I popped the question: "Who is your creator?"
The bot then proceeded to tell me about Dr. so and so and his I.T. lab, etc. This was until the woman who was selling the bots to people discovered that I'd figured out how to get the bot to expose itself so to speak, and they changed the creator response to their own name.
Nice try but I know a bot when I see one.
Another variation: You could try to create a game with some random rules and ask the other side to participate. For instance: "Let's play geography. I'll name a place and then you have to name a place that starts with the second letter of the place I named. I'll start with Jupiter."
For me, the most interesting part of this conversation is that it suggests that we're getting close enough that humans need to work at it in order to sound different from bots. We used to think that the Turing test was something where humans could answer normally but would still be differentiated from programs. Now we're getting to the point where humans (or at least human on AIM) have to make an effort in order to be recognized as different.
-Peter
I've never met a bot worth posing these questions to, but I'd be happy to try things like this:
o If you're using a qwerty keyboard, what letters are between w and t?
o Write "guls" backwards. What is an anagram of "owl"?
o 11 22 33 4* 55 66. What should '*' be?
These questions give the AI a shared reality to work with, unlike the super abstract human psychology (in "why do you like music") or the common 'popular reality', often seen in questions like "what's your favourite music". If an AI could answer these kinds of questions, I might think about calling it intelligent.
I've been posting on the net since 1994 and I still haven't come up with a good sig!
OK; I was dating this girl back in '92 or so. She used to call my BBS and chat with me, which was cool, I suppose. One day, however, I wasn't there, and she got my Eliza-clone chatbot (it would auto answer and play sysop if I didn't answer a chat request within x seconds).
I came home hours later and found the log. She started out all sweet, but slowly became more and more irritated with "my" responses. Finally culminating in questions like, "Don't you love me?" and "Why are you treating me like this?". Of course, the bot gave dumb responses that pissed her off even more, and she broke up with me, right there. Told "me" never to call her again.
And you know what? I didn't. I figured if she was that stupid, I was better off without her.