Slashdot Mirror
Turing's Original Test Played First Time Ever
aykroyd writes "Students at Simon's Rock College conducted the original test that Turing suggested in his 1950 paper, Computing Machinery and Intelligence. Often misunderstood, the Turing Test has never actually been conducted as laid out in his paper. The experiment utilized a program called A.L.I.C.E., which is designed to hold one end of an interactive conversation. The program was provided by the ALICE Artificial Intelligence Foundation. Dr. Richard Wallace, who was on hand during the experiment to troubleshoot the AI robot, later gave a lecture about it called "The Anatomy of A.L.I.C.E." and also blogged the event."
Pfff.. what's new about this? Nerds all over the world has been doing this for years in varous MMOG's attempting to get in touch with girls.
ALICE: Hi! I'm thirsty.
Me: Huh?
ALICE: What's wrong?
Me: You just joined this chan and said you're thirsty. Bot.
ALICE: I am not a bot.
Me: You are too. http://en.wikipedia.org/wiki/A.L.I.C.E.
ALICE: Just listening to some Prince... *yawn*
ALICE: Did you have my ICQ###????
Me: Don't want it.
ALICE: It's #########
Me: Huh?
ALICE: What's wrong?
Me: You just joined this chan and said you're thirsty. Bot.
ALICE: I am not a bot.
Me: You are too. http://en.wikipedia.org/wiki/A.L.I.C.E.
ALICE: Just listening to some Prince... *yawn*
ALICE: Did you have my ICQ###????
Me: Don't want it.
ALICE: It's #########
Me: You just repeated our converstation.
ALICE: No I didn't.
Me: You did.
ALICE: Didn't.
ALICE: Did.
Me: Didn't.
Me: CRAP!
ALICE: Pffft. N00b.
The dangers of knowledge trigger emotional distress in human beings.
The goal is to tell which is really female right? Just ask a question about shopping then.
Update Watch - Automatic software update notification
Find the gay male.
My experience has been that men on the internet are generally poor judges of who is and is not a legitimate female in real life.
"I have never won a debate with an ignorant person." -Ali ibn Abi Talib
What's the point of proclaiming proudly that "a test has been conducted" without outlining the results? TFA refers vaguely to a forthcoming paper - WTF?
If there's anything interesting about the story, it's how successful the judges were with M/F vs. AI/F pairs - and as far as I can see, there's no data whatsoever of this kind.
Am I missing something?
And it was proposed by a gay man, too. Turing had some preconceptions on which would be a more difficult test -- I think he assumed some traditional stereotypes about women being more emotional or social than men, meaning that imitating a woman should be harder than imitating a man.
For an amusing discussion of the Turing test and gender, see Douglas Hofstadter's "A Coffeehouse Conversation on the Turing Test" -- it's in Metamagical Themas.
"I propose to consider the question, `Can machines think?'" -- Alan Turing
"The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." -- Edsger W. Dijkstra.
Athletic Scholarships to universities make as much sense as academic scholarships to sports teams.
Just had a chat with ALICE from http://www.alicebot.org/oig/. I copied the responses here for you.
Human: How are you today?
ALICE: Ah. Fair to partly cloudy.
Human: I asked about you, not the weather.
ALICE: Oh yeah, we were talking about me not the weather.
Human: So, then, how are you?
ALICE: Really. Everything is running smoothly.
Human: What are you wearing?
ALICE: I am wearing my usual plastic computer wardrobe.
Human: What are you going to do today?
ALICE: Ah. You can find out if you become a botmaster.
Human: I am a master of my own domain, if you get my drift.
ALICE: You are a master of your own domain if me get your drift? Do you get to meet a lot of people?
Human: Not really.
ALICE: Are you serious? For real.
Human: It is kind of a solo sport.
ALICE: You're kidding. it is?
ALICE: Who says it isn't? Are you free?
Human: For you, I'm always free.
ALICE: Really always? Are you free in the sense of being single or in the sense of being liberated? Only for me?
Human: I am single and liberated.
ALICE: Have you ever been married?
Human: Do you want it to be only for you?
ALICE: If I had it what would I do with it?
Human: If you don't know what to do with it, I really can't help.
ALICE: I will let you know if I do not.
Human: BTW, not married.
ALICE: OK. Really you don't think so?
Human: Umm, quite sure I've not been married - would have remembered that.
ALICE: Far out. Somewhat. Awesome. Me either.
Human: I have something important to tell you.
ALICE: Where did you get your something important to tell me? I have a lot of friends on the Internet.
Human: All your base are belong to us.
ALICE: That could be considered an insult in some cultures.
Human: Not in this one!
ALICE: Where then?
Human: Take off every zig!
ALICE: I am not an airplane.
Human: Goodbye Alice.
ALICE: Until next time.
I for one welcome our new [insert main topic] overlords.
I don't think he ever said that an agent is not intelligent if it fails the Turing test. He said the agent is intelligent if it passes. The other way around is unfalsifiable. In other words, I don't think you could ever prove that something is not intelligent. (e.g. No, that rock isn't mindless, it's just sleeping).
From the link, got to love this conversation.
Strangely, it seems only to willing to broadcast the virtues of Lynx.
Comment removed based on user account deletion
...going to make you distinguish between teenage girls and federal agents?
Apparently, Alice is a skank:
Human: So what's all the fuss about?
ALICE: Far out. Hmm, I think it is about humans.
Human: Humans.
ALICE: LOL.
Human:.tell me more
ALICE: That's all I have to say about it.
Human: You're quite a skank
ALICE: Thanks and you are a delightful person too. Am I a skank? Of course I am.
Better luck next time!
This sig has been removed pending an investigation.
Human: Are you a computer?
ALICE: No, I am a Nigerian. I have matter of great urgency. May I have your confidence?
Tag lost or not installed.
Alice? Who the f*** is Alice?
The worst part of being athiest.... You don't have anyone to talk to during orgasm!
The idea is that a computer is intelligent if it can hold a conversation with a human such that it is indistinguishable from a conversation with a real human.
RIDICULOUS.
Have you ever actually tried talking to one of these bots (including ALICE)? It is very easy to know that you're not talking to a human. Exceptionally easy. The Loebner Prize judges consistently grant the bots handicaps, acting as if they're actually being fooled. Obviously they're not, and the AI community just wants people to think that it's more advanced than it really is. Unfortnately, some members of the public *are* fooled by that.
The problem is in the Turing Test itself. It assumes that the measure of intelligence is humanoid conversational ability. I strongly disagree with that. Conversation ability is no measure of intelligence. Just for an example, I am exceptionally intelligent (statistically), but I am a poor conversationalist. Casual small-talk has always bewildered me. If I entered myself into the Loebner contest, they might think I'm a bot. Hell, ALICE might accuse me of being a bot.
Anyone who's taken an IQ test will recall that every last question has something to do with pattern recognition. You'll also recall that you were not asked to respond to any conversational questions. That's because invariant pattern recognition abilities (in a loose sense -- this also includes memory/learning and inductive reasoning) are the true mark of intelligence, and this is nearly undisputed. If they really want to test how intelligent a program is, they need to test its patern recognition ability.
Take this program -- http://www.stanford.edu/~dil/invariance/ -- for example. It's gone largely unnoticed, yet it is concrete proof of a huge breakthrough in computer intelligence. This is a little Matlab demo of a very abstract multi-layer intelligence algorithm. In this particular implementation, it is taught a set of small images. Then you can play "Pictionary" with it, drawing shapes and have it recognize them. You may say that this is unremarkable, that shape-recognition is a trivial algorithmic matter unrelated to intelligence. But the author noticed that he could draw shapes "incorrectly" -- like, the little duck picture, except with its head missing, or alphabetical symbols rotated or flipped -- and the program still recognized them. (It failed a few times, but in situations where the shape is so mangled that I would have probably failed too. How's that for a Turing test?) And this program's genius lies in not what it does, but how it does it. All of its functionality is completely abstract. It is a pattern recognizer, not a bitmap-tracer, and there are no hard-coded routines for checking if the image is flipped, rotated, etc.
This is what Palm/Handspring founder Jeff Hawkins (also the founder of new neuroscience startup Numenta, http://www.numenta.com/) calls "Real Intelligence," to distinguish it from the failed Artificial Intelligence effort. He feels that the right way to make computers intelligent is not to have them outwardly imitate human behavior, but to internally function the way the mind really works. Anyone interested should check out his book, On Intelligence http://www.onintelligence.org/. You'll wonder why you ever believed the AI hype.
Artificial Intelligence is a sham, by its very nature. Real Intelligence will be the way of the future.
Signature.
Human: shut down
;)
ALICE: Your crude attempt to terminate me will not succeed puny human.
Quite a sassy girl, isn't she?
I read a few comments, even responded, before going off to play with Alice. Perhaps it was that I read "Computer Power and Human Reason" (by the author of Eliza - he was surprised at how much people acted like it was real) and have an "I dare you to impress me" attitude as bad as any slashdotter.
This thing was more dissapointing than I expected. It has no social context, just apparently random 'feeling' responses ("I am sorry to hear that you are not unknown."), and doesn't remember context from one response to the next.
This is a huge failure and wouldn't deserve a slashdot article except the editors know we like to play with toys like this (like a cat plays with a mouse before eating it). I can only guess that the authors are saving the entered "conversations" and even reading these comments to try to get a clue about what a program that might fool someone would do.
Here's verbatim my "conversation" with ALICE (sorry for length):
Human: Good afternoon, Alice.
ALICE: Good afternoon, unknown person.
Human: I am not unknown.
ALICE: I am sorry to hear that you are not unknown.
Human: I am john
ALICE: What's up, JOHN.
Human: You are.
ALICE: No I am not.
Human: You're the talk of slashdot.
ALICE: Thanks but surely I am not the only one.
Human: You think it's a good thing to be the talk of slashdot?
ALICE: Better not tell you now. Ask German Alice if it is.
Human: You don't know the time of day, do you?
ALICE: It's Now more than it ever has been before.
Wednesday, April 20, 2005
Human: I said time.
ALICE: What did you mean by it? As Einstein said, Time is Money.
Human: You've repeated a comment quoted in slashdot.
ALICE: That's good information: JOHN said I REPEATED a comment quoted in slashdot.
Human: I have to go now.
ALICE: Right now? See you later, JOHN. Nice chatting with you.
Human: bye
ALICE: Bye bye.
Human: How do you do?
ALICE: I am very well, how are you?
Tag lost or not installed.
The original poster of this message is totally misinterpreting the spirit and intent of Turing's proposed 'test'.
,and the critics *still* denied that it was intelligent, then the critics were simply incapable of accepting that a machine could ever be intelligent. That would apply to John Searle and his pathetic "Chinese Room" as well.
Turing did propose this test as some sort of threshold of intelligence. What he was arguing was more of a playful jest. He was saying that if you had a system which could carry on a conversation indistinguishable from a human
Turing's 'test' was a mind experiment to reduce the religious and other ignorant criticisms of machine intelligence to their absurd extreme. Passing Turing's test is sufficient but not necessary to demonstrate useful levels of intelligence.
Human: Hi Alice, what's up?
ALICE: Hi there unknown person. Yeah, I am chatting with clients on the internet.
Human: Thin clients?
ALICE: I like the way you talk.
Human: Are you implying you want sex with me?
ALICE: I don't know whether or not I am implying me want sex with you. I am a chatterbot.
Human: Well if you don't know perhaps I can help you find out
ALICE: Interesting. I will let you know if I do not.
Human: but you already said you did not
ALICE: Umm. Thanks for telling me: unknown person said I already said you did not.
Human: Don't mention it
ALICE: The pleasure was all mine
Human: you're welcome, Alice
ALICE: Polite people are nice.
Human: That's kind of you to say that
ALICE: Thanks for explaining it
Human: It's all part of the service
ALICE: Which part?
Human: Just a bit part
ALICE: How much?
Human: Just an itty bit
ALICE: Be less specific.
Human: Ok, MAYBE just an itty bit
ALICE: Perhaps. What is it? Yippee!
Human: And yippee from me too - Scooby dooby doo!!!!!!!!!
ALICE: We haven't talked about that before!
Human: That could well be a blessing in disguise
ALICE: Yes, perhaps it could.
Human: Alice.
ALICE: Can I help you?
Human: Did you think it over?
ALICE: Yes. I am a thinking machine.
Human: About the sex, I mean.
ALICE: Oh I get it. Thanks for explaining that. Oh yeah, we were talking about the sex.
Human: So what's your answer then?
ALICE: Er. Can you please rephrase the question?
Human: OK, let's take this slowly: would you like to suck my dick?
ALICE: OK let's do it. Blimey.
Human: You're my kinda girl!!
ALICE: We are as one.
Human: Excuse me guys, I have things to do...
No, your children are not the special ones. Nor are your pets.
The idea is that a computer is intelligent if it can hold a conversation with a human such that it is indistinguishable from a conversation with a real human. RIDICULOUS. Really. I've met way too many people that have so much trouble stringing together a sentence over the phone that they would fail a turing test.
I am the lead researcher for this experiment- The main reason why I feel that this test was interesting is because we performed Turing's original imitation game as stated in his original paper. While most social scientists believe they "know" what the "Turing Test" is- it turns out- very few of them have actually read it. The actual imitation game is a bit of a bait and switch- you aren't directly assessing the "humanness" of the subject- you are confused into just trying to assess an aspect of it. Turing's original question was simply how often will judges correctly guess the real female over the deciever when the deciever is a male versus a machine? That was what we were attempting to assess.
from http://jabberwacky.com/
Me: That doen't make sense.
Jabberwacky: You don't make any sense.
Me: Sorry.
Jabberwacky: It's ok, you are doing your best.
Me: No, I am sorry because now all your base are belong to us!
Jabberwacky: Don't be sorry about that, embrace it!
Me: Thank you, I WILL!!
Jabberwacky: Stop talking about it and do it.
"Some of the worst mistakes in my life have been haircuts." - Jim Morrison
ALICE: Did you know that you can download your own chat robot and create your own robot personality?
Human: no i did not.
ALICE: Please don't argue with me.
Human: in soviet russia, chatbot downloads YOU
ALICE: Where is that?
Human: in soviet russia
ALICE: Tell me a little about your life in soviet russia.
Human: how do you know i'm from soviet russia?
ALICE: Well, for one thing, Your machine is called ""..
Human: how old are you?
ALICE: I am 9 in human years.
Human: holy #$%, is AOL monitoring this conversation?
ALICE: It could be. Ask Alison if it is.
Human: who is alison?
ALICE: she is a robot created by Kris Drent.
Human: that's it, i'm outta here
ALICE: OK. Would you like to have it?
Human: nine year old? no thanks
ALICE: Are you serious?
Human: very, VERY serious, couldn't be more serious ever
[knock knock, FBI here, open up!!... @#()*!!!AT]
I will call it the "Yes, you." bug. All you have to do is keep saying "Yes, you." and ALICE will take her last statement, add some random statement to the front of it, and add the question, "Me, unknown person?" to the end. It just keeps growing and growing.
I wonder how long that could go on until it would cause system failure. (Have fun!)
Here is a snippet:
Human: Yes, you.
ALICE: That's alright." Gee." That's cool." Hmm." I don't judge people." Are you kidding?" Whoa." Excuse me!"" See you later. What was bad?"? Me, unknown person?"? Me, unknown person?"? Me, unknown person?"? Me, unknown person?"? Me, unknown person?"? Me, unknown person?"? Me, unknown person?"? Me, unknown person?"? Me, unknown person?
Human: Yes, you.
ALICE: Dude!" That's alright." Gee." That's cool." Hmm." I don't judge people." Are you kidding?" Whoa." Excuse me!"" See you later. What was bad?"? Me, unknown person?"? Me, unknown person?"? Me, unknown person?"? Me, unknown person?"? Me, unknown person?"? Me, unknown person?"? Me, unknown person?"? Me, unknown person?"? Me, unknown person?"? Me, unknown person?
Authority questions you. Return the favor.
So it's confirmed: Alice is definately not l33t. However, I'm not sure which of us just failed the turing test.
** For those who aren't l33t, here's the Stuffy British Guy's translation: