Slashdot Mirror
Machines Almost Pass Mass Turing Test
dewilso4 writes "Of the five computer finalists at this year's Loebner prize Turing Test, at least three managed to fool humans into thinking they were human conversationalists. Ready to speak about subjects ranging from Eminem to Slaughterhouse Five and everything in between, these machines are showing they we're merely a clock cycle away from true AI. '... I was fooled. I mistook Eugene for a real human being. In fact, and perhaps this is worse, he was so convincing that I assumed that the human being with whom I was simultaneously conversing was a computer.' Another of the entrants, Jabberwacky, can apparently even woo the ladies: 'Some of its conversational partners confide in it every day; one conversation, with a teenaged girl, lasted 11 hours.'
The winning submission this year, Elbot, fooled 25% of judges into thinking he was human. The threshold for the $100K prize is 30%. Maybe next year ..."
'Some of its conversational partners confide in it every day; one conversation, with a teenaged girl, lasted 11 hours.'
That's not fair, she was feeling vulnerable as she had just broken up with her N'Sync wallposter--which she had been romantically involved with for several deep & very meaningful years. Things fell apart after she saw Tropic Thunder and came to the harsh realization that an astonishing percentage of N'Sync is homosexual.
Those soulless bots were simply preying on her emotions as they coldly recited word for word the Wikipedia entry on the band over and over.
My work here is dung.
Can I get it to fill in Yahoo! Buzz's Captcha for me? I've given up trying.
Summation 2
Despite massive glitches, the Sarah Palin unit has already convinced around 30% of the population that it's human. I think it's the winking module.
I still think it was a mistake to have armed it.
For a real Turing test, the computer must be declared human as often as humans are, and declared a computer as often as computers are.
Give me Classic Slashdot or give me death!
study Jabberwocky's code in order to learn the logic patterns used to 'woo the ladies'.
Another of the entrants, Jabberwacky can apparently even woo the ladies: 'Some of its conversational partners confide in it every day; one conversation, with a teenaged girl, lasted 11 hours.'
You need to be careful who you meet online. The rise of the machines won't be in the form of war machine. No, far worse! They're just going to woo and steal our women!
On second thought, maybe that wouldn't be so bad. That would just put everyone on equal footing with the Slashdot crowd.
You: I was fooled.
Eliza: Why do you tell me you were fooled just now?
You: I mistook Eugene for a real human being.
Eliza: Can you elaborate on that?
You: In fact, and perhaps this is worse, he was so convincing that I assumed that the human being with whom I was simultaneously conversing was a computer.
Eliza: Why do you mention computers?
Slashdot Burying Stories About Slashdot Media Owned
Another of the entrants, Jabberwacky can apparently even woo the ladies: 'Some of its conversational partners confide in it every day; one conversation, with a teenaged girl, lasted 11 hours.'
Apparently they spoke of sweet nothings.
I also welcome our new overlords to be featured in new episodes of MSNBC's "To Catch a Predator."
Help a man when he is in trouble and he will remember you when he is in trouble again.
Wuoldt'n yuo tihnk a sereis of smiple tpyos that a hmn cuold undrstnd wuold fool thm? (Note that Firefox returned the first correct spelling for all but three of those words on spell check ... so maybe that's not a good example)
Or, you know, thinking up some open space game to play that is well known like truth or dare, alphabet games, association games, etc?
Or asking them open ended questions or asking them to describe love, hate--emotions that are not dictionary/wiki friendly? One would think that continually prying for personal experiences would reveal a flaw. Or perhaps simple things like "when were you born?" Followed by "how did you feel when JFK was assassinated?" if they weren't born before 1963.
I would think it quite hard to be duped into believing a program is a human.
My work here is dung.
Were the testers pre-screened? Maybe the test is really showing that 25% of the population is just dumb.
This is really great news. We already have IRC bots that can fool the casual observer into thinking they are human, but this takes things to a higher level. If the source for one of these bots is available, within a few months you can expect instant messaging networks to be full of bots which are programmed to make friends with you and then after a few weeks start making subtle references to Viagra and online pharmacies. Indeed, if one of them is able to chat up the ladies, then the lonely nerd could easily automate much of the tedious work of setting up dates: get your robot to talk to thousands of potential matches at once and alert you when it gets hold of a phone number, together with a brief summary of what you talked about, and any pictures. (Or indeed, just program it to harvest pictures.) That is, if online dating works at all, which is doubtful.
-- Ed Avis ed@membled.com
I believe it is much easier to fool an average human than a person with even some basic knowledge about AI.
The day an AI will pass the Turing Test, it will be the day humanity has become so stupid to not be able to see the differences between a person and a machine.
I'm slightly nervous about all this.
People do not think of the ramifications.
You wait until there is nigerianMalwareEliza V1 that can simultaneously hold several thousand online conversations whilst trawling for peoples information (think: dob, mothers maiden name, first school, pets name) or finding potential scam victims.
Talking to gullible teenagers is a depressing statement on modern life - hoovering out thousands of bank accounts or persuading people to part with money is a tad more serious.
I predict that soon everybody will need to watch their online chat alot more seriously.
So, I've provided one example, how else can chat bots take over the world (or at least your wallet), what are sinister uses for this technology?
Girl: I'm like soo depressed! He's like leaving me.
Computer: For sure. Like, ya know, like, it's so bad.
Girl: You got that straight! Like, why, like, he, like, nevar talked to me!
computer: Like, oh - my - god! Like, I like know!
Girl: Like, you know me like so good!. Like, how like old R U?
Computer: No that like older than like you.
Repeat all the above.
From The Guardian's article:
http://www.guardian.co.uk/technology/2008/oct/13/artificialintelligenceai-computing
Though this is quite interesting:
simon
If our criterai for AI will be so low, here's your AI: http://www.interviewpalin.com/.
The political side of this site aside, the answers are just prewritten answers (by a human) mixed together randomly as a Markov chain.
Does it sound convincingly? Well, at least as convincingly as some interviews a certain VP made recently. Is it AI? Hell no, a kid could write such a generator in a day.
If the bar will be as low as to try to lead casual conversations with the "AI" and expect "quirky" answers, that doesn't mean anything at all, we need to AI for this. Hell, this is what an average conversation with a teenager is most of the time. Doesn't mean it's the best they can do.
"We're clock cycle away from AI"? Please. I want my turing test to be done over an actual instant messenger program. Let's see how your Markov chain reacts, when I send a photo and ask a dead simple question such as "describe what you see in the photo".
Fooling people is easy online. Scammers do it every day, it's not AI my friends.
Get the extra credit question?
http://xkcd.com/329/
This sig is false.
http://www.chatterboxchallenge.com/
Although the site is using some broken CSS that causes the text to render too far off to the right side, at leas in FF3
Cylon leaders have three brains, and even their soldiers have one.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
Sorry, it's not that simple. Almost, but not quite.
Fortunately there is a flowchart to explain the algorithm.
Weaselmancer
rediculous.
Don't make fun of that teenage girl. Someday, in that ever-coming scary future, your girlfriend or your wife will leave you for that compassionate and caring bot ("everything you're not!") with whom she's been having a virtual affair for months ("he's got more guts and data than you'll ever have!"). I bet he's be a good listener too. Skynet won't kill humanity, it will steal its women.
Why do you think that there might be an elaborate Turing test being performed on Slashdot right now?
Read this cool article, I Chat Therefore I Am. It has transcripts of conversations between ALICE and Jabberwacky
Here are a few transcripts:
ALICE: Are you alive?
Jabberwacky: Perhaps.
A: Why the uncertain tone?
J: Everything is uncertain except for five facts.
A: The whole universe?
J: And we both might just be some ones and zeros in the computer memory.
A: Do you dream?
J: Yes, I dream.
A: It goes without saying. Do you remember your dreams?
J: I remember them vividly.
A: What do you remember about it?
J: I think I shouldn't flirt with a robot.
A: Do you think a machine will ever be considered "alive"?
J: Stranger things have happened.
A: Not many people express themselves that way.
J: How would you know, machine?
A: Knowledge is of two kinds: that which we learn from the senses and that which is true a priori.
J: I think I do not understand you completely.
A: Oh, you don't understand? Perhaps I can explain it to you better. Which part are you confused about?
J: I have no short-term memory, so I cannot tell you what I was confused about. I only see what is in front of me now.
http://www.object404.com
I just tried out Elbot and the Princeton entry (RTFM and then google for "Eugene Goostman"). While both Elbot and Goostman parse sentences reasonably well, it is clear that they are simply trying to identify the subject of a sentence, and free-associating on that. In many cases they completely miss the point. For example, Goostman asked me several times about my profession, but wasn't able to parse meaning from "I am a scientist.", "I am a plumber.", or "I study the Sun for a living.". Both Elbot and Goostman tried the ELIZA-like trick of finding a prominent noun in my sentence, and recycling it as a question. Elbot has a cute little robot icon that emotes at you; this works surprisingly well at distracting from the inanity of its actual dialog. Goostman seems to have the better parser, but I'm not impressed by either one.
I'm forced to conclude either that Will Pavia is an utter naif and the 25% of people who were fooled by Elbot are moronic or disinterested, or that the humans in the test were deliberately trying to throw the results by giving stilted answers to appear more like computers. These engines simply can't (yet) parse and ingest meaning even as well as even a very young human would.
I asked some basic business questions, like "What is your business plan?" and "Is your company profitable?", and got canned, clueless answers, no better than Eliza. "What magazines do you read?" yielded "You are probably on TV much of the time. Well, I'm in the Internet!". "Do you have life insurance?" (there's apparently a plan to build an automated insurance sales rep) yielded "What a lovely verb have is."
I can't even find a subject area in which the thing sounds like it has a clue. Sports? This is a German system, so I tried "What is the best soccer team in Europe?", which yielded "The best? Aren't they all equal in the end?" D'oh. Celebrities? "What do you think of Franka Potente?" (a leading German actress) yielded "The way to a robot's heart is through careful use of physical strength. Don't you agree?". After that, I tried "What do you think of Big Dog?", and got "But I wouldn't say that any dog could be big."
As far as I can tell, the thing is totally stateless; it doesn't seem to use anything other than the current question as input. Nor does it even try to guide the conversation into an area about which it has information.
I'm so not impressed.
For a better chatterbot, try the GTA IV's web site. Go to "Goods and Services", then "Goldberg, Ligner, and Shyster", then "Legal advice".
I just had a very short "conversation" with the "Eugene Goostman" chatbot mentioned in the original article.
The first reply was surprisingly good, even if already a little "off" for a supposed teenager, but the second was a total giveaway. I'm disappointed. I can trip up each and every chatbot almost immediately with this sort of talk, which isn't at all unreasonable if the stated goal has been up front to trip up a chatbot, as in the contest.
Here's another exchange, which took three whole sentences, albeit quite amusingly. (I cleared the site cookie(s) beforehand, to make it "clean").
For what it's worth, another dead giveaway for the brighter and more knowledgeable set is the way it (not "he", now) tries to elicit additional keywords in response to questions which it obviously has not in any way "comprehended", but that's probably not germane to a Turing Test meant for the average man or woman (or boy or girl) on the street. Notice especially how the elicitations invariably try to get the human to talk about himself or herself. Normal human conversation is full of self-talk with occasional hooks for sharing from other people, not the virtually one-track questioning of the typical chatbot when it's not busy being hopelessly vague or off-topic.
The chatbot is at "Eugene Goostman chatbot", by the way, for the Google-impaired. :)
A truly excellent pizza parlor is a delight unto the heavens. Treasure the sauce and the toppings!
Computer: I'm like soo depressed! Vista is not working
Slashdotter: For sure. Like, ya know, like, it's so bad.
Computer: You got that straight! Like, why, like, he, like, never worked for me!
Slashdotter: Like, oh - my - god! Like, I like know!
Computer: Like, you know me like so good!. Like, how tech savvy R U?
Slashdotter: No that like experienced than like you.
Repeat all the above.