Slashdot Mirror
How I Failed the Turing Test
chrisjrn writes "I stubled across this article today, detailing a man's experiences of being added to AIM Screen Name lists - one full of "celebrities" and the other full of "Sex Bots" (he was, of course, neither of these).
Raises a few questions as to how easy it is to get a hold of your screenname, and also of the effectiveness of the Turing Test for AI, in the online world. Or is it just that people aren't bothered trying to tell the humans apart anymore?" Also, it's funny. Don't try to read anything deep into it.
It's that the dialog of a typical IM user can't be distinguished from a brain-dead conversation bot...
The NSA: The only part of the US government that actually listens.
Well, this article shows that at last we came to the point where a bot is comparable to a human being at a chat room. In fact, we didn't get to this point with better AI, but with worse RI (real intelligence, if the term applies to this case).
Ladies and Gentleman: a completely insensitive and unintelligen bot can be more interesting to chat with than a human! Well, at least they write correctly (N07 L@M3 @SS).
Google for sex bots and look at the first link. It's an article that he wrote, and his screen name is in it.
Forget genetic discrimination in the future, I can't even farking sign up for slashdot anymore. Soon I won't be able to get my welfare check because of these stupid turing tests!
Isn't this sort of what the article about captchas a few days back was?
Most AI today is extremely specialized. It's not hard to design something that appears to think, if it only has to check for 3 cases.
The problem with speech is that assumming all humans use perfect rules, which they don't, and assuming all computers know the perfect rules, which they don't as well, creates a logistical nightmare. Computers work well with numbers.
Did he say hi? Yes he did, so let's say hi back.
It is really hard to design a bot that would actually analyze what they are saying.
Did he say hi? Yes, he greeted me with a "hello" "Hello to you too."
My favourite snippet has to be:
jmstriegel: no, really. I'm quite human.jmstriegel: test me if you want
shymuffin32: ok
shymuffin32: why do you like music?
jmstriegel: hmm. i've never really considered that.
jmstriegel: hell, i'm not going to be able to contrive a good answer for that one. ask me something else.
shymuffin32: jeesus, you're worse than eliza
It's not him that's stupid (as claimed elsewhere), it's these shymuffin32 morons.
Those who can make you believe absurdities can make you commit atrocities. - Voltaire
... from those buddies on your list. I really fail to see exactly what the "security" risk is here - if you're hypersensitive about the people messaging you, then you can choose to be hypersensitive, lose some functionality and turn off the "randomness" factor. Most people exchange IM names through some other means of communications, either verbally or written, so this loss of functionality can be sidestepped while maintaining your online secrecy.
Hagrin.com
I mean, have you seen the typical chat room conversation?
:) !
user1: ~~OMG~~
bot1: Want to see my sexy pics?
bot2: Want to see my sexy pics?
bot3: Want to see my sexy pics?
user2: WUT!?
bot1: Want to see my sexy pics?
bot2: Want to see my sexy pics?
user3: LoL
bot1: Want to see my sexy pics?
bot2: Want to see my sexy pics?
bot3: Want to see my sexy pics?
user1: You LOL
bot1: Want to see my sexy pics?
bot2: Want to see my sexy pics?
bot3: Want to see my sexy pics?
user3: STFU LOL!
user2: OMG hAhA!
bot1: Want to see my sexy pics?
bot2: Want to see my sexy pics?
bot3: Want to see my sexy pics?
user1: JK
bot1: Want to see my sexy pics?
I'll believe in AI when a robot can tie shoelaces.
The most convincing AIs I've seen are the bots in FPS games. And they're already programmed to hunt down and kill humans...
"I've got more toys than Teruhisa Kitahara."
Well, actually, his problem in the article is completely different. It's _not_ that he's met people who type worse than bots.
It's that a group of people were told that he's a bot, and nothing (correctly and articulately written) could shake their belief in that. One of them even calls him "worse than eliza" when he tries to argue that he's human.
Some people found a list of bots online, and, you know, that makes it the absolute truth. Everyone on it _has_ to be a bot, because the list says so.
Another group found a list of celebrities, and again, took it as absolute truth. They didn't know _who_ this guy is, _what_ is he supposedly famous for, etc. But OMG, he must be a celebrity because the list says so, and that makes it sooo cooool to talk to him.
Basically it's _not_ the "some people are so stupid they could pass for bots" problem. (Which by itself is very true, but it's not really what TFA is about.) The problem, if you will, is simply "some people are gullible idiots." That's all.
It does leave me with me a bunch of other philosophical and etical questions though. If it's this possible to convince people that John Average is a bot (and in fact, it didn't even involve more "convincing" than writing it on some random list on the internet), what _else_ could you convince them? That John Average is a convicted fellon? A spammer? A paedophile?
And mind you, in this case he got a chance to even try to talk back and plead his case. I can easily think of cases where you don't get that chance. E.g., when a prospective employer googles for your name, you might not even know why you didn't get the job. What completely unrelated Marvin did they find on some bogus list on the Internet, and what image did they build for themseleves out of disparate bits taken out of context?
That said, the problem you mention is very true too. I know I've met people online before, especially in online games, who substantially lowered the bar for a Turing test. It was definitely more fun to talk/play with the bots instead, and you could get more intelligent conversation out of the bots too. Admittedly, online games are a completely different category than IM and chat rooms, but still... It's scary, you know.
A polar bear is a cartesian bear after a coordinate transform.
Georg
"Convincing someone you're human might just be harder than one might think - at least a bit more trouble than just answering a few questions."
Only if that someone is utterly retarded and asks completely retarded questions that don't even have a simple answer. That's the problem there. It's a question so stupid that even I couldn't think of something better to answer there. It's not "what music do you like?" or something else which can get a clear, to-the-point answer. It's "why do you like music?"
Well, try to answer that yourself. Why do you like music? What would you answer there?
Because I sure as heck can't think of any good answer there, generic or not. Screw trying to anwer that in 1 minute on IM. I'm sitting here for the last half an hour thinking about it and still have no bloody idea. Because it's background noise? Well, no, because other background noises (e.g., a lawnmower or some co-workers' chatter) annoy me. What then? I have no clue, and probably 4 out of 5 pyschologists or musicians would have no idea either.
So how would I say that in a way that sounds non-generic? "Hell if I know. I've never thought about it"? Nah, you've just ruled a variant of that as too generic. "Well, why do YOU like it, then?" Nope, sounds like the kind of rephrasing the question back at you that an Eliza program would do.
The only non-generic answer that comes to my mind there is along the lines of "WTF of a retarded question is that? Were you born that stupid, or worked hard to get there?"
By contrast, if shymuffin32 actually had more than a braincell, it would be easy to ask some questions that can get simple, to-the-point answers. In fact, screw questions and answers and try to just have an intelligent conversation.
Want more conclusive? Mix some images in it, which would still throw any AI off the track completely. E.g., point him at a picture of someone holding a siberian cat and see if he comments about the size. (It's one bloody huge breed of cats.) Point him at a drawing of one of the giant guns on rails Germany was planning to build in WW2. See what he thinks about the size of that one. (Tends to get answers between "bloody freaking hell" and "do you think Freud might have something to do with it?") Etc.
A polar bear is a cartesian bear after a coordinate transform.
Some time ago I coupled Perl Eliza module with IM account registered as "Irene17". That module works only for English and my IM network was for non-English users so Irene would welcome anyone with message that she understands English only. I set her status to available for conversation and left it running for a week. That IM network has central directory of users so I was sure that sooner or later someone would find her.
:) frustration with "Good bye" or some insults.
Then I looked at logs of conversations. It turned out that there were people who actually talked with her for quite a while, struggling with English. The scheme was more or less the same. First some usual phrases to start a conversation, then trying to get some information about her and finally realizing that she is unwilling to tell anything about herself
So, in a way, she has passed a Turing test, but the knowledge of English was poor on both sides.
1) people's (AIMers) lower standards for conversation;
2) and also their open mindedness towards what a computer is capable of producing.
I guess the first point is negative and the second positive. The combination leaves a situation where a computer doesn't have to generate anything sophisticated to be tagged as human.
I once administered an informal Turing test using Ray Kurzweil's Cybernetic Poet. I presented to 6 friends several dozen poems, some of which were computer generated (the poems, not the friends...).
People who were computer savvy tended to overestimate what a computer was capable of doing and did rather poorly. Similarly, people who were artistic but not very techie tended to have a very open mind regarding what constituted human poetry (bad grammar, non sequiturs, etc. were ok in an e.e. cummings sort of way) and also did poorly.
The people who did consistently well were those who were neither computer types nor artists, but rather "pure" academics (language specialists, classicists, etc.). They simply used grammar and puncutation as their guide.
i\hbar\dot{\psi}=\hat{H}\psi
Quite possibly, I had never heard of Eliza until I read TFA.
And how does that make you feel?
THE WORLD IS GOING TO END!!!! eventually.