Slashdot Mirror
How I Failed the Turing Test
chrisjrn writes "I stubled across this article today, detailing a man's experiences of being added to AIM Screen Name lists - one full of "celebrities" and the other full of "Sex Bots" (he was, of course, neither of these).
Raises a few questions as to how easy it is to get a hold of your screenname, and also of the effectiveness of the Turing Test for AI, in the online world. Or is it just that people aren't bothered trying to tell the humans apart anymore?" Also, it's funny. Don't try to read anything deep into it.
whoever modded me Flamebait obviously has no sence of humor :)
maybe I was aiming a little high with that one...
Isn't this sort of what the article about captchas a few days back was?
Most AI today is extremely specialized. It's not hard to design something that appears to think, if it only has to check for 3 cases.
The problem with speech is that assumming all humans use perfect rules, which they don't, and assuming all computers know the perfect rules, which they don't as well, creates a logistical nightmare. Computers work well with numbers.
Did he say hi? Yes he did, so let's say hi back.
It is really hard to design a bot that would actually analyze what they are saying.
Did he say hi? Yes, he greeted me with a "hello" "Hello to you too."
Obviously a moderator here has never used chatbots.
Those who can make you believe absurdities can make you commit atrocities. - Voltaire
shymuffin32: why do you like music?
jmstriegel: hmm. i've never really considered that.
jmstriegel: hell, i'm not going to be able to contrive a good answer for that one. ask me something else.
he doesn't give a response that proves he even recognizes the question, instead, he gives a brain-dead answer that could be put into any number of questions.
just like, try harder next time dude
This statement:
Don't try to read anything deep into it.
holds true.
Comparing it to Windows will be a moot point, since El Dorado is going to have a 40% larger code base than XP.
Never underestimate the power of a human doing a half-assed job.
"Learning is not compulsory... neither is survival."
--Dr.W.Edwards Deming
Well, actually, his problem in the article is completely different. It's _not_ that he's met people who type worse than bots.
It's that a group of people were told that he's a bot, and nothing (correctly and articulately written) could shake their belief in that. One of them even calls him "worse than eliza" when he tries to argue that he's human.
Some people found a list of bots online, and, you know, that makes it the absolute truth. Everyone on it _has_ to be a bot, because the list says so.
Another group found a list of celebrities, and again, took it as absolute truth. They didn't know _who_ this guy is, _what_ is he supposedly famous for, etc. But OMG, he must be a celebrity because the list says so, and that makes it sooo cooool to talk to him.
Basically it's _not_ the "some people are so stupid they could pass for bots" problem. (Which by itself is very true, but it's not really what TFA is about.) The problem, if you will, is simply "some people are gullible idiots." That's all.
It does leave me with me a bunch of other philosophical and etical questions though. If it's this possible to convince people that John Average is a bot (and in fact, it didn't even involve more "convincing" than writing it on some random list on the internet), what _else_ could you convince them? That John Average is a convicted fellon? A spammer? A paedophile?
And mind you, in this case he got a chance to even try to talk back and plead his case. I can easily think of cases where you don't get that chance. E.g., when a prospective employer googles for your name, you might not even know why you didn't get the job. What completely unrelated Marvin did they find on some bogus list on the Internet, and what image did they build for themseleves out of disparate bits taken out of context?
That said, the problem you mention is very true too. I know I've met people online before, especially in online games, who substantially lowered the bar for a Turing test. It was definitely more fun to talk/play with the bots instead, and you could get more intelligent conversation out of the bots too. Admittedly, online games are a completely different category than IM and chat rooms, but still... It's scary, you know.
A polar bear is a cartesian bear after a coordinate transform.
The slashdot editor-bots need to be reprogrammed to use coral version instead of slashbombing sites.
(Okay, you have less than 10 seconds to answer).
--
Evan
"$30 for the One True Ring. $10 each additional ring!" -- JRR "Bob" Tolkien
yeah... see, i used to hangout in the customization/skinning community, and cause of that you learn to deal w/ a lot of clueless 14 year olds. If you give them a complex enough answer (although i tend to stick with ones that also happen to be what i believe), you quickly weed out the 14 year olds who have brains, and the ones who don't. The ones who've a clue ask follow up questions and respond like they've thought about what you've said, and the ones who don't respond with "LoL" and then change the subject to something inane.
Anyway, long story short. You want someone to stop asking you frivolous questions? Give them answers.
There are lives at stake here!
but why does a robot need to wear shoes?
Fair enough then ;^) - btw I personally prefer a rich harmonic structure, with interesting modulation.
Those who can make you believe absurdities can make you commit atrocities. - Voltaire
"Convincing someone you're human might just be harder than one might think - at least a bit more trouble than just answering a few questions."
Only if that someone is utterly retarded and asks completely retarded questions that don't even have a simple answer. That's the problem there. It's a question so stupid that even I couldn't think of something better to answer there. It's not "what music do you like?" or something else which can get a clear, to-the-point answer. It's "why do you like music?"
Well, try to answer that yourself. Why do you like music? What would you answer there?
Because I sure as heck can't think of any good answer there, generic or not. Screw trying to anwer that in 1 minute on IM. I'm sitting here for the last half an hour thinking about it and still have no bloody idea. Because it's background noise? Well, no, because other background noises (e.g., a lawnmower or some co-workers' chatter) annoy me. What then? I have no clue, and probably 4 out of 5 pyschologists or musicians would have no idea either.
So how would I say that in a way that sounds non-generic? "Hell if I know. I've never thought about it"? Nah, you've just ruled a variant of that as too generic. "Well, why do YOU like it, then?" Nope, sounds like the kind of rephrasing the question back at you that an Eliza program would do.
The only non-generic answer that comes to my mind there is along the lines of "WTF of a retarded question is that? Were you born that stupid, or worked hard to get there?"
By contrast, if shymuffin32 actually had more than a braincell, it would be easy to ask some questions that can get simple, to-the-point answers. In fact, screw questions and answers and try to just have an intelligent conversation.
Want more conclusive? Mix some images in it, which would still throw any AI off the track completely. E.g., point him at a picture of someone holding a siberian cat and see if he comments about the size. (It's one bloody huge breed of cats.) Point him at a drawing of one of the giant guns on rails Germany was planning to build in WW2. See what he thinks about the size of that one. (Tends to get answers between "bloody freaking hell" and "do you think Freud might have something to do with it?") Etc.
A polar bear is a cartesian bear after a coordinate transform.
You mean AI researchers are to blame when its a hot day and the A/C in my new car doesn't blast me in the face with cool air?
Avoid Missing Ball for High Score
But now that he's in Slashdot, he is a celebrity. At least his ISP will think so.
game over man, game over
If the computer must be indistinguishable from a human intelligence, then it's not about fooling some of the people, it's about fooling all of the people. Not just those of average intelligence, but anyone and everyone they stick in front of the keyboard.
If it has to pass all tests, there's no rational scientific reason to test it against anything but your strongest subjects. If you're designing (say) a door that has to withstand a blow from a 10lb sledge hammer, there's absolutely no point in hitting it with any hammer smaller than a 10 pounder.
If a job's not worth doing, it's not worth doing right.
Ya, but one response like that doesn't fail a turing test, keep asking questions over and over and see if the pattern repeats... Sheesh.
He at least got the context right, and gave an original answer, that's a bare minimum human response. The human shouldn't have to "try" to seem human. (Of course, they also shouldn't be trying to seem less than human or the test is unfair in the other direciton)
Maybe those trying to "administer" Turing tests need to learn to have a bit more of an attention span...
Of course, if the pattern repeats and this happens after an extended conversation, this guy probably just isn't a very good listener.
I think the problem we have here is the definition of "passing" the Turing Test. Turing himself used the term very specifically, as in "passing = fooling the interogator", but he never really went into what constitutes an adequate interrogator! The purpose of the Test in the first place was really only to bring the debate around the question of "can machines think" down out of the realm of theology and make it testable. It seems that he probably had other scientists in mind for interrogators, as that was with whom the debate was. What portion of the population would be fooled is a metric somewhat separate from the test itself, but using that, fooling half the people in a random sample seems like too low a threshold. 100% is probably too high. Turing himself only briefly hinted at this once, saying that it'd take at least 50 years before a machine could pass 70% of the time, and 100 or more years before it could fool anyone.
If a job's not worth doing, it's not worth doing right.