Slashdot Mirror
Machines Almost Pass Mass Turing Test
dewilso4 writes "Of the five computer finalists at this year's Loebner prize Turing Test, at least three managed to fool humans into thinking they were human conversationalists. Ready to speak about subjects ranging from Eminem to Slaughterhouse Five and everything in between, these machines are showing they we're merely a clock cycle away from true AI. '... I was fooled. I mistook Eugene for a real human being. In fact, and perhaps this is worse, he was so convincing that I assumed that the human being with whom I was simultaneously conversing was a computer.' Another of the entrants, Jabberwacky, can apparently even woo the ladies: 'Some of its conversational partners confide in it every day; one conversation, with a teenaged girl, lasted 11 hours.'
The winning submission this year, Elbot, fooled 25% of judges into thinking he was human. The threshold for the $100K prize is 30%. Maybe next year ..."
Has anyone done a similar test except with all humans? I'd be curious what the ratio is then. That's the number a computer would have to beat.
You are exactly right, and that is why I think Tubleweed's comment is going to be the funniest in this thread.
The idea that humans, any human, is a fine example of perfection for AI researchers to aim for is like saying that ANY OS is a fine example of perfection to aim for. Simply because we don't abandon or throw away non-perfect humans as a rule does not mean that all are intelligent, or worthy of copying.
Support NYCountryLawyer RIAA vs People
Were the testers pre-screened? Maybe the test is really showing that 25% of the population is just dumb.
I believe it is much easier to fool an average human than a person with even some basic knowledge about AI.
I'm slightly nervous about all this.
People do not think of the ramifications.
You wait until there is nigerianMalwareEliza V1 that can simultaneously hold several thousand online conversations whilst trawling for peoples information (think: dob, mothers maiden name, first school, pets name) or finding potential scam victims.
Talking to gullible teenagers is a depressing statement on modern life - hoovering out thousands of bank accounts or persuading people to part with money is a tad more serious.
I predict that soon everybody will need to watch their online chat alot more seriously.
So, I've provided one example, how else can chat bots take over the world (or at least your wallet), what are sinister uses for this technology?
I think this is pretty impactful. All jokes aside, the fact that Jabberwacky held an 11 hour conversation with a teenage girl is pretty astonishing. Obviuosly, a conversation of that nature is going to be all about emotion - not logic, reason or an empirical display of intelligence. Isn't that the point for AI to seamlessly interface with us? (I realize it's not necessarily the scope of the Turing test). Humans are teriible at logic and reason. Emotion is one of the key components which defines us as a species. I know a lot of humans who couldn't carry on an 11 hour conversation which primarily focussed on emotion... let alone with a teenage girl discussing nothing but fluff, pop-culture, or black and white ideologies.
I actually think it's funny, interesting and astonishing at the same time!
Oh yeah... I, for one, welcome our new teenage girl conversationalist... never mind...
Games have rules, so you could theoretically pass that test by giving the bot all the rules to commonly played games (Calvinball, OTOH, that would be a good test).
Open ended questions are great, but I'm not sure how they're not wiki friendly. If I ask a bot, "what is love", I'd expect to get back an answer not dissimilar to what's on wikipedia - "emotions relating to a sense of strong affection".
Obviously, Wikipedia is not sentient, so you're going to have to do better than just asking facts/dates (any good ELISA will have a solid backstory).
No, I'd bet that a sufficiently good bot would be able to pass the Turing test with 100% accuracy - but still be not sentient.
Let me throw this out there, though: Intelligence is those processes by which animals obtain and retain information about their environments, and use that information to make behavioral decisions (Kamil, A. C. 1987 A synthetic approach to the study of animal intelligence. Nebr. Symp. Motiv. 7, 257â"308.). Cognition, by my own definition, is the ability to solve a single problem multiple ways (quotidian expedience) and the ability to copy behaviors seen in other individuals (mimicry). It's not a perfect definition, but it's a damn sight closer to a true AI than trying to get ELISAs to pass the Turing Test.
Obligatory Soundbite Catchphrase
Seriously though, 25% of the people were fooled by Elbot? I just tried having a conversation with it, and it constantly replied with non sequiturs. If this is the kind of conversation that 25% of the population have, then I am much more worried about the state of humanity than I am about the level of AI.
I am TheRaven on Soylent News
Appropriate response: "I'm sorry, I don't download files from random strangers... haven't you been following the news about all the ways you can get spyware and viruses?"
That's what I would say, even if it weren't a Turing Test.
The thing you have to realise is that most women just want a "listener" when they're feeling emotional. As long as you give occasional signs that you are paying attention, they'll believe you were "listening" more than if you actually try to have a real conversation with them. They don't want answers, they just want someone to be there. So talking to a distressed teenage girl is one of the easiest tests you could get.
That's what "Men are from Mars, Women are from Venus" says anyway. I tried not saying anything back one time when my mum got annoyed at me, and she totally thought I was "listening" to her more than usual! Before too many jokes about the only woman in my life being my mum, I must point out that I did have a girlfriend around that time, but I was faaaar too late in reading the book to save that relationship!
These programs sound pretty good though - the next steps after this are to integrate speech synthesis and recognition, then integrate them into computer games and you can have the computer opponents hurling abuse at you, or just talking about how your day went :)
which is totally what she said
Or it could go the other way. First...if they give AI a convicing voice...then they can use it and fire all real meat operators on the sex phone lines.
Then, if they can built a realistic android, in female form, that will do anything you want sexually, yet will not give you a disease, have unwanted kids, no commitment (eg ability to take half your stuff), and shut up on request....well, I'd say real meat women are gonna be in trouble.
With proper AI, and advanced robot tech...well, they could make the 'perfect' woman for what men want them for.
Reminds me of the joke:
What is a Cinderella '10'?
A girl that fucks and sucks till midnight...then turns into a pizza and a six-pack.
Light travels faster than sound. This is why some people appear bright until you hear them speak.........
The day an AI will pass the Turing Test, it will be the day humanity has become so stupid to not be able to see the differences between a person and a machine.
I'm mixed on that. That was my first reaction, and then I thought, but if that AI is talking with stupid people, then isn't it at human level?
The other thing is are you calling anyone that doesn't notice that it's a robot/AI stupid? By default, I don't even think of what the other person is. I don't know or care if you are white/black/green, male/female/both/neither, or which political views you hold. On slashdot, I only know you by the 3-4 sentences that you type. Is that enough for anyone really to judge one way or another if someone is human or a robot? Nope.
I'd suggest a good portion of slashdot could be robots, and we'd never notice.
I agree with you...it's responses were very clearly "canned". The judges clearly have very low standards for what it takes to be "human"...though maybe we should just be saying "intelligent".
Give a hand, not a hand-out.
I'd say real meat women are gonna be in trouble.
I fear you've overlooked a rather important function that "real meat women" bring to the human race.
Get your own free personal location tracker
Oh...I'd dare say the earths population would drop a bit. Not that that's a bad thing actually.
I mean, let's face it...I'd have to guess that out of the number of times a guy is fucking, the times he is actually actively wanting to become a father has got to be remarkably low.
Light travels faster than sound. This is why some people appear bright until you hear them speak.........
I don't know if it's this way with every programmer, but I tend to apply our common logic tools to every problem I encounter. Briefly speaking, I try to 'debug life'. It's nice, everything I want I usually get through these mentally sketched 'algorithms'. But trying to argue with a girl about a problem in a logical way really doesn't work. When you think you had all the variables fixed and an obvious overall picture that she can't disagree, that's when she'll bring things that aren't even related to the problem, just to confuse things up.
So, I couldn't agree more with you, and emphasize your 3rd item.
`echo $[0x853204FA81]|tr 0-9 ionbsdeaml`@gmail.com
Wasn't there a Futurama episode about this?
Honesty may be the best policy, but by process of elimination, dishonesty is the second best policy.
Maybe that will be what ends up saving the human race. The only people having babies will be the people who actually have thought about it, and have come to the decision that they want to raise children. Rather than all the people who have made a bad decision, and now have to do their best to raise a child, despite never wanting to have a child in the first place. Personally, I have children, and I love it. But I think there is a large percentage of people who had children who had children who didn't want to, or who should have waited until later.
Anthropic principle: We see the universe the way it is because if it were different we would not be here to see it.
or need someone to bitch to about their girfriends... who just nods and agrees
I think this is pretty impactful. All jokes aside, the fact that Jabberwacky held an 11 hour conversation with a teenage girl is pretty astonishing. Obviuosly, a conversation of that nature is going to be all about emotion - not logic, reason or an empirical display of intelligence. Isn't that the point for AI to seamlessly interface with us? (I realize it's not necessarily the scope of the Turing test). Humans are teriible at logic and reason. Emotion is one of the key components which defines us as a species. I know a lot of humans who couldn't carry on an 11 hour conversation which primarily focussed on emotion... let alone with a teenage girl discussing nothing but fluff, pop-culture, or black and white ideologies.
I know you're actually trying to say that this is impactful, because it means that Jabberwacky is able to incorporate emotional reasoning into it's conversations. But I think you're using a lot of sexist stereotypes, and are seriously underestimating the thinking skills of teenage girls. I don't know where you come from, but where I come from, teenage girls are sharp and clever, and have a tendency to win debate tournaments, math olympiads, and generally get better grades in school.
If you actually sat down and looked at the train of thought that's going on with teenage girls, you might be surprised at the amount of logic that's being used. They're just using different inputs and premises than guys do, and tend to focus on a sort of social networking logic. For example: Say that Jane is dating Dave; and Jane is also part of the Gardening Club at school. Jill is also part of the Gardening Club, has a crush on Dave, and is trying to attract him. How does Jane keep Dave's interest, when Jill is tempting him? Already, you've got a problem that probably requires set theory, network graphs, and game theory to solve. And the way that teenage girls are going to solve these kinds of situations is with exactly those kinds of tools and methods... "i do this, she does that, i do this other, she responds, and then her reputation is toast" is just a rephrasing of game theory with time series analysis. "if we convince Mary to talk Jill into joining Theater, then Jill won't be around to attract Dave" is just a rephrasing of set theory, with a bit of social network analysis tossed in for good measure. Sure the conversation and analysis will be interspersed with talk about emotions and teenage vernacular. But to say that it's lacking empirical displays of intelligence, logic, or reasoning; well, I think you're really underestimating what's going on in the heads of teenage girls.
Also, most guys don't develop the maturity and interest to investigate these social networking problems until they're in college or later. But teenage girls routinely solve these kinds of problems while they're teenagers. And they do require logic and analysis; just a different sort of logic than people with sexist expectations have regarding what constitutes logic. To say that teenage girls don't use logic is probably naive and perhaps a bit sexist.
Until you realize you're on slashdot. Then it sounds like the only option.
Of Code And Men
In other words, you have perfect debugger, which doesn't work. Get the hint.
I hope your logic professor demonstrated, that logic is rather crude tool, which discards a lot of information in order to be comprehensible.
Also, you think that you have all the parameters fixed, but there are uncountable, do you really think a person (especially a child) can list all factors?
To make matters worse, logic is a way to make mistakes with more certainty. Have you got your axioms right? Are you sure all operations are done correctly. At least girls are more immune to this vulnerability (judging form this thread).
I know you've been mod'ed +1 Funny, but as a male who spent over a decade interacting solely with women for 9 hours out of the day, as a matter of survival I've learned that what you've said is right.
The key to realize is that while some D&D players meta-game in their games, women meta-meta-game or meta-meta-meta-game their entire lives.
A good example to explain this would be if you ask a man where get bought a stake. He'll say "Joe's Market" or something. As a woman the same question and she'll respond "Why? What's wrong with it?" (I've purposely chosen one of the simplest meta-levels a woman may operate at so the men can maybe understand.)
Now when men assume women are operating at a direct level and women assume men are operating at a meta-meta-level, you can imagine the confusion that results. One side thinks the other is acting irrationally (like a meta-gaming character would seem to a non-meta-gaming character) and the other things the first is a dolt (like a meta-gaming character would think of a non-meta-gaming character).