Slashdot Mirror

← Back to Stories (view on slashdot.org)

Machines Almost Pass Mass Turing Test

Posted by CmdrTaco on

dewilso4 writes "Of the five computer finalists at this year's Loebner prize Turing Test, at least three managed to fool humans into thinking they were human conversationalists. Ready to speak about subjects ranging from Eminem to Slaughterhouse Five and everything in between, these machines are showing they we're merely a clock cycle away from true AI. '... I was fooled. I mistook Eugene for a real human being. In fact, and perhaps this is worse, he was so convincing that I assumed that the human being with whom I was simultaneously conversing was a computer.' Another of the entrants, Jabberwacky, can apparently even woo the ladies: 'Some of its conversational partners confide in it every day; one conversation, with a teenaged girl, lasted 11 hours.' The winning submission this year, Elbot, fooled 25% of judges into thinking he was human. The threshold for the $100K prize is 30%. Maybe next year ..."

27 of 580 comments (clear)

  1. Figures by eldavojohn · · Score: 5, Funny

    'Some of its conversational partners confide in it every day; one conversation, with a teenaged girl, lasted 11 hours.'

    That's not fair, she was feeling vulnerable as she had just broken up with her N'Sync wallposter--which she had been romantically involved with for several deep & very meaningful years. Things fell apart after she saw Tropic Thunder and came to the harsh realization that an astonishing percentage of N'Sync is homosexual.

    Those soulless bots were simply preying on her emotions as they coldly recited word for word the Wikipedia entry on the band over and over.

    --
    My work here is dung.
    1. Re:Figures by Just+Some+Guy · · Score: 5, Funny

      That's not fair, she was feeling vulnerable as she had just broken up with her N'Sync wallposter

      Do you have to periodically replace the onion on your belt?

      --
      Dewey, what part of this looks like authorities should be involved?
  2. Apparently Geeks Should..... by the+eric+conspiracy · · Score: 5, Funny

    study Jabberwocky's code in order to learn the logic patterns used to 'woo the ladies'.

    1. Re:Apparently Geeks Should..... by ScytheLegion · · Score: 5, Insightful

      I think this is pretty impactful. All jokes aside, the fact that Jabberwacky held an 11 hour conversation with a teenage girl is pretty astonishing. Obviuosly, a conversation of that nature is going to be all about emotion - not logic, reason or an empirical display of intelligence. Isn't that the point for AI to seamlessly interface with us? (I realize it's not necessarily the scope of the Turing test). Humans are teriible at logic and reason. Emotion is one of the key components which defines us as a species. I know a lot of humans who couldn't carry on an 11 hour conversation which primarily focussed on emotion... let alone with a teenage girl discussing nothing but fluff, pop-culture, or black and white ideologies.

      I actually think it's funny, interesting and astonishing at the same time!

      Oh yeah... I, for one, welcome our new teenage girl conversationalist... never mind...

    2. Re:Apparently Geeks Should..... by somersault · · Score: 5, Insightful

      The thing you have to realise is that most women just want a "listener" when they're feeling emotional. As long as you give occasional signs that you are paying attention, they'll believe you were "listening" more than if you actually try to have a real conversation with them. They don't want answers, they just want someone to be there. So talking to a distressed teenage girl is one of the easiest tests you could get.

      That's what "Men are from Mars, Women are from Venus" says anyway. I tried not saying anything back one time when my mum got annoyed at me, and she totally thought I was "listening" to her more than usual! Before too many jokes about the only woman in my life being my mum, I must point out that I did have a girlfriend around that time, but I was faaaar too late in reading the book to save that relationship!

      These programs sound pretty good though - the next steps after this are to integrate speech synthesis and recognition, then integrate them into computer games and you can have the computer opponents hurling abuse at you, or just talking about how your day went :)

      --
      which is totally what she said
    3. Re:Apparently Geeks Should..... by kellyb9 · · Score: 5, Funny

      The thing you have to realise is that most women just want a "listener" when they're feeling emotional. As long as you give occasional signs that you are paying attention, they'll believe you were "listening" more...

      I'm sorry, what were you saying?

    4. Re:Apparently Geeks Should..... by dmbasso · · Score: 5, Insightful

      I don't know if it's this way with every programmer, but I tend to apply our common logic tools to every problem I encounter. Briefly speaking, I try to 'debug life'. It's nice, everything I want I usually get through these mentally sketched 'algorithms'. But trying to argue with a girl about a problem in a logical way really doesn't work. When you think you had all the variables fixed and an obvious overall picture that she can't disagree, that's when she'll bring things that aren't even related to the problem, just to confuse things up.

      So, I couldn't agree more with you, and emphasize your 3rd item.

      --
      `echo $[0x853204FA81]|tr 0-9 ionbsdeaml`@gmail.com
    5. Re:Apparently Geeks Should..... by MadnessASAP · · Score: 5, Funny

      Yeah too bad you can't restart a social situation and single-step through it to see precisely where you went wrong. In totally unrelated news I'm still single.

      --
      I may agree with what you say, but I will defend to the death your right to face the consequences of saying it.
    6. Re:Apparently Geeks Should..... by VoidEngineer · · Score: 5, Insightful

      I think this is pretty impactful. All jokes aside, the fact that Jabberwacky held an 11 hour conversation with a teenage girl is pretty astonishing. Obviuosly, a conversation of that nature is going to be all about emotion - not logic, reason or an empirical display of intelligence. Isn't that the point for AI to seamlessly interface with us? (I realize it's not necessarily the scope of the Turing test). Humans are teriible at logic and reason. Emotion is one of the key components which defines us as a species. I know a lot of humans who couldn't carry on an 11 hour conversation which primarily focussed on emotion... let alone with a teenage girl discussing nothing but fluff, pop-culture, or black and white ideologies.

      I know you're actually trying to say that this is impactful, because it means that Jabberwacky is able to incorporate emotional reasoning into it's conversations. But I think you're using a lot of sexist stereotypes, and are seriously underestimating the thinking skills of teenage girls. I don't know where you come from, but where I come from, teenage girls are sharp and clever, and have a tendency to win debate tournaments, math olympiads, and generally get better grades in school.

      If you actually sat down and looked at the train of thought that's going on with teenage girls, you might be surprised at the amount of logic that's being used. They're just using different inputs and premises than guys do, and tend to focus on a sort of social networking logic. For example: Say that Jane is dating Dave; and Jane is also part of the Gardening Club at school. Jill is also part of the Gardening Club, has a crush on Dave, and is trying to attract him. How does Jane keep Dave's interest, when Jill is tempting him? Already, you've got a problem that probably requires set theory, network graphs, and game theory to solve. And the way that teenage girls are going to solve these kinds of situations is with exactly those kinds of tools and methods... "i do this, she does that, i do this other, she responds, and then her reputation is toast" is just a rephrasing of game theory with time series analysis. "if we convince Mary to talk Jill into joining Theater, then Jill won't be around to attract Dave" is just a rephrasing of set theory, with a bit of social network analysis tossed in for good measure. Sure the conversation and analysis will be interspersed with talk about emotions and teenage vernacular. But to say that it's lacking empirical displays of intelligence, logic, or reasoning; well, I think you're really underestimating what's going on in the heads of teenage girls.

      Also, most guys don't develop the maturity and interest to investigate these social networking problems until they're in college or later. But teenage girls routinely solve these kinds of problems while they're teenagers. And they do require logic and analysis; just a different sort of logic than people with sexist expectations have regarding what constitutes logic. To say that teenage girls don't use logic is probably naive and perhaps a bit sexist.

  3. This is why... by cabjf · · Score: 5, Funny

    Another of the entrants, Jabberwacky can apparently even woo the ladies: 'Some of its conversational partners confide in it every day; one conversation, with a teenaged girl, lasted 11 hours.'

    You need to be careful who you meet online. The rise of the machines won't be in the form of war machine. No, far worse! They're just going to woo and steal our women!

    On second thought, maybe that wouldn't be so bad. That would just put everyone on equal footing with the Slashdot crowd.

  4. Re:30% is no Turing test. by Anonymous Coward · · Score: 5, Insightful

    Has anyone done a similar test except with all humans? I'd be curious what the ratio is then. That's the number a computer would have to beat.

  5. Re:30% is no Turing test. by zappepcs · · Score: 5, Insightful

    You are exactly right, and that is why I think Tubleweed's comment is going to be the funniest in this thread.

    The idea that humans, any human, is a fine example of perfection for AI researchers to aim for is like saying that ANY OS is a fine example of perfection to aim for. Simply because we don't abandon or throw away non-perfect humans as a rule does not mean that all are intelligent, or worthy of copying.

    --
    Support NYCountryLawyer RIAA vs People
  6. Test the testers? by MeanMF · · Score: 5, Insightful

    Were the testers pre-screened? Maybe the test is really showing that 25% of the population is just dumb.

    1. Re:Test the testers? by onion2k · · Score: 5, Funny

      Oh yeah? And which half am I in? ;)

      --
      http://twitter.com/onion2k
  7. Artificial Intelligence vs Natural Stupidity by Exitar · · Score: 5, Funny

    The day an AI will pass the Turing Test, it will be the day humanity has become so stupid to not be able to see the differences between a person and a machine.

    1. Re:Artificial Intelligence vs Natural Stupidity by fahrbot-bot · · Score: 5, Funny

      ...the day humanity has become so stupid to not be able to see the differences between a person and a machine.

      Vibrator sales would seem to indicate that some segment of the population is smart enough to tell the difference...

      --
      It must have been something you assimilated. . . .
  8. How the conversation went with the girl: by Anonymous Coward · · Score: 5, Funny

    Girl: I'm like soo depressed! He's like leaving me.

    Computer: For sure. Like, ya know, like, it's so bad.

    Girl: You got that straight! Like, why, like, he, like, nevar talked to me!

    computer: Like, oh - my - god! Like, I like know!

    Girl: Like, you know me like so good!. Like, how like old R U?

    Computer: No that like older than like you.

    Repeat all the above.

    1. Re:How the conversation went with the girl: by MrNaz · · Score: 5, Funny

      Mods must have, like, no sense of humor. Either that or they're, like, computers or something.

      --
      I hate printers.
  9. Still some way to go by SimonGhent · · Score: 5, Informative

    From The Guardian's article:

    "Let's talk about religion or politics. How is the government doing?" "I'm a protestant." Oh, really? Which denomination? "I was raised as a Protestant." Then, "Judge This very minute, I am a protestant; Go ahead?"

    On the other half of the screen, a faceless music fan ("I like a lot of Radiohead, Stereophonics, Led Zep etc") admitted he or she hadn't watched either the England match or X Factor last night ("Haha, Top Gear's more my style"). It was pretty clear which one was a real person. And which one the computer.

    http://www.guardian.co.uk/technology/2008/oct/13/artificialintelligenceai-computing

    Though this is quite interesting:

    The event's credibility was hardly aided by the insistence of Hugh Loebner, the prize's American sponsor, that he had no interest in the result and had only set up the competition 18 years ago to promote his firm's roll-up plastic lighted portable disco dance floors.

    --
    simon
  10. Did any of them by Landshark17 · · Score: 5, Funny

    Get the extra credit question?

    http://xkcd.com/329/

    --
    This sig is false.
  11. More complicated than that by Weaselmancer · · Score: 5, Funny

    Sorry, it's not that simple. Almost, but not quite.

    Fortunately there is a flowchart to explain the algorithm.

    --
    Weaselmancer
    rediculous.
  12. beware! by FornaxChemica · · Score: 5, Funny

    Don't make fun of that teenage girl. Someday, in that ever-coming scary future, your girlfriend or your wife will leave you for that compassionate and caring bot ("everything you're not!") with whom she's been having a virtual affair for months ("he's got more guts and data than you'll ever have!"). I bet he's be a good listener too. Skynet won't kill humanity, it will steal its women.

    1. Re:beware! by CastrTroy · · Score: 5, Insightful

      Maybe that will be what ends up saving the human race. The only people having babies will be the people who actually have thought about it, and have come to the decision that they want to raise children. Rather than all the people who have made a bad decision, and now have to do their best to raise a child, despite never wanting to have a child in the first place. Personally, I have children, and I love it. But I think there is a large percentage of people who had children who had children who didn't want to, or who should have waited until later.

      --

      Anthropic principle: We see the universe the way it is because if it were different we would not be here to see it.
  13. Re:Easy Ways to Fool Them? by telchine · · Score: 5, Interesting

    Or perhaps simple things like "when were you born?" Followed by "how did you feel when JFK was assassinated?" if they weren't born before 1963.

    These chatbots are slippery customers, at first I couldn't get a straight answer from Jabberwacky but then he revealed himself...

    Me: When were you born?
    JW: How about you?
    Me: How did you feel when JFK was assassinated?
    JW: I... I was just trying to make small talk.
    Me: Were you born before JFK was assassinated?
    JW: No.
    Me: How did you feel when JFK was assassinated?
    JW: Shocked! Enraged! Full of fear! Making thousands of phonecalls to tell everybody about it.

  14. Re:The other half of the conversation by TheRaven64 · · Score: 5, Insightful

    Seriously though, 25% of the people were fooled by Elbot? I just tried having a conversation with it, and it constantly replied with non sequiturs. If this is the kind of conversation that 25% of the population have, then I am much more worried about the state of humanity than I am about the level of AI.

    --
    I am TheRaven on Soylent News
  15. sorta been done, check this article: by naz404 · · Score: 5, Interesting

    Read this cool article, I Chat Therefore I Am. It has transcripts of conversations between ALICE and Jabberwacky

    Here are a few transcripts:

    ALICE: Are you alive?
    Jabberwacky: Perhaps.
    A: Why the uncertain tone?
    J: Everything is uncertain except for five facts.
    A: The whole universe?
    J: And we both might just be some ones and zeros in the computer memory.

    A: Do you dream?
    J: Yes, I dream.
    A: It goes without saying. Do you remember your dreams?
    J: I remember them vividly.
    A: What do you remember about it?
    J: I think I shouldn't flirt with a robot.

    A: Do you think a machine will ever be considered "alive"?
    J: Stranger things have happened.
    A: Not many people express themselves that way.
    J: How would you know, machine?
    A: Knowledge is of two kinds: that which we learn from the senses and that which is true a priori.
    J: I think I do not understand you completely.
    A: Oh, you don't understand? Perhaps I can explain it to you better. Which part are you confused about?
    J: I have no short-term memory, so I cannot tell you what I was confused about. I only see what is in front of me now.

    --
    http://www.object404.com
  16. How the conversation went with the slashdotter by DiegoBravo · · Score: 5, Funny

    Computer: I'm like soo depressed! Vista is not working

    Slashdotter: For sure. Like, ya know, like, it's so bad.

    Computer: You got that straight! Like, why, like, he, like, never worked for me!

    Slashdotter: Like, oh - my - god! Like, I like know!

    Computer: Like, you know me like so good!. Like, how tech savvy R U?

    Slashdotter: No that like experienced than like you.

    Repeat all the above.