Slashdot Mirror

← Back to Stories (view on slashdot.org)

Turing Test Passed

Posted by samzenpus on from the almost-human dept.

schwit1 (797399) writes "Eugene Goostman, a computer program pretending to be a young Ukrainian boy, successfully duped enough humans to pass the iconic test. The Turing Test which requires that computers are indistinguishable from humans — is considered a landmark in the development of artificial intelligence, but academics have warned that the technology could be used for cybercrime. Computing pioneer Alan Turing said that a computer could be understood to be thinking if it passed the test, which requires that a computer dupes 30 per cent of human interrogators in five-minute text conversations."

6 of 432 comments (clear)

  1. Re:Thirty percent? by NoNonAlphaCharsHere · · Score: 5, Insightful

    Because most humans would fail?

  2. Re:Thirty percent? by nitehawk214 · · Score: 5, Insightful

    By random chance you would detect the computer 50% of the time, so that should be the goal.

    Still 30% as "passing" seems unreasonably low.

    --
    I'm a good cook. I'm a fantastic eater. - Steven Brust
  3. Re:Turing Test Failed by EuclideanSilence · · Score: 5, Insightful

    It's a bit of an underhanded way to pass to pretend to be someone who doesn't speak English natively. The point of the test is to have a conversation for 5 minutes, not 5 minutes of "oh I can't understand you because I'm from Ukraine".

  4. Re:A pretty low requirement by tangent · · Score: 5, Insightful

    I'd say we keep raising the bar.

    "If a computer can play chess better than a human, it's intelligent."
    "No, that's just a chess program."

    "If a computer can fly a plane better than a human, it's intelligent."
    "No, that's just an application of control theory."

    "If a computer can solve a useful subset of the knapsack problem, it's intelligent."
    "No, that's just a shipping center expert system."

    "If a computer can understand the spoken word, it's intelligent."
    "No, that's just a big pattern matching program."

    "If a computer can beat top players at Jeopardy, it's intelligent."
    "No, it's just a big fast database."

  5. The 'test' was fixed by Camael · · Score: 5, Insightful

    What has been conducted precisely matches Turing's proposed immitation game.

    While they may have matched the letter of it, they subverted the spirit of the test. This quote from the programme maker in particular is highly suggestive that they lowered the standards :-

    The computer programme claims to be a 13-year-old boy from Odessa in Ukraine.

    "Our main idea was that he can claim that he knows anything, but his age also makes it perfectly reasonable that he doesn't know everything," said Vladimir Veselov, one of the creators of the programme. "We spent a lot of time developing a character with a believable personality."

    To illustrate what I mean by lowered standards, imagine if I set up the same test, with 10 entries, and I tell the judges some of them are 2 year old babies playing on the keyboard. Armed with this information, some of the judges are likely to interpret even gibberish as typed by a human and it is not too farfetched to get more than 30% of them to agree.

    This "result" is bollocks and a pure publicity stunt conveniently on falling on the 60th anniversary of Turing's death.

    I want to see the actual transcripts which do not appear to have been released so far, which in itself is highly suspicious.

    1. Re:The 'test' was fixed by Dr.+Spork · · Score: 5, Insightful
      Here was a sample of a hypothetical conversation from Turing's original article:

      Interrogator: In the first line of your sonnet which reads "Shall I compare thee to a summer's day," would not "a spring day" do as well or better?

      Witness: It wouldn't scan.

      Interrogator: How about "a winter's day," That would scan all right.

      Witness: Yes, but nobody wants to be compared to a winter's day.

      Interrogator: Would you say Mr. Pickwick reminded you of Christmas?

      Witness: In a way.

      Interrogator: Yet Christmas is a winter's day, and I do not think Mr. Pickwick would mind the comparison.

      Witness: I don't think you're serious. By a winter's day one means a typical winter's day, rather than a special one like Christmas.

      I think the problem is that the way Turing was picturing the test, the human interrogators would be as smart as Turing and his friends, people who actually know how to ask probing questions. When you look at the conversation above, you see that he had in mind a program that does things which is decades beyond of what chatbots can do today. Everybody is dissing the Turing test, and if it has a problem, it's in that Turing overestimated people, in assuming that they actually know how to have conversations of significance. I still think there is something deeply significant about the Turing test, but in the one that I'm picturing, the interrogators must all be broadly educated experts on natural language processing with specific training in how to expose chatbots. And there should be money on the line for the interrogators: $1000 bonus for each correct identification, $2000 penalty for incorrect identification, no penalty for "not sure". If the majority of such experts can be fooled by an AI under these circumstances, then I think we should all be impressed.