Slashdot Mirror

← Back to Stories (view on slashdot.org)

Upgrading the Turing Test: Lovelace 2.0

Posted by Soulskill on from the just-make-sure-to-skip-version-9.0 dept.

mrspoonsi tips news of further research into updating the Turing test. As computer scientists have expanded their knowledge about the true domain of artificial intelligence, it has become clear that the Turing test is somewhat lacking. A replacement, the Lovelace test, was proposed in 2001 to strike a clearer line between true AI and an abundance of if-statements. Now, professor Mark Reidl of Georgia Tech has updated the test further (PDF). He said, "For the test, the artificial agent passes if it develops a creative artifact from a subset of artistic genres deemed to require human-level intelligence and the artifact meets certain creative constraints given by a human evaluator. Creativity is not unique to human intelligence, but it is one of the hallmarks of human intelligence."

5 of 68 comments (clear)

  1. This sounds interesting... by Anonymous Coward · · Score: 0, Insightful

    Lovelace 2.0

    So, you are blindfolded and have to figure out if it's a human deep-throating you, or the latest Flashlight?

  2. Turing test is fine by itzly · · Score: 5, Insightful

    There's nothing wrong with the Turing test, but it needs to have some thought put into the set up and execution, plus competent judges.

    1. Re:Turing test is fine by narcc · · Score: 3, Insightful

      Did you miss the last 64 years of research and philosophy? The last hold-outs, save the most delusional, we're knocked out by Searle in 1980.

      It's only controversial for those who haven't read Turing's paper, or have completely failed to understand it.

      Eliza, for example, highlights the massive failure in Turing's reasoning -- The question "can machines think" is not equivalent to the question "Are there imaginable digital computers which would do well in the imitation game?"

      Weisenbaum found the response to his program from non-technical staff disturbing.

      Secretaries and nontechnical administrative staff thought the machine was a "real" therapist, and spent hours revealing their personal problems to the program. When Weizenbaum informed his secretary that he, of course, had access to the logs of all the conversations, she reacted with outrage at this invasion of her privacy. Weizenbaum was shocked by this and similar incidents to find that such a simple program could so easily deceive a naive user into revealing personal information.

      ( From Eliza to A.L.I.C.E. )

      Further, the so-called "Turing test" hasn't held still. Not even in his 1950 paper! (Turing proposed multiple variations on the test, if you'll recall.) Since then, a number of different versions of the "Turing test" have appeared, none of which are (like Turing's variations) are equivalent to one another!

      If you need a *really* simple argument: The results of any variation of the "Turing test" are completely subjective. Consider a program that fools 100% of one set of interrogators may completely fail to fool even 10% of another set.

      --
      Required reading for internet skeptics
  3. moving target by Anonymous Coward · · Score: 2, Insightful

    This is just making the "Turing test" into a moving target. The Turing test makes sense, and if you have a long enough test you can eventually rule out the "abundance of if statements."

  4. Inference is Hard by infogulch · · Score: 3, Insightful

    'In the following sentence: "Ann gave Sue a scarf. She was very happy to receive it." Does "she" refer to Ann? (yes/no)'
    A series of similar and increasingly difficult inference questions like this one can usually knock over an AI pretty easily, while not being too difficult for humans.