Slashdot Mirror

← Back to Stories (view on slashdot.org)

Turing Test Competition At CalTech

Posted by timothy on from the move-near-a-major-target-first dept.

Charles Dodgeson writes "The Turing Tournament at Cal Tech wants to know if you can program an emulator that will play games like a human, or if can you write detector that can correctly sort the wetware from the software. Before you get too excited, the "games" are very limited things. But there is a $10,000 prize for the winner. You can read the gory details."

8 of 173 comments (clear)

  1. No Hacking Around! by core+plexus · · Score: 3, Interesting
    From the description: " The goal of this project is to study human behavior in repeated games, and to find and document the best algorithms available. The goal is not to probe for weaknesses in the Linux operating system. Each participant has permissions to read, write and execute in your own directory (and any subdirectories of it.) You do not have permissions to read, write or execute programs in any other directory on the host computer. Any attempt to read or write from directories to which you do not have privileges will be considered an act of bad faith, and your algorithm will be disqualified from further competition in the tournament. http://turing.ssel.caltech.edu/node19.html

    Well! Never mind, then.

    Fight with computer brings SWAT team

  2. Re:get it right by Anonymous Coward · · Score: 1, Interesting

    The Caltech site copyright says "Caltech" not "caltech"

  3. Interesting.. by metlin · · Score: 5, Interesting

    From the site:

    ...
    An emulator is computer program that takes as input a stage game file, and gives as output a dataset file. ...
    The input file is in the file game.nfg, and the output file should be written in the file dataset.txt, both of which should reside in the same directory as the executable program. Note that the file game.nfg will be written into each emulator's directory by the tournament program prior to running your emulator. ...

    This is exactly the way ACM ICPC contests are conducted, except that if the systems crash for any reason, you're not given extra time to make up for it :-(

    And sadly, from the site:

    Languages supported:


    The computer program that you submit (for either an emulator or a detector) must be written in a combination of one or more of the following languages:

    C or C++
    java
    Perl
    Mathematica
    Gambit GCL

    I would have expected them to atleast add shell-scripting to this - very useful under such conditions to do some Q&D work, or would be taken for granted that since the shell can be a part of the OS, you are free to use it?

    Also, would have been nice if they'd added Python to the list, and more importantly Forth (yes, despite what you've heard, Forth is indeed useful, just look at Arthur T Murray's Mind Project).

  4. Turing Test? by $carab · · Score: 5, Interesting

    As someone who spent a lot of time working with an actual Turing Test bot, I'd just like to say that the term "Turing Test" really only applies to a "conversation" between computers and people, not emulating behavior in games, etc. I mean, when Turing wrote Computing Machinery and Intelligence, this isn't really what he envisioned to be a Turing Test.

    From what I could gather, this is a lot closer to a programming tournament rather than a Turing Test...

    1. Re:Turing Test? by Vagary · · Score: 3, Interesting

      As someone who spends more time reading research papers than working with an actual Turing Test bot, I'd just like to point out that the academic community has abstracted the term "Turing Test" into something a little more useful:

      A Turing Test is a means for judging the humanness of the behaviour of a software system. The test consists of giving similar input to both a human and an implementation of the software and then comparing or subjectively judging the output. This is often done in an interactive fashion.

      Idealised, a Turing Test consists of a human and a program receiving a bitstream and sending a bitstream back. In most examples, this bitstream consists of text in a natural language and the input and output are expected to occur interactively. However it seems likely that intelligent aliens would fail this if they didn't know the language or could not respond at a speed acceptable to the judge. Therefore your traditional definition of "Turing Test" is seen to be arbitrary.

      And no, citations are not available upon request.

  5. Re:useless by Charles+Dodgeson · · Score: 3, Interesting
    This is not the Loeber prize, which Pinker was describing correctly. This is a much more constrained and narrow contest, focusing on a very well studied portion of human behavior.

    Does anybody actually read the stuff pointed to before posting?

    This actually is useful, but not for AI. There is a whole branch of what is called "experimental game theory". Getting something that plays these games like humans is interesting (well, to me at least).

    --
    Prime numbers are exactly what Alan Greenspan says they are -S. Minsky
  6. AI tournament without AI languages? by jtdubs · · Score: 5, Interesting

    Amazing. They are having an AI tournament, and their supported language list includes C, C++, Java, Perl, Mathematica, and something called the Gambit Command Language.

    Where the hell are the good AI languages? Functional languages? Lisp? Scheme? Caml? SML? Hell, I'd settle for Python.

    Justin Dubs

  7. Pseudoscience! I call bullshit! by rufusdufus · · Score: 5, Interesting

    This 'contest' reminds of the scene in Ghostbusters where Bill Murray manipulates pre-cog cards to get women. Really, if a ESP buff were to present something like this to the Amazing Randi he would not accept it on the grounds that it was too easy to manipulate.

    First and foremost, there is a large sum of money being bandied around. The participants are incented to win by monetary payouts, and two payouts of $10,000 dollars are at stake as well. When games with this high of stakes are being played, great caution is generally used by the house.

    But look at the rules of the game...there basically are none. Participants are identified by e-mail address; no rule is specified about the number of entries per person. Also, no rule is specified about collusion between entrants (detectors and emulators). It doesn't take an einstein to figure out how to bias the results of the experient by making enough colluding entries. It is funny this, given that the games themselves are *about* collusion. Its a joke.

    Next, notice that there really isnt any way for anybody to tell if the results of the experiement are meaningful, or if they have been manipulated. Its not based on a falsafible proposition and is not scientific. Its no different from any method used by psychic hoaxers of the past. The creators of the 'contest' can manipulate the data, and direct the winnings to their confederates.

    Finally, the bizarre nature of the contest should raise some flags. They are giving out $10,000 for a program that would have been hacked together in a couple of months at most. What kind of value could you expect from a contest like this?