Slashdot Mirror
Turing Test Competition At CalTech
Charles Dodgeson writes "The Turing Tournament at Cal Tech wants to
know if you
can program an emulator that will play games like a
human, or if can you write detector that can correctly sort the wetware from the software.
Before you get too excited, the "games" are very limited things. But there is a $10,000 prize for the winner. You can read the gory
details."
According to The Jargon Dictionary wetware is:
/wet'weir/ n. [prob. from the novels of Rudy Rucker] 1. The human nervous system, as opposed to computer hardware or software. "Wetware has 7 plus or minus 2 temporary registers." 2. Human beings (programmers, operators, administrators) attached to a computer system, as opposed to the system's hardware or software. See liveware, meatware.
wetware
I didn't know what it meant... figured other people may not either.
sig.
its "caltech" NOT "Cal Tech"
The rules are written in a very obscure minimalist fashion, so it took me a while to figure this out but: the game has not been defined! Your program is to get an input file and process it in a manner similar to a human. Currently the website is lacking examples of human output. Therefore, from an Information Theory perspective, we know absolutely nothing about the game.
Now, since I wasted my time figuring this out, I also decifered the instructions: basically the pool consists of a bunch of humans and "emulators" (programs). Each one is given a set of input files that they are supposed to transform into output files. Then the set of output files is run through a detector (human or machine?) that gives the participant some score. You win if your score is most like the humans' scores.
The Turing Test was developed by Alan Turing as one of his "free time endeavors." (I swear, that man had too much time on his hands.) Either way, the Turing Test has been the unsurpassed test for true AI and a human-wetware algorithm. The Turing Test dictates that only when a computer is fully capable of handling a conversation with a human, being able to respond to questions in logical, grammatically correct formats AND being able to learn new slangs and vocabularies - that is true AI (according to Turing). I think this CalTech game is about the same - it's not merely making a computer make mistakes and saying human-like taunts. It's truly.. being human. Those once-in-a-while lucky shots, beginner's luck, fresh leg advantages, the works. That's what they're looking for. Besides, it's CalTech. I think they deserve a little more credit than just merely modifying existing AI. =)
Page sucks!
Oh, er, hmm. Sorry about that.
As part of a "branding" attempt after around WWII, California Institute of Technology refers to itself as "Caltech", not "Cal Tech".
See this Caltech Institute Archive.
you must have missed this node and possibly this one as well
Step 1:
Make up a set of game boards and have a group of humans each play the game on those boards. Each human will play once on each board. This gives us real human data to compare the software to.
Step 2a:
Let each of the submitted emulators play the game on every one of the boards created in Step 1. We now have a set of results for each human and each emulator on all the game boards
Step 2b:
For every detector that was submitted, give if every set of results. It returns its answer for which it thinks are humans and which are emulators in a very precise way. We now have a matrix of (number of humans + number of emulators) x (number of detectors), where each element is a mathematical answer to 'is this a human player'.
Step 3:
Repeat and take the average score. The Detector that was right the most wins.
Step 4:
The emulator that fooled the most detectors wins. If there's a tie (for either emulators or detectors) in the 95% confidence interval for the model used to compute scores, then the prize is shared among the tied entries
Jason
ProfQuotes
They are having an AI tournament, and their supported language list includes C, C++, Java, Perl, Mathematica, and something called the Gambit Command Language.
They're having an AI tournament on something that's more related to Game Theory and which is why GCL has been mentioned.
GCL is a HLL that's used for testing game theory related approaches. It supports a lot of important factors in game theory related operations, like vectorization and form representaion switching.
Read this Caltech site for more on GCL.
GCL may not be very well known outside the AI/GT areas, since its used more in a purely CS research oriented environment. I think it started out as a series of C++ libraries for GT related stuff.
Prime numbers are exactly what Alan Greenspan says they are -S. Minsky
As a recent alum (2002), I can shed some light on some things.
:-) )
Why is this in the Deparment of Humanities and Social Sciences?
From the URL, this Tournament is being run by the Social Sciences Experimental Laboratory (SSEL), not the CS department. The SSEL has been one of the leaders in experimental economics research (read: actually testing all those crazy theories you hear in economics classes).
Why is there money involved?
All experiments by the SSEL involve money. As an undergraduate, I participated in many experiments, mostly involving trading "commodities" in simple (and sometimes not-so-simple) markets. We were paid based on our performance. If I had an off night, I got paid $5 for 2 hours of the experiment. If I had a good night, I could make upwards of $80. Yes folks, this is real money we're talking about here. Since the point is to test people's economic thinking, you must make your decisions based on a real outcome, otherwise the data gathered is invalid.
Why then are they doing this test?
I don't work for the SSEL (and never have), but here's why I think they're doing this: Since they're interested in not only individual human behavior, but also how individuals interact and make choices based on the actions of other individuals, it would be useful to design a computer program that mimics other human's behavior. If other humans think this program acts like a human, then you can do two things: you can take the specifications of the computer program and figure out what qualities of the program humans have. As well, you can then replace humans with the computer program in real experiments (this not only allows you to test the limits of the program, but also to save money
And as an alum (who was not too fond of his time there, but still feels compelled to defend Caltech), it's *Caltech*, not *Cal Tech* or *Cal-Tech* (but if you're feeling lazy, *caltech* is all right too).
nak
How can the human create information without an algorithm?
What you are citing with Godel proves that humans must use algorithms too. It's just that the algorithms are very complex and not understood. The is no reason we can't learn and duplicate the human algorithms, and that's what this contest is all about.
You do a nice job proving that a little knowledge is a dangerous thing.
Jason
ProfQuotes