Slashdot Mirror
Turing Test Competition At CalTech
Charles Dodgeson writes "The Turing Tournament at Cal Tech wants to
know if you
can program an emulator that will play games like a
human, or if can you write detector that can correctly sort the wetware from the software.
Before you get too excited, the "games" are very limited things. But there is a $10,000 prize for the winner. You can read the gory
details."
According to The Jargon Dictionary wetware is:
/wet'weir/ n. [prob. from the novels of Rudy Rucker] 1. The human nervous system, as opposed to computer hardware or software. "Wetware has 7 plus or minus 2 temporary registers." 2. Human beings (programmers, operators, administrators) attached to a computer system, as opposed to the system's hardware or software. See liveware, meatware.
wetware
I didn't know what it meant... figured other people may not either.
sig.
Page sucks!
Oh, er, hmm. Sorry about that.
As part of a "branding" attempt after around WWII, California Institute of Technology refers to itself as "Caltech", not "Cal Tech".
See this Caltech Institute Archive.
you must have missed this node and possibly this one as well
The game has been defined. It's fairly classic payoff matrix used in game theory. Look up the Prisoner's Dilemma (http://serendip.brynmawr.edu/bb/pd.html) problem. The goal of the game is maximize your outcome (while minimizing the opponent's).
Step 1:
Make up a set of game boards and have a group of humans each play the game on those boards. Each human will play once on each board. This gives us real human data to compare the software to.
Step 2a:
Let each of the submitted emulators play the game on every one of the boards created in Step 1. We now have a set of results for each human and each emulator on all the game boards
Step 2b:
For every detector that was submitted, give if every set of results. It returns its answer for which it thinks are humans and which are emulators in a very precise way. We now have a matrix of (number of humans + number of emulators) x (number of detectors), where each element is a mathematical answer to 'is this a human player'.
Step 3:
Repeat and take the average score. The Detector that was right the most wins.
Step 4:
The emulator that fooled the most detectors wins. If there's a tie (for either emulators or detectors) in the 95% confidence interval for the model used to compute scores, then the prize is shared among the tied entries
Jason
ProfQuotes
As a recent alum (2002), I can shed some light on some things.
:-) )
Why is this in the Deparment of Humanities and Social Sciences?
From the URL, this Tournament is being run by the Social Sciences Experimental Laboratory (SSEL), not the CS department. The SSEL has been one of the leaders in experimental economics research (read: actually testing all those crazy theories you hear in economics classes).
Why is there money involved?
All experiments by the SSEL involve money. As an undergraduate, I participated in many experiments, mostly involving trading "commodities" in simple (and sometimes not-so-simple) markets. We were paid based on our performance. If I had an off night, I got paid $5 for 2 hours of the experiment. If I had a good night, I could make upwards of $80. Yes folks, this is real money we're talking about here. Since the point is to test people's economic thinking, you must make your decisions based on a real outcome, otherwise the data gathered is invalid.
Why then are they doing this test?
I don't work for the SSEL (and never have), but here's why I think they're doing this: Since they're interested in not only individual human behavior, but also how individuals interact and make choices based on the actions of other individuals, it would be useful to design a computer program that mimics other human's behavior. If other humans think this program acts like a human, then you can do two things: you can take the specifications of the computer program and figure out what qualities of the program humans have. As well, you can then replace humans with the computer program in real experiments (this not only allows you to test the limits of the program, but also to save money
And as an alum (who was not too fond of his time there, but still feels compelled to defend Caltech), it's *Caltech*, not *Cal Tech* or *Cal-Tech* (but if you're feeling lazy, *caltech* is all right too).
nak