Turing Test 2: A Sense of Humor

mhackarbie writes "Salon has a great story, Artificial Stupidity, about the Loebner Prize, a yearly contest that for over 10 years now has offered a $100,000 prize to anyone who can create a program to pass the Turing Test. The best part is the resulting fiasco that develops between the eccentric philanthropist who started the contest and extremely annoyed AI Researchers such as Marvin Minsky."

Comments

Sundman

[tcdk]

John Sundman who has written this article has also written a quite interestion book called Cheap Complex Devices (he mentions is in the article).

It's kind of wierd and strange - the idea is that the novel was one of two novels written by a computer program.

I've reviewed it here.

Competence vs Performance

[clasher]

I don't know if it is the case in this instance but the Turing Test rubs some the wrong way because it is a pretty lousy test for intelligence. The turing test measures the performance of something not it's competence.

What we see is what the computer does and not what goes on behind the scenes, which many people believe is important in positing intelligence in a agent. One of the major problems with behaviorism was that it initially took into account only how an animal performed and not what it was thinking. Sure the rat could learn the maze when it is rewarded for running thorught it, but it could also learn the maze (competence) by being pulled through it on a little cart or when it was completely sated. The performance of something may be important in judging its intelligence but it is far from the only factor. Imaginge a person in a paralyzed state, they have the competance but lack the ability to performance.

Like I said this may not be the issue as discussed in the article, but it is one caveat to the Turing Test.

Why the Turing Test is a waste of time

[profBill]

The following is an excerpt from an article by Drew McDermott about the "Red Herring Test". I always thought it pointed out quite well why the Turing Test seems like such a waste of time.

What confuses most people is that they mistake Turing's attempt to avoid the question for an attempt to answer it. But anyone who believes that Turing's test is an interesting test for intelligence is guilty of behaviorism, not a crime in itself, but shameful in anyone who believes in cognitive science, the antithesis of behaviorism. Of course, it is probably true that a system that could fool a trained panel of experts into believing it intelligent would in fact be intelligent, but it is blatant waste of experts' time to have them sit on such panels, when they should be inquiring about how minds actually work.

Compare the following hypothetical case: Human explorers land on a planet whose inhabitants are somewhat technologically backward. The locals are impressed by human gadgets, especially radio. They decide to try and understand it, so they rustle up some philosophers in order first to arrive at a criterion for something's being a radio. Their first cut is that a radio is a device that emits sounds whenever similar sounds are made in the control room of the earthlings' spaceship. But others object that this criterion does not rule out ordinary telephony, so the criterion is modified. Perhaps they arrive at something like, ``A radio is a device that emits sounds similar to those made in the earthlings' spaceship while suspended from the ceiling by a nonconducting string.''

This is all amusing, but a waste of time if the aliens really want to understand radio. No one needs an ironclad behavioral criterion for ``radiohood,'' assuming that there are plenty of indisputably genuine radios around to study. Such a study might eventually lead to a deeper definition of radio as ``A receiver of signals encoded as modulated electromagnetic waves,'' but by the time the definition was available it would be relatively unimportant, when stacked up against the theory of electromagnetism.

Similarly with intelligence. If we ever have a theory that explains it, we will no longer care about distinguishing bogus understanding from the real thing. We will have a rich theory based on concepts we can now barely imagine, just as radio is based on something as unlikely as invisible electromagnetic waves.

Re:Why are they upset?

[manyoso]

That is not fair. Minsky might have percieved that this was the case, but it doesn't follow that it was. Loebner gave a perfectly good explanation for the clause (See below) and it seems pretty hypocritical that Minsky fumes that Loebner uses his name as co-sponsor in advertising ;)

From: loebner@ACM.ORG (Hugh Loebner)
Newsgroups: comp.ai
Subject: Minsky Co-sponsor of Loebner Prize!
Date: 8 Mar 1995 16:48:36 GMT
Organization: ACM Network Services
Lines: 63
Message-ID:

In Message ID Minsky writes:

>In article loebner@ACM.ORG writes ....
>>17.The names "Loebner Prize" and "Loebner Prize Competition" may be used by
>>contestants in advertising only by advance written permissionof the Cambridge
>>Center, and their use may be subjecttoapplicableicensingfees. Advertising is
>>subjecttoapprovalbyrepresentativesoftheLoebn e r Prize Competition.Improper or
>>misleading advertising may result in revocationoftheprizeand/or other actions.

>[Some words concatenated to enforce the 80-character line length
>convention.]

>I do hope that someone will volunteer to violate this proscription so
>that Mr. Loebner will indeed revoke his stupid prize, save himself
>some money, and spare us the horror of this obnoxious and unproductive
>annual publicity campaign.

>In fact, I hereby offer the $100.00 Minsky prize to the first person
>who gets Loebner to do this. I will explain the details of the rules
>for the new prize as soon as it is awarded, except that, in the
>meantime, anyone is free to use the name "Minsky Loebner Prize
>Revocation Prize" in any advertising they like, without any licensing
>fee.

1. Marvin Minsky will pay $100.00 to anyone who gets me to
"revoke" the "stupid" Loebner Prize.

2. "Revoke" the prize means "discontinue" the prize.

3. After the Grand Prize is won, the contest will be
discontinued.

4. The Grand Prize winner will "get" me to discontinue the
Prize.

5. The Grand Prize winner will satisfy The Minsky Prize criterion.

6. Minsky will be morally obligated to pay the Grand Prize
Winner $100.00 for getting me to discontinue the contest.

7. Minsky is an honorable man.

8. Minsky will pay the Grand Prize Winner $100.00

9. Def: "Co-sponsor": Anyone who contributes or promises to
contribute a monetary prize to the Grand Prize winner .

10. Marvin Minskey is a co-sponsor of the 1995 Loebner Prize
Contest.
-------------
BTW

The language that Minsky finds so offensive was added
by the Prize Committee because of a possible mis-representation
regarding the contest made by an annual prize winner.

No fees have been requested of any winner, nor do I anticipate
of any fees ever being requested. Rule 17 merely protects the
Loebner Prize from misrepresentation in advertising.

Re:What is Intelligence?

[Stuntmonkey]

Or, to put it another way, Intelligence means "I don't understand how you thought that".

This is an interesting view, and there are a couple of corollaries if you take it seriously:

• Corollary 1: Intelligence depends on how a calculation is done, not merely on its results. In other words, the Turing test is fundamentally flawed. A chess-playing Deep Junior is unintelligent because we know that in principle we could look at its code and understand the rules behind its behavior (in this case, alpha/beta search plus a position evaluator). By contrast, Kasparov is intelligent because he evaluates the gestalt of a position in a way that we cannot analytically understand. Even if we could closely monitor his neural activity and follow the computation in his brain, it is a collective activity of so many constituent neurons that it is not "conceptually reducible" to us as Deep Junior's algorithm is. One would conclude that even if Kasparov were to lose to a computer, he would still be playing the game intelligently whereas the computer would not.
• Corollary 2: True intelligence can only be achieved by systems that are opaque to "why did you conclude that?" kinds of analyses. Collective/swarm intelligence would be opaque in this way, but rules/knowledge-based systems would not. This would imply that the "traditional" AI approach is doomed to failure.

Re:I particurly like how Loebner

[loebner]

No.

Minsky specifically offered a $100 prize to anyone who would "get me" to discontinue the contest, and hoped that someone would win it. I pointed out that the winner of the Grand Prize would satisfy Minsky's criterion and that Minsky would be obligated to reward him/her = co-sponsoring the contest.