Human Interference In Computer Chess Championship?

migstradamus writes "In a twist with interesting implications for the computer chess world, the intervention of a human programmer and a human arbiter have had a decisive impact on the World Computer Chess Championship that finished today in Graz, Austria. What happens when a programmer acts against his creation's best interest? ChessBase has an eye-witness report on the dilemma. This year's event was already controversial due to the disqualification of one of the programs midway through for being derivative of an open source program."

Comments

why the need for operators?

[gl4ss]

why not just make them battle through some computer programs, with _no_ human interaction?

much easier, faster, you could have online competitions as well with the same system.

Re:why the need for operators?

[migstradamus]

Eliminating the humans entirely has been suggested off and on for many years. Many of the same programs play online all the time without a human operator and engine-engine matches on the same PC are standard. Using a neutral server as intermediary would seem to be the best plan. Adding network capability wouldn't be any harder than making a GUI, certainly.

Tradition is probably the biggest impediment. Having humans making the moves on a real board, pressing a clock, and writing the moves down allow them to use regular arbiters and the human rules for the most part. That's how we ended up with the mess this year.

The rule in question about claiming a repetition draw BEFORE you make your move is just to make sure you confirm it's a draw on your own clock time. This makes sense for humans, but since a computer can detect repetitions trivially, enforcing that rule in a comp-comp event is like having a rule about no talking or eating at the board for them.

The uber comp-chess guys are splitting hairs about whether it was the GUI or the program itself that claimed the draw, and whether or not it actually claimed anything or was just pointing out that the repetition had occurred. This is mostly foolish because of course if the computer had seen anything better to play it wouldn't have repeated the position three times whether it was aware of the implications of the repetition or not. So any such repetition should be taken as a draw.

If the programmers want to add threefold repetition awareness, and most have, in order to use it for contempt purposes, that's great. (That way they can tell it to avoid repetitions against weaker opponents or in must-win situations unless the alternative is fatal. This is what we call the contempt setting.)

At the end of the day, the letter of the law was followed correctly. Because the machine did not follow the obsolete FIDE rules and claim the draw before making the move, the claim would have been disallowed no matter what the programmer wanted to do. (One hopes.) But the event highlights several weaknesses of using human rules in machine events and in letting operators interfere with programs at all.

Re:why the need for operators?

[bwt]

This is mostly foolish because of course if the computer had seen anything better to play it wouldn't have repeated the position three times whether it was aware of the implications of the repetition or not.

Actually the computer playing the white pieces (Shredder) was in a position that it evaluated as overwhelmingly better. Because of a bug in its programming, it didn't factor 3-fold repitition when it had a forced win. It had engaged in the repition in order to fully calculate all the way to mate. When it found the winning line it happened to go through a position that had been repeated twice before and it wasn't programmed to avoid the draw in a line with a forced win.

I actually think the ruling was absurd -- the computer did announce to its human user ahead of time the move it intended to play and the fact that it was a threefold repitition. The programmers intent in this situation was perfectly clear. To put the burden on the human to decide whether or not to claim the draw kind of defeats the idea of it being computer chess. Clearly the computer was observing that the draw rule was satisfied, even if it didn't say "I claim a draw".

As a result of this ruling, I see no reason that a human user couldn't throw a game on purpose for the benefit of a third party competitor by ignoring the computers move and playing another. I mean after all perhaps e2-e4 is just an observation of a legal move.

If anything, the program playing the black pieces should have been DQ'd from the program for allowing human discretion in the situation.

The ICGA: A board of old imposters

[Anonymous Coward]

LIST doesn't use bitboards as I am told so how could it be a Crafty clone at all??

Dann Corbit had seen the source of a former version and he judged all as completely different to CRAFTY. Ulli Tuerke (COMET) say that the two progs are
totally different in their behaviour.

These imposters should imediately retire after this tournament. People like Bruce Moreland and other younger characters should lead the union of computerchess. Fritz Reul has his examins in mathematics this week so it is a crime to disturb him at his home.

What I find interesting is the following quote:

"The program List is suspected to be a clone of the program Crafty. Autor Fritz Reul failed to prove otherwise and allowed a final deadline to pass."

So apparently an author's reputation and integrity can be maligned on "suspicion." What is truly tragic is the "assumed guilty" posture of the accused. LIST was suspected to be a clone, and was disqualified because the author failed to prove otherwise. How incredibly unjust.

Furthermore, it does not appear that the ICGA followed thier own rule:

"Each program must be the original work of the entering developers. Programming teams whose code is derived from or including game-playing code written by others must name all other authors, or the source of such code, in their application details. programs which are discovered to be close derivatives of others (e.g., by playing nearly all moves the same), may be declared invalid by the Tournament Director after seeking expert advice. For this purpose a listing of all game-related code running on the system must be available on demand to the Tournament Director." [emphasis added]

LIST was only suspected of being a clone; it was not discovered to be a clone. The rule, as written, places the burden on the ICGA to prove it is a close derivate of another before disqualification; it does not place the burden on the accused to prove that it is not a derivative. Thus, the rule is inapplicable to the present situation.

The ICGA needs a procedure to follow in resolving these disputes. Apparently it has none, so it made a procedure up at the expense of an author's reputation. Furthermore, the ICGA has now possibly damaged the author's reputation beyond repair. Allegations of copyright infringement are serious concerns in the software community. Finally, the ICGA should have accommodated the accused author's schedule - is it too much to ask to give a person a small reprieve while he tends to examinations rather than publicly call into question the author's integrity in a worldwide publication on the Internet.

A public retraction is in order, and an apology.

Re:The ICGA: A board of old imposters

[You're All Wrong]

"The ICGA needs a procedure to follow in resolving these disputes"

But it does have one - the inspection of the code.
Fritz refused to permit the inspection of his code.
ICGA run this show, if Fritz doesn't play by their rules, he's out on his ear.
I thought the flexibility offered by the ICGA was perfectly respectable. Remember - this is in the _middle_ of a tournament, decisions need to be made sooner rather than later.

However, I respect Dann Corbit, from my exchanges with him in other fields and believe him to be honest and entirely trustworthy and professional.

So quite probably Fritz is _innocent_ of plagiarism, but _guilty_ of stubornness.

It is their show. Like it or lump it.

I'd like to know what would happen if he were now were to submit his full program source. Would the ICGA lift or shorten the ban? (He is still guilty of not following the expected protocol, after all.)

YAW.

Treat it as a bug...

[JMZero]

A draw isn't automatic, it needs to be claimed. As such, there's a decision to be made. The program in question wasn't set up to make that decision clear. It's the same as if it didn't tell the operator what piece to promote a pawn to. Is the operator just to make an assumption and drop a queen? That's silly - it's a program bug.

In the future, this just needs to be a requirement - the message box needs to say "I claim a draw - three repetitions". In addition, the program needs to be smart enough not to mention anything if a draw claim is available in a favorable position.

Re:Treat it as a bug...

[You're All Wrong]

But how can the notification of a three move repetition be not an instruction to claim the draw? If all the other programs use the same ambiguous announcement, then you could say that it's not ambiguous, and that it actually _means_ "claim a draw".

However, as some ambiguity does remain it's a good safeguard for the ICGA to specify a list of acceptable phrases that are to be taken to mean "claim a draw" (and thus direct instructions the hyuman may not decline to follow).

Using principles of human linguistics (if that's what people say when they mean X,then it means X) and looking at things in time order:
1) the stronger program had a crap bug.
2) the weaker program did actually draw first it notified the outside world that that state had occured.
3) the human driving it followed the protocol for making the TD aware of this.
4) the TD failed to understand the situation.
5) the human driver broke both the rules by not following the computer's instructions and protocol by continuing to play.

So all three parties went awry here.
However, there's nothing against the rules in having bugs, so the first place where something went wrong was the _TD_ dismissing the information he was given as not requiring immediate resolution.

However, the TDs have a very difficult job, and it's an unfortunate situation that's occured.

If I were on a committee (I am for other games with strict protocols, and by heck, we've had a lot worse than this in our time), in review I'd:
- award the draw to the computer that claimed it.
- admonish the player for breach of protocol. (perhaps disqualification for one tournament).
- get lots of feedback from all competing authors, the ICGA exists _for_ them, and must serve their common interests. Yes, rules (protocol) meeetings can be excedingly boring, but it's only when you thrash things out that you can reach conclusions.
- issue an unambiguous directive regarding ambiguous statements.

YAW.

Not an Open Source Issue

[Isao]

This isn't being looked at because it's open-source, but because code may have been used without credit (plagerism) and the code may be too similar to the original (tourney rules prohibit gameplay too similar to other programs).

The author also has failed to reveal his code to the committee despite several opportunities.

Hmmm

[JMZero]

However, as some ambiguity does remain it's a good safeguard for the ICGA to specify a list of acceptable phrases that are to be taken to mean "claim a draw" (and thus direct instructions the human may not decline to follow).

I think that's the important thing here - make it clear that the computer is responsible for the decision and there's no more silliness here. It's an unfortunate situation - and it arose purely due to unclear rules. I don't think anyone needs admonishment, and I think the resolution reached is fair enough to all parties.

They may also need a rule in place in the odd case that neither computer claims a draw in a repeating situation.

Meh?

[JMZero]

If a draw claim is available (to your opponent), then you are by definition not in a favorable position.

The option to draw is only available to the person who has the move. Thus, it's quite possible that a draw claim is available while you are in a favorable position (and thus choose not to take it). In this situation, you'd also want to make it unavailable to your opponent's next turn by breaking the repetition.

See the regular FIDE rules for how this works.

Re:LIST was disqualified, not Fritz

[You're All Wrong]

Fritz is, believe it or not, Fritz Reul's, first name.

A computer program cannot "refuse to permit the inspection of his code".
Computer programs cannot be "_innocent_ of plagiarism, but _guilty_ of stubornness".

It was evidently a human that was being talked about. A human who is called Fritz by dint of his name being, of all things, Fritz.

YAW.

You misunderstand...

[JMZero]

Shredder may have made a mistake, but it was a gameplay mistake - not a bug. We're in agreement on this, and I'm not quite sure how you thought otherwise after reading my post.

The bug was in Jonny's program - it identified a possible draw condition without being clear that it wanted to take the draw. While it may have been clear in this situation that taking the draw was the right move, there are many other situations when taking the draw is the wrong play. In my last post, which I'm not sure you read, I compared this to the choice of which piece to promote a pawn to.

Imagine if I dialog box popped up that said "Promote Pawn". For the purposes of a competition like this, that would be a bug. It moves the burden of decision to the operator, which isn't a real option.

Why else would it identify this than to claim the draw?

When conventions aren't formalized, there's a capacity for misunderstandings. There's all sorts of ambiguous interfaces for this that could be made. What if the program had simply had a label showing "third repetition"? Would that be clear enough? Why not just standardize on a message that has a clear intent - a popup box saying "Draw claimed: third repetition" (or some other standardized wording). Similar standardization should be made for any other decision that the program may need to make during the course of a game.

PS: I realize that this isn't the most natural way to read the situation. But it's the best way, I think, to resolve the situation to satisfaction. Why? Because it places the blame on the competitor (the program) rather than on an outside force (the operator or the judge).

Admonishing the operators or judges might help, but fixing the programs (via standardization of this sort of decision) will fix the problem forever.