Slashdot Mirror

← Back to Stories (view on slashdot.org)

When Chess Players Blunder

Posted by Soulskill on from the knight-to-sand-trap-6 dept.

An anonymous reader writes: Joe Doliner has done a statistical analysis of mistakes in rated chess games. He used a chess engine called Crafty, which is capable of not only finding mistakes, but quantifying how bad they are. After crunching all the matches on chessgames.com in 2014, which amounted to almost 5 million moves, Crafry found only 67,175 blunders that were equivalent to a 2-pawn deficit or worse. With a pair of graphs, Doliner shows how mistakes decrease as player rating increases, as you'd expect. According to the trendline, gaining 600 rating points roughly halves the number of mistakes a player makes. He made the data and tools available in a public repository for others to dig into.

36 of 47 comments (clear)

  1. Cause meet Effect. by thesupraman · · Score: 4, Insightful

    I would suggest that making half as many mistakes gains you about 600 rating points, rather than the other way around.
    Unless those points can be magically sprinkled on a player in some form..
    But hey, cause and effect seem to be highly, shall we say, flexible these days.

    (yes, I know its all semantics here, but hey..)

    1. Re:Cause meet Effect. by gslavik · · Score: 1

      I don't think the article means that there is a causal effect to decreasing mistakes. It is looking at the correlation between points and number of mistakes from the viewpoint of the rating graph. I would agree with your statement as to the cause (half the mistakes leads to ~600 point rating increase).

    2. Re:Cause meet Effect. by Paradise+Pete · · Score: 4, Insightful

      I would suggest that making half as many mistakes gains you about 600 rating points

      I think the big problem is that a "mistake" is being defined so crudely. I'll bet players make about the same number of mistakes no matter how good they are, but what they actually qualify as a mistake gets smaller and smaller.
      It's certainly true for me. I play at about a 1900 level now and was 2200 in my youth. I make tons of mistakes, but at the same time hardly a week goes by when somebody doesn't accuse me of using a computer. In other words, they don't see the mistakes. But I know they're there and to players better than I they are obvious mistakes. But almost none of them are "2 pawn +" blunders.

    3. Re:Cause meet Effect. by smallfries · · Score: 2

      It does provide an interesting approach to training though. If we haven't produced a grandmaster for decades then we could simply increase the ratings of our national contenders by hundreds points so that they make fewer mistakes. It doesn't even matter if they are really that could, we could just keep doing it until they play like machines.

      --
      Slashdot: where don knuth is an idiot because he cant grasp the awesome power of php
    4. Re:Cause meet Effect. by Anonymous Coward · · Score: 1

      2p + blunders are relatively frequent in amateur games I think. As you said earlier it depends on what you call a mistake. You get into a really complicated position and start trading off pieces. You each get 4 pieces say of the others. But someone comes out a bishop ahead or whatever. But is it a mistake? To a master it would be but to people at your level of skill the sequence was too complicated to realize that you'd end up behind. Which leads too:

      maybe it isn't just not making mistakes that makes people get 600 more points. Maybe they notice the equivalent mistakes of their opponent: it isn't just the mistakes you don't make it is the mistakes you catch.

    5. Re:Cause meet Effect. by Rockoon · · Score: 2

      I think the big problem is that a "mistake" is being defined so crudely.

      ..especially since he is using a horrible chess engine to decide what a mistake is.

      Crafty, the engine he used is rated 2825
      Stockfish, (arguably) the best engine available (and its open source) is rated 3390

      ..for a difference if 3390 - 2825 = 565

      So by his own research numbers the engine he used for his blunder analysis blunders about twice as often as the free Stockfish which he could have used.

      ...garbage in, garbage out.

      --
      "His name was James Damore."
    6. Re:Cause meet Effect. by gnasher719 · · Score: 3, Interesting

      You don't need the worlds best chess engine to find blunders. A really superior chess engine might get 0.1 points advantage in every pair of moves, and after 30 moves you haven't got a chance. But we are talking here about blunders with 2.0 points disadvantage in one move.

    7. Re:Cause meet Effect. by itzly · · Score: 1

      The ultimate definition of "mistake" is a move that turns a winning position into a draw or loss, or a drawn position into a loss. If you have a winning position, with 3 winning moves, any of those 3 moves is good, and the rest are bad. A top player is more likely to pick one of the good moves, and thus make fewer mistakes overall.

    8. Re:Cause meet Effect. by PacoSuarez · · Score: 1

      Although ultimately you are right, there is a useful probabilistic approach to this besides considering the theoretical value of a position. Since chess is very far from being solved, you don't really know the theoretical value of very many positions. However, you can make models for the probability of the outcome from a position (a choice of an engine is such a model). A blunder is then a move that reduces the expected score for the player that made it, by some amount.

      And as others have said, he should have used Stockfish.

    9. Re:Cause meet Effect. by Rockoon · · Score: 2

      A really superior chess engine might get 0.1 points advantage in every pair of moves, and after 30 moves you haven't got a chance. But we are talking here about blunders with 2.0 points disadvantage in one move.

      I've spent years following the TCEC and watching Crafty get demolished again and again and again, and its never via the slow 0.1 grind that you are imagining. You are talking out your ass right now.

      Crafty regularly (*almost every game*) makes blunders giving what the top engines see as +3 pawns and higher advantage. It then often takes crafty several more moves to finally see that its lost, and then it regularly mis-evaluates that as even worse than it really is (because its failure to see goes both ways.)

      --
      "His name was James Damore."
    10. Re:Cause meet Effect. by tehlinux · · Score: 1

      For this study, blunders were defined as moves that lose the equivalent of 2 pawns or more of material. Any engine is sufficient.

      --
      Most linux users don't know this, but the man pages were named after Chuck Norris. Chuck Norris fsck'ing hates noobs!
    11. Re:Cause meet Effect. by itzly · · Score: 1

      Chess will likely never be solved, but it's possible to get more accurate results by using a better engine. Especially since Stockfish is open source, there's not really a good reason to go with Crafty.

    12. Re:Cause meet Effect. by gmiller123456 · · Score: 1

      Those ratings for those engines are mostly worthless. They only apply to the specific machine in a specific configuration that competed, and the computer was allowed to manage its own clock. Here, analysis was limited to two seconds per move, and likely on very low end hardware and memory compared to the machines that are rated. Just upgrading the hardware for Crafty would close the rating gap at least some. And you'd be very hard pressed to say ratings for full games played at normal time controls still translates that accurately to 2 secs per move without all that dedicated RAM, CPU, and storage.

  2. chess is a game with by invictusvoyd · · Score: 1

    A limited set of rules and a large set of possible moves. Somewhat similar to programming . That makes me wonder if a good chess player could potentially be a good programmer ?

    1. Re:chess is a game with by Anonymous Coward · · Score: 1

      That makes me wonder if a good chess player could potentially be a good programmer ?

      A good midwife could potentially be a good programmer, but so could a bad one.
      So yes, a good chess player could potentially be a good programmer, there is nothing that says that he can't.
      Heck, being able to think more than one step ahead pretty much guarantees that the person would be a good developer in general if you slap a couple of years of education and experience on top of it.

  3. Real Chess Players... by telchine · · Score: 1, Interesting

    Real chess players don't blunder... they make tactical bluffs - Just like professional poker players!

    1. Re:Real Chess Players... by invictusvoyd · · Score: 1

      Probably, a "potential" unreal chess player .. There is a difference ..

    2. Re:Real Chess Players... by ThatsMyNick · · Score: 2

      Gambits are not blunders (or even mistakes). I cant open Crafty's website right now, but I sure hope it can differentiate between them.

    3. Re:Real Chess Players... by Anonymous Coward · · Score: 1

      Sure. Crafty is measuring whether the apparent value of a player's position suddenly declines.

      If our analysis of chess was perfect, all player positions would have one of three values, "You can win from here", "You can force a draw from here" or "Too bad, surrender". But in practice Craft has a lot of grey area where it can see that e.g. having an extra pawn is good but it isn't sure whether it's good enough to win. This grey area value is what's being measured.

      So e.g. swapping a pawn for an improved position on one flank may make good sense. But if in fact your opponent can take the pawn without you shoring up that flank you just gave away a pawn, which is dumb.

      I think the grandparent post about poker players is joking. Sometimes a pro player does something extraordinary to throw off their opponents. But sometimes they do just screw up and everybody knows it, except, they hope, other players sat at the table who could (perhaps) believe it was a crazy bluff.

      For example, I've seen a pro player pushing hard, seemingly overjoyed to still be putting their chips into the pot even though the turn and river can't help a lot of hands. Their opponent is puzzled but likes their own hand too much to fold, so they stay in... and then both hands are revealed and the pro makes... A high and loses to whatever their opponent liked so much (maybe a pair of Kings or something that pairs twice on the flop). The pro gets up to look closer at the board - it's a rainbow of mid-value cards. They've been assuming they had a nut flush since the flop because they mistook a club for a spade! They just bet a lot of money on garbage because they misread a single card. THAT is not a bluff, that's a mistake, and a lot of professionals are smart enough to just tell the table what they did rather than pretend they bluffed so hard with air.

    4. Re:Real Chess Players... by Kjella · · Score: 3, Interesting

      That's different, in poker you bluff when the probabilities are such that your opponent will lose calling you in the long run. In chess you're creating an objectively worse but more complicated position in the hope that your opponent doesn't have the skills, time and preparation/experience to play it optimally. I rarely do it when we're even, but I've gotten better at doing it when I'm behind.

      I remember one game in particular where I'm an officer down (-3) so I sack a few pawns (-3 + -2 = -5) to make his king open for attack, but it managed to set up a Q+R combo that he couldn't see a better defense against than sacrificing his queen for my rook (-5 +9 -5 = -1) and with Q vs exposed king I was able to eat a few more pawns and eventually go on to win the game. I did a computer eval on it and at worst it claimed my position was -8 as after the pawn sacks I'd lose another officer by force.

      But I won. If I'd just play passively trying to make my -3 position not worse he'd probably quietly swap pieces one by one until it was a winning advantage. I know I sometimes create such traps against significantly lower rated opponents, I know the threat is possible to defend against but why not see if he recognizes it because if he doesn't it could be an easy win. To genuinely outplay someone where they don't make any significant blunders is certainly possible but easy wins count as much as the hard ones.

      --
      Live today, because you never know what tomorrow brings
    5. Re:Real Chess Players... by justthinkit · · Score: 1

      I remember one game in particular where I'm an officer down

      Ok, that was a new term to me, so I dusted off my Wiki and found it is a term for Bishop in just three languages/cultures, with the predominant one being Russian.

      --
      I come here for the love
    6. Re:Real Chess Players... by Kjella · · Score: 1

      Ok, that was a new term to me, so I dusted off my Wiki and found it is a term for Bishop in just three languages/cultures, with the predominant one being Russian.

      Actually it was a mistranslation from Norwegian where "offiser" = officer in English refers to any non-pawn piece, I don't recall if I was a knight or bishop down or I'd have said so. Apparently the correct way of saying that in English is to say I was a piece down according to meaning three here.

      --
      Live today, because you never know what tomorrow brings
    7. Re:Real Chess Players... by colinwb · · Score: 1

      "That's different, in poker you bluff when the probabilities are such that your opponent will lose calling you in the long run. In chess you're creating an objectively worse but more complicated position in the hope that your opponent doesn't have the skills, time and preparation/experience to play it optimally. I rarely do it when we're even, but I've gotten better at doing it when I'm behind."

      A good example of this at a very high level of skill is Geller - Korchnoi (Candidates Quarterfinal 1971) Sicilian Defense: Scheveningen. Classical Variation moves 22. Rg4 h5!?

  4. Blunder of the Century by DNS-and-BIND · · Score: 5, Interesting
    Deep Fritz 10 - Kramnik,V (2750). Man vs Machine Bonn, Germany, 27.11.2006. Kramnik played 34...Qe3?? The blunder of the century.

    Kramnik played the move 34...Qe3 calmly, stood up, picked up his cup and was about to leave the stage to go to his rest room. At least one audio commentator also noticed nothing, while Fritz operator Mathias Feist kept glancing from the board to the screen and back, hardly able to believe that he had input the correct move. Fritz was displaying mate in one, and when Mathias executed it on the board Kramnik briefly grasped his forehead, took a seat to sign the score sheet and left for the press conference, which he dutifully attended.

    In the post match press conference Vladimir Kramnik confirmed that he had not blundered out of exhaustion, and had been calculating very well right to the end. He had no real explanation for the oversight that happened right at the end.

    Kramnik in the press conference: "It was actually not only about the last move. I was calculating this line very long in advance, and then recalculating. It was very strange, some kind of blackout. I was feeling well, I was playing well, I think I was pretty much better. I calculated the line many, many times, rechecking myself. I already calculated this line when I played 29...Qa7, and after each move I was recalculating, again, and again, and finally I blundered mate in one. Actually it was the first time that it happened to me, and I cannot really find any explanation. I was not feeling tired, I think I was calculating well during the whole game... It's just very strange, I cannot explain it."

    Thus the question that everyone was asking remained unanswered: how can a player of Kramnik's caliber, a world champion who hovers around the Elo 2800 mark, overlook a mate in one move? Naturally there is no logical explanation â" we have to delve into the realm of pattern recognition and the psychology of human perception if we want to understand anything.

    The rest of the article turns the board around, looks at it from Kramnik's position, and tries to get into his head to see what he was thinking. Personally, I think it's what I call "sniffing your own butt" when you get so inside yourself, you stop thinking about the rest of the world. You then perform bizarre actions which seem quite reasonable to you. This happens in groups as well. It helps to explain things like how pro-worker governments of the 20th century murdered millions of workers. There's just nobody there to second-guess your thinking, and even if there was, they would be heavily punished for speaking out and contradicting you. This is where crowdsourcing shines.

    Unfortunately, Kramnik had somehow not registered the threat generated by the Fritz move 34.Nxf8. The white queen threatens mate in one on h7 (where it is protected by the knight). Black does nothing to neutralise this threat with his move 34...Qe3. And so, after he played it, Kramnik was immediately mated by the computer.

    But how could he not have seen the threat after 34.Nxf8. An explanation was proffered by a very experienced chess player and trainer, Alexander Roshal, who is also the editor of the Russian chess magazine "64".

    Alexander told us that the mating pattern that occurred during the game, with the white queen protected by a knight on f8 (as in the screen shot above), is extremely rare in chess. It is not one of the patterns that chess grandmasters automatically have in their repertoire. This was confirmed by a GM commentator in Bonn, who after Kramnik's move did not notice that it was a blunder and started discussing White's options â" but not the mate in one.

    --
    Shutting down free speech with violence isn't fighting fascism. It IS fascism!
    1. Re:Blunder of the Century by mynamestolen · · Score: 1

      Ta for interesting bit of chess history. I think you're way off the mark in your analysis of mass murdering regimes. The pattern recogntion theory seems much more likely.

      --
      work in progress
    2. Re:Blunder of the Century by hcs_$reboot · · Score: 1

      And this is what makes another big difference between a computer and a man, a computer would never make such an obvious mistake, even at level 1.0. And it's a relief: even at that top level, chess players do have emotions, still.

      --
      Slashdot, fix the reply notifications... You won't get away with it...
    3. Re:Blunder of the Century by DNS-and-BIND · · Score: 1

      It's pattern recognition as well. The regime, accustomed to dealing with threats, classifies the people as a threat and takes the same action they always take. It just doesn't occur to anyone what they're doing because they're in an ivory tower, much like Kramnitz alone against the computer. There's just no-one to second-guess, and if there were they'd be shushed.

      --
      Shutting down free speech with violence isn't fighting fascism. It IS fascism!
    4. Re:Blunder of the Century by MozeeToby · · Score: 3, Interesting

      In one of the Kasparov vs Deep Blue games, Deep Blue made a blunder due to a coding error (it didn't see any viable move but rather than resigning it simply picked a move at random). Kasparov was so convinced that the computer was confused by the play that he thought Deep Blue was looking 20+ moves ahead in the end game, a thought that terrified him. Kasparov won the game, but a lot of people say his confidence was so shaken by that one move that he played significantly differently the rest of the series.

    5. Re:Blunder of the Century by Scutter · · Score: 1

      When the Universe is summed up at the end of time, we'll learn that someone from the future had been sent back to avert the total annihilation of the entire human race and the best way to do it undetected was to force this one chess blunder. One tiny change at one tiny but pivotal point in history. It's the only explanation for why Kramnik can't explain it himself.

      --

      "Tell me doctor, with all of your defenses, are there any provisions for an attack by killer bees?"
    6. Re:Blunder of the Century by nelk · · Score: 1

      Deep Fritz 10 - Kramnik,V (2750). Man vs Machine Bonn, Germany, 27.11.2006. Kramnik played 34...Qe3?? The blunder of the century.

      Kramnik played the move 34...Qe3 calmly, stood up, picked up his cup and was about to leave the stage to go to his rest room. At least one audio commentator also noticed nothing, while Fritz operator Mathias Feist kept glancing from the board to the screen and back, hardly able to believe that he had input the correct move. Fritz was displaying mate in one, and when Mathias executed it on the board Kramnik briefly grasped his forehead, took a seat to sign the score sheet and left for the press conference, which he dutifully attended.

      In the post match press conference Vladimir Kramnik confirmed that he had not blundered out of exhaustion, and had been calculating very well right to the end. He had no real explanation for the oversight that happened right at the end.

      Kramnik in the press conference: "It was actually not only about the last move. I was calculating this line very long in advance, and then recalculating. It was very strange, some kind of blackout. I was feeling well, I was playing well, I think I was pretty much better. I calculated the line many, many times, rechecking myself. I already calculated this line when I played 29...Qa7, and after each move I was recalculating, again, and again, and finally I blundered mate in one. Actually it was the first time that it happened to me, and I cannot really find any explanation. I was not feeling tired, I think I was calculating well during the whole game... It's just very strange, I cannot explain it."

      Thus the question that everyone was asking remained unanswered: how can a player of Kramnik's caliber, a world champion who hovers around the Elo 2800 mark, overlook a mate in one move? Naturally there is no logical explanation â" we have to delve into the realm of pattern recognition and the psychology of human perception if we want to understand anything.

      The rest of the article turns the board around, looks at it from Kramnik's position, and tries to get into his head to see what he was thinking. Personally, I think it's what I call "sniffing your own butt" when you get so inside yourself, you stop thinking about the rest of the world. You then perform bizarre actions which seem quite reasonable to you. This happens in groups as well. It helps to explain things like how pro-worker governments of the 20th century murdered millions of workers. There's just nobody there to second-guess your thinking, and even if there was, they would be heavily punished for speaking out and contradicting you. This is where crowdsourcing shines.

      Unfortunately, Kramnik had somehow not registered the threat generated by the Fritz move 34.Nxf8. The white queen threatens mate in one on h7 (where it is protected by the knight). Black does nothing to neutralise this threat with his move 34...Qe3. And so, after he played it, Kramnik was immediately mated by the computer.

      But how could he not have seen the threat after 34.Nxf8. An explanation was proffered by a very experienced chess player and trainer, Alexander Roshal, who is also the editor of the Russian chess magazine "64".

      Alexander told us that the mating pattern that occurred during the game, with the white queen protected by a knight on f8 (as in the screen shot above), is extremely rare in chess. It is not one of the patterns that chess grandmasters automatically have in their repertoire. This was confirmed by a GM commentator in Bonn, who after Kramnik's move did not notice that it was a blunder and started discussing White's options â" but not the mate in one.

      Chess blindness strikes again!

      --
      No keyboard detected. Press F1 to continue.
  5. Horrible graph design by Gumpu · · Score: 1

    It is an interesting article, but the one who made the graph should might want to change jobs.

    The first graphs has four shades of blue blue and on top of that the areas are colored using gradients.

     

  6. The Logical Conclusion by Akratist · · Score: 2

    This makes a great deal of sense if you stop and think about chess as a mathematical problem of sorts. While it's not been "solved" in the same way that checkers has, there is obviously an optimal solution for chess, and with greater player expertise, the closer they are getting toward reaching that solution, even if it's perhaps not humanly possible to fully solve it. The less experienced player is somewhat like a person who doesn't correctly understand how to balance an equation or who habitually forgets to carry numbers when adding, or some other analogy. A more experienced player is a bit like your calculus professor in college who might have forgotten to put a sign on a number or something trivial that doesn't really affect the process of work (even though the class dipshit feels like pointing out the prof's "typo"). A grandmaster is so good that you don't even understand their mistake when you were told what the mistake was.

  7. "a public repository" by slashdice · · Score: 1

    Yeah, a public repository on github. What was that other place? Oh yeah, sourceforge. I think google delisted them for serving malware. Kind of funny how that used to be hot shit in the open source world. If that doesn't prove your standards are too low, I don't know what does.

    --
    Copyright (c) 1990 - 2014 Dice. All rights reserved. Use of this comment is subject to certain Terms and Conditions.
  8. Comment removed by account_deleted · · Score: 1

    Comment removed based on user account deletion

  9. Re:blundering upon a saving discovery by tonytally · · Score: 1

    I remember reading, some years ago, about a conversation between a distinguished scientific programmer and the head of NASA, shortly after the first moon landing. He commented that he was amazed that such a complex program as that on the Lunar Lander worked well. The head of NASA confided that, in fact, the sign of gravity in the code was in with the wrong sign, and it was only discovered by accident the day before the launch.

  10. Re:blundering upon a saving discovery by Nehmo · · Score: 1

    ... the first moon landing. ... the sign of gravity in the code was in with the wrong sign, and it was only discovered by accident the day before the launch.

    Do you have the details on this? It's a pretty good one if true.

    --
    (||) Nehmo (||)