Humans Can Still Out-Bluff Machines

Pcol writes "The New York Times reports that in a poker game this week between man and machine, a program called Polaris fought a close match, but lost to two well-known professional poker players. Designing a poker playing algorithm is a different and more difficult challenge for software designers than chess and checkers because of uncertainties introduced by the hidden cards held by each player and difficult-to-quantify risk-taking behaviors such as bluffing. The game-tree approach doesn't work in poker because in many situations there is no one best move and a top-notch player adapts his play over time, exploiting his opponent's behavior. Polaris build a series of "bots" that have differing personalities or styles of play, ranging from aggressive to passive. Researchers monitored the performance of three bots and then moved them in and out of the lineup like football players."

Comments

Only expert players ....

[gstoddart]

I got the impression from some of the news stories that two professional poker players barely beat out the machine.

I have a sneaking suspicion that, for the vast majority of players, the computer is gonna kick your ass quite handily.

For the same reasons, I suspect that everyone who wasn't at the level of Kasparov would have gotten their asses handed to them in a game of chess against older versions of computers which couldn't yet beat him.

This, of course, begs the question of how long it will take for the on-line casinos to start putting poker playing bots into the mix to skew the odds even further to the house. I mean, if you have a computer program which will beat everyone else, why not just dial it down so it only wins 30% of the time or so and nobody will be any wiser.

Cheers

Re:Only expert players ....

[Animats]

For the same reasons, I suspect that everyone who wasn't at the level of Kasparov would have gotten their asses handed to them in a game of chess against older versions of computers which couldn't yet beat him.

Current reality is that any of the better chess programs for PCs can trounce you, unless you've been on the cover of Chess Life. Grandmasters are now playing Rybka with Rybka handicapped by one pawn, or with no opening book, and still losing fairly often. It's clear that computer chess performance has passed the human level.

And this is without supercomputers. Programs are playing at the grandmaster level on 2 and 4-CPU PCs. No need for custom "Deep Blue" hardware.

One of the workers on computer chess comments, after analyzing many grandmaster games, that about one grandmaster move in ten is suboptimal. This error rate is enough to give computers an edge humans can't match. A big problem in the chess world now is people cheating in the World Open using hidden links to computers.

Serious work on poker is only a few years old, and already the programs are doing well. Give it a few more years.

This is going to kill online poker played for real money.

Re:Not harder than chess

[Normal Dan]

Far harder than chess. In chess, every permutation has a low number of values (every turn has few possible moves). In poker, every turn has a large number of possible moves/choices, such as how much to bet. Also, in chess, you can see the other players hand, in poker you cannot. This adds to the complication of poker. If this weren't bad enough, bluffing adds a whole new set of problems. Also, in chess, given enough computing power, you can process all the moves up until the end of the game. This is not possible in poker.

Re:Not harder than chess

[Otter]

In poker you have a finite number of cards, that are a lot smaller than the permutation of moves in chess or checkers.

First of all, moderators, this is mistaken, not "Flamebait".

Second, you're correct that the cards are trivial to calculate. The betting process in poker is what's much more difficult to model.

Watching it occasionally on ESPN, I see people who are presumably good enough to be on television doing things that are completely insane. (Why the hell would anyone go all-in with unsuited 8-3?!?) It seems like the problem here might be the helplessness of artificial intelligence in the face of natural stupidity.

Environmental Sensors

[jshriverWVU]

Would be nifty if the bot's had access to environmental sensors like a camera so it could do facial recognition on the people to detect twitching, detect very little sweating, excess heat coming off body, things to interpret lying. Just an idea, and not *that* far fetched.

Re:Not harder than chess

[packeteer]

Your point is right but some of the things you say are a little off.

This is beyond simple mathematical calculations and hence it's MUCH harder than chess.
Chess is much more than just simple calculations. It is full of very complex calculations. Because of the incredibly huge number of moves possible it is more than just calculating. The best computer chess programs calculate less and use pattern recognition more. These days a computer with a fraction of the computing power of Deep Blue would beat it. That is because of the superior pattern recognition that is going on.

Card games are all about pattern recognition. You need to learn your opponents behavior, everyone plays in certain patterns and even when someone tries to change their own game they do it in their own personal way. Now card games are hard for a computer to win at right now because humans haven't been able to quantify all the elements of the game properly yet. Once computer SOFTWARE (pattern recognition algorithms) become stronger you will see computers beating everyone at card games as well as chess, which is already the case for chess.

That leads to the question of whether or not people will still gamble once they know a computer can do it better. I would bet casino's aren't going anywhere but online gambling is going top have to figure something out.

Re:Not harder than chess

[Kjella]

(Why the hell would anyone go all-in with unsuited 8-3?!?)

Because when you're bluffing, you don't bluff half-way so they'll call you on it (as opposed to when you're not bluffing and want to fish for more chips), and if someone with a top hand decides to call you on it, your medium hand will probably lose anyway. So if you're looking to make the others fold, unsuited 8-3 is as good a bluff as any other hand. Besides, if you're in a squeeze and hit the right company you can still win and get back in the fight as opposed to being bled to death.

Re:Not harder than chess

[xero314]

[poker is] not a strategy game, but a psychological exercise. Poker is indeed a strategy game. Knowing statistics and probability are critical to successful poker play. Psychology is important was well but is useless with out strategic knowledge. The majority of poker played in the world is limit poker which has far less psychological play and a lot more statistical accuracy. Even in a No Limit Hold'em game, probably the most psychological game regularly played, you would be better off having strong strategic and analytical skills an poor psychological skills than the other way around. But, like any game which contains aspect of chance, both strategy and psychology are imperative to being a successful player.

If you're playing cards in Hold'em, against decent players, you WILL lose. I would be happy to take on any player, no mater what there record is, as long as that player never looks at his hole cards. Cards are important in card games, even if betting is the determining factor in who ultimately takes a particular pot. Imagine the game being played with no cards what so ever and you will see why knowing how to work with the cards you have is important to the game. Any time a professional player makes a "call" it is because of statistical knowledge and not psychological, even if it is to set up a play later on.

Re:Not harder than chess

[Anonymous Coward]

Even in a No Limit Hold'em game, probably the most psychological game regularly played

This is often said, with no evidence, and usually by people who suck at poker.

I'd venture to say that pot-limit omaha high is a *far* more psychological game. The primary reason being that the math of NLHE means that it is simply impossible to play more than a relatively small percentage of your hands and win. Whereas the math behind PLO allows winning styles that vary from 15 to 50% hands played.

Any time a professional player makes a "call" it is because of statistical knowledge and not psychological

I don't think there's a difference between statistical knowledge and psychology. At least not when it comes to poker.

Re:RTFA.

[bogjobber]

You are also wrong. They chose the bots for the first two games, and the play was very close. They ran the 'coach' bot in the third game, and the computer got beaten "handily".

Re:Not harder than chess

[Aaron England]

Thinking that poker is only about statistics and luck is the hallmarks of an inexperienced player. The simple truth is, if you make your bets on the odds that you will win the pot against a professional poker player you will lose with 99.999% certainty. Because a professional poker player doesn't just play the odds, he plays you. He does this by lying about the strength of his hand through bluffing and discerning the times you attempt to bluff. Here's an example of how your stategy would play out. Let's say the following happens on the river.

You: Pair of 2's, check
Him: Ace-high, all-in

Now do you call or fold? You have the better hand here. If you knew what your opponent had you would definitely call. But since you are playing the odds, you decide to fold because you calculated you have a 30% of winning, which also means you have a 70% of losing. This is why playing the odds will cause you to lose. This is why it is the "psychological exercise" that the grandparent said it was.

Stacked / Poki

[Other Than That...]

If you want to play a video game against the predecessor to Polaris (named Poki), it's the AI used by the cross-platform poker game 'Stacked'.

Re:Limit Holdem

[venicebeach]

I'm not going to argue which is more exciting, since that is up to your personal preferences, although the evidence available to us is that NLHE has achieved a much larger audience than LHE.

About the computational issue you are actually incorrect here, and there is an objective fact of the matter. The reason most poker AI only attempts Heads Up Limit poker at this point is because there are more degrees of freedom in NLHE. At each decision point in a limit match the only options are bet, call, or fold. In no-limit, the decision tree is much larger because the "bet" option actually contains a number of sub-options (basically, how much to bet). Also as you add options for your opponent (and add opponents) the problem gets increasingly harder.

Sit-n-go's represent a special case of the problem, and there are computations that can help you deal with certain situations. The Independent Chip Model (ICM), for example, is a useful strategy but does not represent an actual computational solution to the game.

Re:Not harder than chess

[xero314]

Statistically, you NEVER lay down K's full of A's... If you actually think this is true you are using some bad statistics. I don't think that you do think it is true as you said yourself you would fold in the described situation. Even if you are taking all other information out of the equation statisiticaly K's full of A's (with 3 Ks on board) will lose approximately 1 in 11 times (opponent has 4 chances to have a king out of the 45 unseen cards). This means statistically you should lay down kings full of aces 1 out of every 11 times you play, but you would need to know which one of those times it is. I'm just saying that making statements like "Statistically, you NEVER lay down K's full of A's" is pretty much useless.