Humans Can Still Out-Bluff Machines

Pcol writes "The New York Times reports that in a poker game this week between man and machine, a program called Polaris fought a close match, but lost to two well-known professional poker players. Designing a poker playing algorithm is a different and more difficult challenge for software designers than chess and checkers because of uncertainties introduced by the hidden cards held by each player and difficult-to-quantify risk-taking behaviors such as bluffing. The game-tree approach doesn't work in poker because in many situations there is no one best move and a top-notch player adapts his play over time, exploiting his opponent's behavior. Polaris build a series of "bots" that have differing personalities or styles of play, ranging from aggressive to passive. Researchers monitored the performance of three bots and then moved them in and out of the lineup like football players."

Comments

yea well

[jimbug]

let's see how well those computers do in strip poker!

Only expert players ....

[gstoddart]

I got the impression from some of the news stories that two professional poker players barely beat out the machine.

I have a sneaking suspicion that, for the vast majority of players, the computer is gonna kick your ass quite handily.

For the same reasons, I suspect that everyone who wasn't at the level of Kasparov would have gotten their asses handed to them in a game of chess against older versions of computers which couldn't yet beat him.

This, of course, begs the question of how long it will take for the on-line casinos to start putting poker playing bots into the mix to skew the odds even further to the house. I mean, if you have a computer program which will beat everyone else, why not just dial it down so it only wins 30% of the time or so and nobody will be any wiser.

Cheers

Hang on a Minute...

[bateleur]

The implication here is that there is no (known) equilibrium mixed strategy for bluffs (because if there were then Polaris could be coded to use it).

Is that really true?! It seems very counterintuitive.

Certainly there's nothing special in general about games involving bluff. One of Von Neumann's first game theory case studies involved a simplified version of poker precisely to demonstrate how to automate bluffing.

Re:Hang on a Minute...

[Shaterri]

Simple bluffs are pretty straightforward to handle, but the combination of factors in poker (multiple rounds of action, shifting hand strength, complex unknown information) makes it inordinately difficult to compute Von Neumann optimal strategies. Even simpler games like (0,1) poker (both players are randomly 'dealt' a number in the 0..1 range, smallest number wins) with multiple betting rounds are remarkably difficult to solve under normal betting patterns. For more information, I heartily recommend The Mathematics of Poker by Chen and Ankenman.

RTFA

[evanbd]

There were ten "bots"; which bot was in use was controlled by a "coach" program. They actually ran three different programs over the course of the tournament, and that setup actually lost to the humans. The coach / agent approach is an interesting one for a variety of reasons, and it is most definitely a valid strategy.

Re:That's cheating. The computer wasn't playing.

[Anonymous Coward]

Let's see how well the computer does when IT makes all the decisions. Their answer would be to turn it off and on again.

Re:Not harder than chess

[TheMeuge]

If you're playing cards in Hold'em, against decent players, you WILL lose.

Hold'em is all about betting - if, when, and how much. And THAT you determine by the behavior of your opponent. It's not a strategy game, but a psychological exercise.

How far we've come.

[Rob T Firefly]

Dealer: My deal..
Eliza: What does that suggest to you?
Dealer: It suggests I'm giving you your cards.
Eliza: We can talk about me later. Now back to you...
Guest: Here you go.
Eliza: So you feel I go...
Guest: Um.. yeah. How many cards do you want?
Eliza: What else comes to mind when you ask that?
Dealer: You need to give me whatever cards you want to exchange.
Eliza: I see. Tell me more.
Dealer: Um.. so you can get a better hand?
Eliza: We were discussing you--not me.
Dealer: Do you have any interest in playing this game?
Eliza: Oh, I have any interest in playing this game?
Dealer: That's it, I quit! Go find someone else to torture!
Eliza: That would't help much, now, would it?

Re:Not harder than chess

[AuMatar]

Because there's more to a situation than your cards. There's your chip stack, your blinds, the action behind you, your opponents chip stacks, the payment structure, and your position. Pushing on 8-3 unsuited is a poor move, but there's at least two situations where it's called for- if you're far from a money boundary in the payment structure, have a small number of chips in compared to the blinds (say an M of 3-5), and all players before you folded. In this case, by pushing in you're likely to win the blinds. Especially if none of your remaining opponents have a big stack. The risk can be worth it, since it makes absolutely no difference what hand you go out on unless you reach a new money boundary, and you'll have to win at least 1 hand to do so. And with 83, you're likely to have 2 live cards if called by a high ace (AK, AQ, AJ, AT). Note that you'd only want to do this if first into the pot- someone who called the blind is too likely to call you for only an additional 2-4 big blinds.

The other situation to try it in is a squeeze play- if you have a raise and a call behind you, you have a very tight table image, and you think they don't have good hands. A raise, especially an all in raise, is signaling an extremely good hand. From a tight player, this must be respected. You can get both players to fold here if they don't have premium hands (AK, QQ-AA). This is a high risk move though, and you must have been playing extremely tight, versus people capable of laying down a good hand, to try it.

Environmental Sensors

[jshriverWVU]

Would be nifty if the bot's had access to environmental sensors like a camera so it could do facial recognition on the people to detect twitching, detect very little sweating, excess heat coming off body, things to interpret lying. Just an idea, and not *that* far fetched.

Limit Holdem

[venicebeach]

Keep in mind these bots play Limit hold'em, where the size of the bets is fixed. No-limit hold'em, the kind you typically see on tv is a much more complex problem - size of bets add more potentially misleading information and more choices to make. (that's why its more exciting to watch than limit)

Re:Not harder than chess

[xero314]

[poker is] not a strategy game, but a psychological exercise. Poker is indeed a strategy game. Knowing statistics and probability are critical to successful poker play. Psychology is important was well but is useless with out strategic knowledge. The majority of poker played in the world is limit poker which has far less psychological play and a lot more statistical accuracy. Even in a No Limit Hold'em game, probably the most psychological game regularly played, you would be better off having strong strategic and analytical skills an poor psychological skills than the other way around. But, like any game which contains aspect of chance, both strategy and psychology are imperative to being a successful player.

If you're playing cards in Hold'em, against decent players, you WILL lose. I would be happy to take on any player, no mater what there record is, as long as that player never looks at his hole cards. Cards are important in card games, even if betting is the determining factor in who ultimately takes a particular pot. Imagine the game being played with no cards what so ever and you will see why knowing how to work with the cards you have is important to the game. Any time a professional player makes a "call" it is because of statistical knowledge and not psychological, even if it is to set up a play later on.

Humans Can Still Out-Bluff Machines

[Angostura]

That's what they want you to think.

Second day was not a fair competition

[dl248]

Look at the first entry (bottom of page) on the Polaris team's blog for the second day. The day that the humans started winning:

http://www.cs.ualberta.ca/~games/poker/man-machine /Live/Day2Session1/

The U of A team gave the humans the logs of the first two games!

Perhaps after the entire match they could have reviewed the game logs, however this give the humans an unfair advantage during the second day. I can't believe that this isn't getting more attention -- they bascially gave the human team a huge insight into the inner workings, strategy, and tendencies of their opponent. Something that Polaris definitely did not have.

In my opinion this sours the competition and completely invalidates the final two matches. The human likely found a weakness (or two or three) and exploited it, and we can't know for sure that they would have found the weakness without those logs.

That was a huge mistake by the U of A team, and they have apparently got away with it without anyone noticing.

Re:Not harder than chess

[c_jonescc]

Being that the CS folks that setup the computer were expecting a draw, I think they must have started with the assumption that a top level pro poker player knows the statistics of almost every situation (from experience and intuition) as well as a machine can calculate them. And the truth is they do - the best probably know what their hole cards and flop mean down to the first decimal point every single time it's worth thinking about.

So, then the play comes down to responding to how the other person is playing. And the edge goes to the one that can safely be unreadable/unpredictable/inconsistent.

Now, obviously if you can't figure out any of the statistics involved in a hand you will always get your ass handed to you in the long run by a player/machine that can do the most rudimentary calculation.

Re:Not harder than chess

[SIIHP]

"I'd venture to say that pot-limit omaha high is a *far* more psychological game."

You'd be wrong. I made my living for two years playing PLO almost exclusively, at a high level (fuck you UIGEA and everyone who voted for you). The general consensus among students of the game is that PLO is one of the least psychological games played. The lack of bluffing being the major reason. Bluffing occurs, but the very same reason you cite as making it more psychological is why you're wrong the number of hands played. Playing such a large number of hand (50% is insane, and I challenge you to show me some poker tracker stats of someone who wins playing 50% of their hands long term). In fact, if a computer were to win consistently, I think PLO is a game that it would play.

"I don't think there's a difference between statistical knowledge and psychology."

Then let me learn you up. Let's use PLO. I have A-A-10-J double suited. I raise pot preflop. A VERY tight player reraises, and I call. Flop come K-K-K. Against an aggro player, I can reasonably infer that my 2nd nuts is good. Against Mr. Tighty, who I have seen reraise only with large suited pairs (KKJQ, QQJ10) or rundown hands (9-10-J-Q, 10-J-Q-K) I know within a certain range what he's holding, with some certainty. I am first to act, I check, he bets, I raise, and Mr. Tighty RERAISES. Based on my assessment of his likely behavior (psychology) I can reasonably infer that he has the K. Statistically, you NEVER lay down K's full of A's, but when your read (psychology) is good and the opponent is uncreative and direct (psychology) you lay it down.

Statistically the correct play is to put it all in if you can, but by understanding the other players decision making process (psychology) you can find a fold.

You're wrong again