Artificial Intelligence in Poker

Markian Hlynka writes "The University of Alberta's research into Poker AI is featured in this New York Times article. There is also detailed discussion of the game of Poker, and the 'new breed' of players who have honed their abilities online. See the U of A's poker project for more information."

Comments

Strip Poker Games

[beders]

If the AI is too good, we can't get the girls naked.

Mmm, EGA boobies...

Poker AI? riight...

[KDan]

Poker is not a card game, it's a people game (aka don't play the cards, play the people). It's all about bluffing and reading other people's bluffs. I'm baffled that people even bother playing poker on the internet. Even with webcams the game wouldn't be the same at all.

Daniel

Re:Poker AI? riight...

[AssFace]

reading the article is too much I see.

it references that and points out how much of "reading the player" is overhyped and easily faked out.

whereas the real information is in the trade at hand - the exchange of money. watching the bets and the amounts in them at varying spots in the game.

I have a few friends that have won online tournaments and they approach it from a very mathematical point of view. They do very well in person or over the net.
Using the "read" approach, unless the read is of the play on the table, is only going to work with people that aren't aware of the read and therefore not faking the attributes.

I personally prefer to look for the security holes in the online software :)
(There was a famous one in '96 or so where the system was using the random function built in - I think in Turbo Pascal IIRC - they had it exposed by posting their random code on the net to prove that they were being fair. A consulting firm then exploited that to show that they only needed to see one or two cards beyond what was in their card to then show what everyone else in the game was holding... there is much higher security in it all these days, and better/smarter programming).

Another firend in college found a site that had a hole, not in the security, but in the method at which they gave out tokens - as long as you kept playing, there was a reward of some number of tokens as an incentive to keep you playing.
He then ran some numbers and proved that with that, they were open to an exploit of the Martingale system. He ran it on them for a good amount of time and it failed - he basically proved that their code was cheating on the inside.
He called them on it and after a few heated e-mails, got all his money back and was banned from the site.
I could go on and on - but that is going off topic.

Re:Poker AI? riight...

[secret_squirrel_99]

Poker is not a card game, it's a people game (aka don't play the cards, play the people). It's all about bluffing and reading other people's bluffs. I'm baffled that people even bother playing poker on the internet

Except that people play a certain way and develop easily (especially for an AI) recognizable patterns. Those patterns are just as recognizable, perhaps even more so online where the number of hands played per hour is so much greater.

Players can get broken loosely (particularly for Texas Holdem, but also for other games) into a small group of profiles and their play patterned according to that.

While an academic study may be new, commercial software to do this has been available for years. In particular Turbo Texas Holdem from Wilson software does an outstanding job of simulating different types of players and play conditions

and if you really believe that people skills and not card or math skills are all that you need, I'd invite you to come to Atlantic City and sit in any of the games I regularly attend. We'd love to have you.

Re:Poker AI? riight...

[rkent]

Yes and no. For limit hold 'em, there is an (approximately) "correct" way to play based on pot odds, likelihood of drawing the hand you need, etc. Early rounds of no-limit tournies approximate this, with the occasional all-in raise to shake things up, but I'd have to agree with you that in later rounds against clearly competent players, no-limit is very much a social game.

Which leads to the larger issue: poker is a game of incomplete information; you don't KNOW what your opponent holds. You can make estimations based on past play and game conventions (eg, a bet from early position usually indicates AA, KK, or AK), but you don't know for sure, and this raises the possibility of deception.

The problem with that is, game theory generally models strategies to combat players who are playing (rationally) to win. Not all players play like this, or at least not apparently based on the strength of cards. I think most emulators are going to get screwed on bluffs.

But still, in lower-limit games, people are loose enough that bluffing doesn't really help (Lee Jones: "generally, you're going to have to show down with the best hand to win"), so a decent AI could at least maintain a winning margin, and so could an actual human who played tightly enough to take advantage of this. I don't know. You sure can't make zillions playing cards online, and it's definitely a while before the "deep blue" of poker.

Re:Poker AI? riight...

[drfireman]

You're partly right. No-limit hold'em is about things like bluffing. The kinds of poker most people play in public cardrooms (mostly limit hold'em and stud) are more about the cards, although understanding your opponents is critical to doing well at either one. Either way, online poker has a lot to offer. It's still about understanding your opponents, you just don't get as much information by way of physical behavior. There are tons of bad players online -- players who make fundamental mistakes (i.e., don't play their cards well) and who don't adjust to their opponents well (both in failing to take proper advantage and in playing easily exploited strategies). It's true that the game isn't the same at all. But many of the differences are positive, and like live poker, it's still a game of skill (a fact that's well obscured by short-term variance).

(Blatant plug: I'm a little biased, the new edition of my book ("Serious Poker," an introduction to the serious game) has a chapter on online poker. But I do believe online poker has a lot to offer, and the sites do offer poker for play money as well.)

If it's "Smart Poker"

[ike6116]

Then does the AI cheat?

Not always what you expect

[Kombat]

I played strip poker with this computer I met in university once. Things were going great until I popped off the cover and found a positively ancient motherboard.

Gross! It was like I'd just walked in on my Walkman(TM) while it was rewinding.

sign in front of a speak-easy:

[Flabby Boohoo]

Poker in the front, Liquor in the rear.

Addiction?

[Qzukk]

Now, he is able to play every night between 10 p.m. and 3 a.m. while his daughter sleeps in the next room.

If this was UT or Quake, this entire article would be about how he was destroying his life, and getting ready to go on a rampage.

But instead, its just a game of cards, and he's gambling with his family's money, but thats OK.

The Algorithm

[taped2thedesk]

while(handSucks)
{
screen.displayPopupAdForPorn(); // distract user
hand.throwWorstCardUnderTable();
hand.pullAceFromSleeve();
}

Of course, the bot doesn't cheat:

"""Q: Why are the bots such filthy rotten cheaters?!?!

A: Poki does not cheat. Poki connects to the online server just like any other player, and does not have access to any other player's private cards. The server's random number generator is sound (although not as sophisticated as most online servers). Any weird or suspicious outcomes are simply the result of luck . This is a normal part of poker. If you believe otherwise, you are more than welcome to play somewhere else.""" (from the FAQ)

I mean, come on - it's a normal part of poker :)

Re:Tell me...

[rkent]

You RTFA:

Peter Muller, a friend of Mr. Rao's who has played against the same bot, said the approximations in the game-theory model left a weakness and limited the bot's chances to do more than break even. Game-theory models usually assume that every player uses the best possible strategy, something that rarely if ever happens with humans.

"An optimal game theoretic strategy might ensure that you don't lose, but it won't be effective at exploiting an opponent's weaknesses," Mr. Muller said. "The best players learn how to exploit predictability, but don't do it often enough so that the opponents catch on."

In other words, it's easy to bluff a computer; you just play strongly and it'll assume you have a good hand and probably fold to you. Unless it's got a good hand, in which case you're screwed. Or if it has adoptive modelling that remembers how often you bluff, then you're REALLY screwed. Generally, though, it sounds like the Alberta AI just plays tightly, using "classes" of hands to avoid getting confused by the billions of possible hands, which does limit losses, but doesn't generally win big.

NYT WANTS TO STEAL YOUR THOUGHTS!

[Thud457]

Please, please, PLEASE! I cannot stress this enough, never, ever, for any reason, register to read a New York Times story! THEY WANT TO STEAL YOUR BRAIN!

If you register online to read a story, their spybots automatically pin down your location using an algorithm based on the well-know scientifical principal that YOUR COMPUTER IS TRANSMITTING AN IP ADDRESS! Using this "IP ADDRESS", they can scan MSN mapquest and find out where you live. Once they have that information, it is a simple matter to send a priority override to point the NSA mind-control satellites (when they're not otherwise busy zapping agriglyphs into English wheatfields) at your house to read your mind. Then they steal your precious intellectual property, which they license to SCO!

For non-robots, a simplified poker method

[RobertB-DC]

I have never been any good at poker... in high school, playing nickel-ante poker, I lost about $25 to just one of my friends. Typically, after about 15 minutes of play, everyone was playing with "my" money.

But recently, I spent some quality time with a hand-held poker game, and played the "hundreds or thousands" of games as described in the article. Not enough to become an expert, but I did come up with a technique to make my 100 credits last longer.

I hacked away as much complexity as I could. The heart of my method is to forget about the effect of getting two cards you need. The chances of getting two specific cards is something like 1/52 * 1/52 = 1/2704 -- too small to care about. So the entire method is about the next card.

Of course, I put it online: How To Lose Less At Video Poker. At the risk of slashdotting my own server, I'm curious if anyone can find any obvious flaws in the method.

I found this Java-based tutorial that purports to generate the "optimum payout" -- it often disagrees with me, presumably because it's trying for big payouts. My method doesn't promise profit, only smaller losses.

An important disclaimer: I've never used my method with any non-trivial amount of actual cash. Here in Texas, there are video poker machines in every Quickie Mart, but I just don't see the appeal. Now, if they would put in a Pac-Man machine...

Sigh....

[peterwayner]

I'm a big fan of fair use. I hate the DMCA. But behavior like this just makes me wonder. The free registration at the NYT is not that much of a pain. Sheesh. The newspaper world is being very cool, at least compared to the music and movie business. Let them make a few bucks on the ads so they can pay me.

How Computers can win at poker

[tpengster]

Poker is primarily an odds game, that is to say it is all math. There are three places where a decision has to be made. The first decision is, "Should I pay to see the next card?" This is called Drawing. The second decision is, "I have a decent hand, but my opponent raised me. Is he bluffing?". The third decision is, "Should I try to bluff?".

Odds come into play everywhere. When you are Drawing, you must have the correct odds or else you will lose money in the long run. That is to say, if you have a 1 in 4 chance of hitting your straight on the next card, you must have at least 3:1 odds to Draw. (The pot must have $3 for every $1 you pay). There is also the concept of "implied odds" - predicting how much will be in the pot at the end of the hand and not just at the present.

When deciding whether or not to bluff, you must know the odds of your bluff succeeding, and add that to the odds of you hitting your out on the next card. At that point the calculation becomes the same pot-odds calculation described above. This involves some reading of your opponent; you have to know how often he will call, and how often the bluff will be sucessful. Luckily, computers can be pretty good at modeling and seeing patterns, probably much better than humans. It seems that Neural nets and other well-developed AI techniques would be very good at modeling these behaviors and predicting future ones. Calling bluffs will require the same type of knowledge.

Some have asked how it's possible to read patterns on the internet. Some people don't really have patterns in their game, they just call everything. These people will lose because they put too much money in the pot, they don't have the odds for the bets and calls they're making. Mostly, decent players have patterns in how they bet, for example they will bet when they only have 4 out of 5 flush cards. (A Semi-bluff). Computers have an advantage here because they can introduce a random element that humans cannot reproduce.

The recent winner of the World Series of Poker, Chris Moneymaker, had never played in a live game until the WSOP, he had only played internet games. This probably gave him good fundamentals in reading people based on their bets, and good math fundamentals.

Some have also questioned the wiseness of playing internet poker, since it is just "gambling". Well I'll tell you a little secret, poker isn't really gambling, poker is a skill game and especially with so many bad players out there who think it is just luck, hoping they'll get lucky, it's easy to win money. That's why the same players consistently win thousands of dollars online. For more information on poker strategy and reviews of online casinos, see this site: PokerTips.org

It could be harder than one could imagine...

[Mensa Babe]

Some people I have spoken with few days ago about this research told me they are worried that if the intelligence is too high this AI could refuse to risk its money gambling... I am not quite sure if they were serious though.