Slashdot Mirror
Artificial Intelligence in Poker
Markian Hlynka writes "The University of Alberta's research into Poker AI is featured in this New York Times article. There is also detailed discussion of the game of Poker, and the 'new breed' of players who have honed their abilities online. See the U of A's poker project for more information."
She'll probably want dinner first, though.
New York Times article reviewed. Gotta put an end to this.
If the AI is too good, we can't get the girls naked.
Mmm, EGA boobies...
Brocklesby Park Cricket Club
And when to use a Beowulf cluster to simulate every possible strategy.
Poker is not a card game, it's a people game (aka don't play the cards, play the people). It's all about bluffing and reading other people's bluffs. I'm baffled that people even bother playing poker on the internet. Even with webcams the game wouldn't be the same at all.
Daniel
Carpe Diem
I say we help him beta test not only his program, but also help him stress-test his web server.
Then does the AI cheat?
Are you secure enough in your masculinity to run 'man touch'?
For those who don't want to subscribe:
WHEN an accountant named Chris Moneymaker won $2.5 million in the World Series of Poker last May, the chatter in the poker world wasn't focused on his skillful bluffing, his tremendous luck or even the aptness of his surname. Everyone wanted to know how a man who had never before sat down at a tournament table could clean out so many skilled professionals.
While the Las Vegas hype machine focused on the rags-to-riches tale of a man who parlayed a $40 entrance fee into a huge pot, many poker players recognized that the amateur's success signaled the arrival of a new age in the game. Mr. Moneymaker may never have been in the same room as other players in a tournament of Texas Hold'em poker, but he had played extensively online, where the game is faster but the money is just as real. He was as much a rookie as Ichiro Suzuki, who joined the Seattle Mariners after nine years in the Japanese major leagues.
The online poker saloons that nurtured Mr. Moneymaker, 27, are just the beginning. Many players hone their craft with simulation software that allows them to test strategies by playing out thousands or even millions of hands. Some researchers are building software opponents that use sophisticated concepts from economics and artificial intelligence to seek out the best strategy, then use the knowledge to beat human players. The experience of playing thousands of games in roadhouses and casinos is being eclipsed by a cyborg-like intelligence produced by humans weaned on machine play.
The changes in the nature of the game are both subtle and striking. The advantages of some well-understood strategies are being tuned, and others are being abandoned. Some online enthusiasts, for instance, are even suggesting that the value of any information gleaned from watching the opponent's body for telltale tics or gestures is overrated. These so-called tells are too easily manipulated. More information comes in the pattern of bets, raises and calls. The money, they say, talks.
The biggest factor propelling change may be the speed of technology. Players do not wait while someone shuffles and deals. Chips do not need to be counted or watched. Computers handle the accounting, often finishing hands in as little as 30 seconds.
Steve Badger, the editor of the Web site playwinningpoker.com and winner of the 1999 World Series in a game called Omaha Hi-Lo, says that online poker halls are appealing because of their convenience.
"You could play them every day," he said. "You're able to play two games at the same time. Or you can sit and read or vacuum or do any infinite number of things while waiting for the next hand."
The online halls also offer substantially better rates. Most casinos pay for the lights and the dealer by subtracting either a fixed amount or a percentage from the pot. This levy, known as the rake, is often about $3 to $5 a hand in physical casinos, but about $1 or less online.
The rake depends on the stakes, which can be lower than those at physical casinos. Some online tables have minimum bets as low as 25 cents, an amount that makes learning the game cheaper. The speed of the game, however, ends up raising the amount at risk because 60 to 100 hands can be played in an hour. Higher minimum bets of $5, $10 or more are also common at tables with the better players.
Gautam Rao, a well-known Canadian player, said he stopped going to casinos in 2000, not long after his daughter was born, "because of the smoke and distance.''
"I told my wife I had to find a way to play online," he said. Now, he is able to play every night between 10 p.m. and 3 a.m. while his daughter sleeps in the next room.
"The rake is much less," he said. "The number of hands is much more. There are never any misdeals. There are never any issues related to tipping. The average cost of winning a pot is so much less. It's so much more efficient."
The speed of play lets players work through the thousands of apprentice hands faste
I am dyslexia of borg - your ass will be laminated.
Wouldn't you prefer a nice game of chess?
You are standing in an open field west of a white house, with a boarded front door. There is a small mailbox here.
I played strip poker with this computer I met in university once. Things were going great until I popped off the cover and found a positively ancient motherboard.
Gross! It was like I'd just walked in on my Walkman(TM) while it was rewinding.
Like woodworking? Build your own picture frames.
To create a poker AI you just have to figure out what the odds are of getting something after your first hand. Then based on odds the computer will decide which cards to get rid of and which to go for. This will typically result in the computer not going for the long shot royal flush and instead going for the safer full house which is more likely to happen.
A computer theoretically could be as good or even better than the average human at poker. It is able to calculate the odds of winning and is therefore able to make the best choice possible.
What would be really amazing would be if the computer was able to calculate based on how many cards other players turn in and adjust itself as neccesary.
Can you imagine having to try to look at the computer and imagine if it is bluffing? Talk about poker face...
is it included?
It's just a BloJJ
You can never learn the true art of poker unless you can read the other players tells, and unless the computer also throws in virtual signs I doubt cyber players could ever climb the ranks.
Poker in the front, Liquor in the rear.
You have to type in little poker face emoticons. :-/
KappaStone
Now, he is able to play every night between 10 p.m. and 3 a.m. while his daughter sleeps in the next room.
If this was UT or Quake, this entire article would be about how he was destroying his life, and getting ready to go on a rampage.
But instead, its just a game of cards, and he's gambling with his family's money, but thats OK.
If I have been able to see further than others, it is because I bought a pair of binoculars.
Now, they also say the machine has to be able to bluff, but the trick was to get it to do it the right amount, and at the right time postionally. Reading the opponent isn't as important as seeing the right situation in the cards.
Of course, the bot doesn't cheat:
"""Q: Why are the bots such filthy rotten cheaters?!?!
A: Poki does not cheat. Poki connects to the online server just like any other player, and does not have access to any other player's private cards. The server's random number generator is sound (although not as sophisticated as most online servers). Any weird or suspicious outcomes are simply the result of luck . This is a normal part of poker. If you believe otherwise, you are more than welcome to play somewhere else.""" (from the FAQ)
I mean, come on - it's a normal part of poker :)
You RTFA:
Peter Muller, a friend of Mr. Rao's who has played against the same bot, said the approximations in the game-theory model left a weakness and limited the bot's chances to do more than break even. Game-theory models usually assume that every player uses the best possible strategy, something that rarely if ever happens with humans.
"An optimal game theoretic strategy might ensure that you don't lose, but it won't be effective at exploiting an opponent's weaknesses," Mr. Muller said. "The best players learn how to exploit predictability, but don't do it often enough so that the opponents catch on."
In other words, it's easy to bluff a computer; you just play strongly and it'll assume you have a good hand and probably fold to you. Unless it's got a good hand, in which case you're screwed. Or if it has adoptive modelling that remembers how often you bluff, then you're REALLY screwed. Generally, though, it sounds like the Alberta AI just plays tightly, using "classes" of hands to avoid getting confused by the billions of possible hands, which does limit losses, but doesn't generally win big.
With an AI opponent you play so the opponent loses their clothes as fast as possible.
With a live opponent you play so you both lose your clothes at approximatly the same rate.
Boffoonery - downloadable Comedy Benefit for Bletchley Park
Limit Hold 'Em is a game of patterns. There are a lot of variables to take into account--position, pot odds, expected value, number of bets/raises already in, your cards, and many many more. You make playing decisions based on all of these variables. For example, in late position with few callers you can consider playing cards that you would fold in early position. Then there's bluffing and semi-bluffing.
Look at "Positively Fifth Street" by James McManus. He talks a little bit about the Alberta project and the rationale behind it.
If you register online to read a story, their spybots automatically pin down your location using an algorithm based on the well-know scientifical principal that YOUR COMPUTER IS TRANSMITTING AN IP ADDRESS! Using this "IP ADDRESS", they can scan MSN mapquest and find out where you live. Once they have that information, it is a simple matter to send a priority override to point the NSA mind-control satellites (when they're not otherwise busy zapping agriglyphs into English wheatfields) at your house to read your mind. Then they steal your precious intellectual property, which they license to SCO!
the preceding comment is my own and in no way reflects the opinion of the Joint Chiefs of Staff
I have never been any good at poker... in high school, playing nickel-ante poker, I lost about $25 to just one of my friends. Typically, after about 15 minutes of play, everyone was playing with "my" money.
But recently, I spent some quality time with a hand-held poker game, and played the "hundreds or thousands" of games as described in the article. Not enough to become an expert, but I did come up with a technique to make my 100 credits last longer.
I hacked away as much complexity as I could. The heart of my method is to forget about the effect of getting two cards you need. The chances of getting two specific cards is something like 1/52 * 1/52 = 1/2704 -- too small to care about. So the entire method is about the next card.
Of course, I put it online: How To Lose Less At Video Poker. At the risk of slashdotting my own server, I'm curious if anyone can find any obvious flaws in the method.
I found this Java-based tutorial that purports to generate the "optimum payout" -- it often disagrees with me, presumably because it's trying for big payouts. My method doesn't promise profit, only smaller losses.
An important disclaimer: I've never used my method with any non-trivial amount of actual cash. Here in Texas, there are video poker machines in every Quickie Mart, but I just don't see the appeal. Now, if they would put in a Pac-Man machine...
Stressed? Me? Of course not. Stress is what a rubber band feels before it breaks, silly.
"Artificial Intelligence" is common in home poker games-Its called BEER
The trick is to make sure your opponents are sufficiently fueled by "Artificial Inteligence" and you will come out way ahead
---"What did I say that sounded like 'Tell me about your day?'"---
Okay, I know this is a lil' off topic, but as one of the (presumably) many slashdot readers who play poker...has anyone found an online poker room that you can play for real $$$ where I don't need to download some windowsware to play?
I've toyed around with the java-based yahoo rooms (which last I checked, didn't have a real $$$ option). All of the big name poker rooms that I've seen through friends require a windows based client. I've been dying to give it a try.
I'd also be interested in anything anyone wants to post under this thread about poker room security. Are there many malicous online poker room opponents out there? I've seen a few cases where someone was about to sweep in a hold'em hand, holding the nuts(*), only to be booted off the game at the last second...any thoughts?
(*) Attn Trolls: This is an actual poker term.
While poker is an interest game to tackle, I think I'd have to agree with others here that it is more of a people game and hard to a machine to understand the nuances of "bluffing" and other things that we silly humans do.
What I'd really like to see is computer AI able of playing the Japanese/Chinese board game called Go at advanced levels.
As it is right now, the best Go AI is only at intermediate levels it terms of skill. Here is a URL comparing Chess to Go...
http://www.villagenet.com/users/bradleym/Compare.h tml
"To confine our attention to terrestrial matters would be to limit the human spirit." -Stephen Hawking
Bad enough I have to play with newbies at the blackjack tables, now I have to play with HAL at the poker table...
"Fold Dave..."
Why worry? Each of us is wearing an unlicensed "nucular" accelerator on his back.
Sig changed for readability by G.W.
I'm a big fan of fair use. I hate the DMCA. But behavior like this just makes me wonder. The free registration at the NYT is not that much of a pain. Sheesh. The newspaper world is being very cool, at least compared to the music and movie business. Let them make a few bucks on the ads so they can pay me.
Poker is primarily an odds game, that is to say it is all math. There are three places where a decision has to be made. The first decision is, "Should I pay to see the next card?" This is called Drawing. The second decision is, "I have a decent hand, but my opponent raised me. Is he bluffing?". The third decision is, "Should I try to bluff?".
Odds come into play everywhere. When you are Drawing, you must have the correct odds or else you will lose money in the long run. That is to say, if you have a 1 in 4 chance of hitting your straight on the next card, you must have at least 3:1 odds to Draw. (The pot must have $3 for every $1 you pay). There is also the concept of "implied odds" - predicting how much will be in the pot at the end of the hand and not just at the present.
When deciding whether or not to bluff, you must know the odds of your bluff succeeding, and add that to the odds of you hitting your out on the next card. At that point the calculation becomes the same pot-odds calculation described above. This involves some reading of your opponent; you have to know how often he will call, and how often the bluff will be sucessful. Luckily, computers can be pretty good at modeling and seeing patterns, probably much better than humans. It seems that Neural nets and other well-developed AI techniques would be very good at modeling these behaviors and predicting future ones. Calling bluffs will require the same type of knowledge.
Some have asked how it's possible to read patterns on the internet. Some people don't really have patterns in their game, they just call everything. These people will lose because they put too much money in the pot, they don't have the odds for the bets and calls they're making. Mostly, decent players have patterns in how they bet, for example they will bet when they only have 4 out of 5 flush cards. (A Semi-bluff). Computers have an advantage here because they can introduce a random element that humans cannot reproduce.
The recent winner of the World Series of Poker, Chris Moneymaker, had never played in a live game until the WSOP, he had only played internet games. This probably gave him good fundamentals in reading people based on their bets, and good math fundamentals.
Some have also questioned the wiseness of playing internet poker, since it is just "gambling". Well I'll tell you a little secret, poker isn't really gambling, poker is a skill game and especially with so many bad players out there who think it is just luck, hoping they'll get lucky, it's easy to win money. That's why the same players consistently win thousands of dollars online. For more information on poker strategy and reviews of online casinos, see this site: PokerTips.org
Some people I have spoken with few days ago about this research told me they are worried that if the intelligence is too high this AI could refuse to risk its money gambling... I am not quite sure if they were serious though.
Karma: Positive (probably because of superiour intellect)
What you described is the same reason I got out of playing yahoo poker. The model they were using at the time filled out a table with AI players.
Players would agree to play check-raise. check around the table (not bet) until an AI bet. Then they would progressivly raise until the AI folded. Annoying as all hell, no fun, and just made sure the twelve year olds got an ego boost from 'winning' a 50k pot.
A decent AI model for poker could be developed, but it lacks the fundamental strength of poker; the social interaction. Computers will play a great game of chess, mostly because the entire game can be predicted entirely off of what is showing the board. Poker, on the other hand, is all about the bluff. BS'ing the guy across from you. Statistics can tell you what he probably has. But the supposed "art" of reading a player can tell you nearly much.
This is sort of like the people that use cheat cards at a blackjack table. These cards have a color coded table that tell you how you should play a particular hand (i.e. the dealer has a 6 showing, you've got 13, Hit - stand - double down etc). It takes a way the human element of the game, which makes it fun in the first place.
A blackjack AI could be written by a novice programmer in a couple of hours, but still wouldn't do that much better than human.
There are some people that if they don't know, you can't tell 'em.
Invite several people over, buy a deck of cards and a pack of beer, and play. Just turn off the computer if you feel the need to code a quick AI in lisp. It will be there for you later, ready to greet you.
-Libertarian secular transhumanist
Poker has had quite a resurgence as of late, both due to online play and media coverage. Chris Moneymaker was on Letterman one night and the World Poker Tour can be caught on the Travel Channel and sometimes older ones on ESPN or ESPN2. It's actually fairly enjoyable to watch. Nothing like seeing a guy go all in with for 800 grand on a stone cold bluff whilst his opponent has a pair (you see the hole cards on the tv broadcasts, at least on the Travel Channel). Some talk smack, some wear sunglasses, some play conservative, though most are fairly aggressive. I've learned quite a bit by watching them play. As others have mentioned and as the article alludes to, the game is more mathematical than reading your opponents, but that's not really what I've seen from watching the pros on the WPT. It's all about the people, the chips, the cards, and the math all seem secondary to your read on your opponent and your style and reputation. It's all very interesting, I suggest you catch a broadcast.
This reminds me of strategies for programming Ro-Sham-Bo (Rock, Scissors, Paper). The "safest" strategy would be to randomly choose rock, scissors, or paper every time. Your winning percentage would approach 50%, but so would your opponent's. Ah, but if you're competing against other pairs of players, and they're all following that strategy, then it's just dumb luck who will win. For there to be any point to the competition, you have to assume your opponent has some non-random strategy, such that you could beat it and get >50% wins if you could figure out what it was. Of course, your opponent is making the same assumptions about you. And so begins a world of strategies on how to make your opponent think you're being predictable, when you're really just fooling him into making a choice you can predict. Of course, if your opponent knows you're fooling him, he will then know you're intention and gain the advantage. And so on and so on (similarities to the Iocaine Powder sequence from the Princess Bride are more than coincidence).
I just wanted to point that out as a counter to the posts advocating a purely statistical approach in which the program folds anything not likely to win. In the optimal case (there is no house rake, no ante, and no bluffing) it is as interesting as flipping a coin to see who wins. And even a small amount of bluffing will cause it to lose.
Well, there are a few things to consider.
1) It's a violation of the rules to do this on any of the online poker sites. If you're busted, you'll lose ALL of the money you've paid in. (buy in, registration, etc.) They might even press charges.
2) Perfect poker strategy is only perfect in a vacuum. When playing against real players, you can make more money by playing imperfectly. In fact, if you always play perfectly, the game will change to conpensate for you, and leave you playing poorly.
"People who do stupid things with hazardous materials often die." -- Jim Davidson on alt.folklore.urban
No, it's not a troll. :-)
But a few years ago, checkers was solved as a mathematical problem. There is a computer program that can play a perfect game of checkers, all the time. That project was headed by Jonathan Schaeffer, one of the people involved in this Poker AI project.
Just a footnote, to let you all know that this group has some serious history in gaming AI.
"People who do stupid things with hazardous materials often die." -- Jim Davidson on alt.folklore.urban
It's about low stakes Texas Hold'em. As I'm preaching to people who don't follow links, I'll explain that in hold'em, your hand is drawn from the best five out of your two personal ("pocket") cards, plus five common ("board") cards that everyone can see and use. You can even just play the five common cards if they're better than that 8 in your pocket. You tend to get strong hands, but then again so does everyone else. Hold'em is generally played with big tables, so chances are that someone has a strong hand each round. You don't get extended rounds of raising, and there are no huge wins to be made. Coming out on top of a night of hold'em involves long term risk management, not a single guts-or-glory Hollywood dramatic climax.
As for bluffing, go ahead and try. There are only four rounds of betting on each hand. Experienced players will fold early, so you won't get much of their money anyway, and excitable noobs will tend to stick it out and call you out with their regular full houses and flushes, making it expensive for you to try to bluff. You'll quickly find yourself playing to your hand, not to the other players, and you won't (indeed, can't) get yourself into a steely eyes, car-keys-in-the-pot ego clash.
I wish, I wish, oh how I wish people wouldn't predicate their discussions based on what they've learned from Mel Gibson movies.
If you were blocking sigs, you wouldn't have to read this.
The first ever complex program I wrote was, in fact, an attempt to make an intelligent poker player. It was written in BASIC for an IBM with an 8086 processor and about 7-feet long when printed out. It made ASCII representations of the cards and I had my own random number generator that used the time the program started as the "seed" value. It had an independent routine for "bluffing" and made it's more rational decisions based upon what I was to eventually learn was a pretty decent implementation of "fuzzy logic". ...problem is, it played lousy poker...and I could never figure out why. At least it followed the rules though.
A goal is a dream with a deadline
There was a large push made by some great mathematicians in coordination with poker proffesional to make a poker "robot" they called Loki. What they found was over a large period of time it could be profitable at low limit tables, playing limit hold'em. However, once the game changed to no-limit, there was simply no way for it to deal with the fluctuating betting amounts. It could pick up on patterns and beat many online limit players. I would be willing to bet that they could design a sophisticated enough program to beat even higher-limit limit hold'em games, but I'm guessing right now they weren't quite confident enough to trust it with a serious bankroll! A poker bot which could play in no-limit tournaments would be almost impossible to program, as anyone who has played in no-limit tourneys know, its very different from cash games. Sorry if anyone allready posted about this, I didnt have time to read everything...
You're playing against other people. If you did play "perfectly" you would never lose at poker, but it's impossible to play perfectly, because perfectly means that you would have to know whether other players that are after you would bet/call/fold/raise (thus you would always know the exact pot odds), which means that already with this perfect play you must analyze the individual player. With this, actually, the computer can do fairly well if it can see a huge amount of the previous hands that a person has played and thus they will be able to know the person's general strategy. But, then again, part of the game is changing how you play. If the computer, or another person, has you pegged with a specific playing style, and either the evaluation is inaccurate for whatever reason (players do get incomplete information) or the player varies his playing strategy, which good players do. But what I mean by all this is that, in order to play "perfectly", the computer would have to have an algorithm that defines the perfect play at each point in the game. Statistically, there can exist a perfect way of playing particular hands so that you will win money in the long-run. However, you probably have to integrate a certain amount of randomness into your own play so that you're not predictable (because, otherwise you're only playing hands where you have a good hand already or a good draw, thus everyone knows what you have by what you bet and will fold when you have a hand, and bet when you do not bet, thus causing you to lose money or at least not make money) and human behavior analysis of other people so that you can look at a person's behavior and their betting, correlate it with the person's past behavior, and use that to help you to decide what to do. This is a big task, with a lot of human so-called "intuition" and mathematical analysis based on a person's past performance.
The other part of this is that even if you're playing in a statistically perfect manner and it's working (probably because you're against bad players, or you switch tables enough), it's not necessarily the best way to play against you're opponents. Because it's statistically perfect, you might be making money from your opponents. However, with another strategy, one that's tailored to your opponents rather than tailored to the statistics of the card, you could devise a strategy that can net you a lot more money than the statistically perfect strategy. And, of course, you're also not varying your strategy, so someone can pick up on it totally and play so as to not give you any money, or someone might pick up on little things that lower the amount of money you make off of hands.
You can't play "perfectly" because you're playing against people, and you're playing against them in a game with, literally, gazillions of combinations, both in card possibilities, in the betting of a particular hand, and in how your opponents act.
"The price of freedom is eternal vigilance." - Thomas Jefferson