$100,000 Poker Bot Tournament

Costa Galanis writes "The LA Times is reporting that a poker tournament will be held where engineers will be able to pit their automatic poker-playing programs against each other in a tournament similar to the upcoming World Series of Poker main event, with a 100,000 dollar cash prize for the winning program. The article mentions how the recent rise in popularity of poker has encouraged many to try and create the poker equivalent of chess' Big Blue, the chess playing computer program that defeated the world's top chess player in a widely publicized event, and also talks about how many engineers also are trying to make bots that are good enough to play and beat human players for money in online casinos."

Comments

Bluffing.

[glrotate]

Any poker player will tell you bluffing is where it's at. Without bluffing you play the odds and it just becomes a simple game of chance. The bluffing algorithms are were the interesting work will be.

Re:Bluffing.

[kraada]

Granted, bluffing is definitely a good thing. That's why a really good poker bot absolutely must bluff. I presume these bots use game theoretical algorithms to decide when and how often to bluff.

If two bots were identical and one bluffed 3% of the time (and didn't bluff away all of his chips), in the long run the bluffing bot should win. Because the non-bluffing bot will believe the bluffing bot has a hand those extra 3% of hands, and thus the bluffing bot will in the long run win more than half of the hands and do better in the long run.

The interesting question is how often one should program the computer to bluff in what situations. . .

Re:Bluffing.

[wmajik]

Parent is somewhat right. Bluffing can be an integral part of the game, depending on the format of the game being played, along with the limit played (amount of cash wagered).

In a high stakes game or no-limit game, bluffing is very common, because every bet and action often involves a significant amount of money and little mistakes over the course of a session can end up costing large sums of money. Thus, bluffing becomes a viable weapon in these game formats because you can use your opponents' fear of making mistakes against him.

However, in a small stakes game, bluffing is often close to impossible, as many players are simply put, unbluffable. With the current poker boom, the skill level of the average player has decreased considerably; often causing poorly skilled players to play hands in a very losing fashion, such as showing Ace high at the showdown. Against these type of players, a bluff is generally quite ineffective and a losing proposition, since the theory behind bluffing is to force your opponent to fold a better hand. Thus, when your opponent simply does not fold, the point becomes moot.

As such, it would actually be easier to create a bot that plays low-stakes poker, as a non-bluff game involves simple math, decision trees and a bit of fuzzy logic. What it is not however, is a game of chance, as it is still a profitable game that has edges to be exploited.

This has been a bit off topic, but I wanted to clear up the notion that poker comes down to chance, when there is very solid mathematical theory behind it.

Re:Bluffing.

[cpeikert]

Without bluffing you play the odds and it just becomes a simple game of chance

Not true at all. To be a successful player, you must detect patterns in your opponents' play so you can infer whether you are likely to be ahead of them, and how to maximize the pot when you think you're going to win. This is very difficult for computers to do, even with sophisticated learning algorithms.

Re:What's The Catch...?

[bobbozzo]

The only thing the house gets in poker is table fees or tournament fees.

Re:What's The Catch...?

[Quill]

The only thing the house gets in poker is table fees or tournament fees.

And by that, you mean the house definitely wins every time.

Be easy to beat if its anything like...

[Zeussy]

Microsoft Poker on Windows 3.1. Just make a huge bet and all the computers Fold :P

1. Let openents Place you their bets.
2. Place a stupidly huge bet.
3. They fold.
4. Profit!

Deep Blue

[Anonymous Coward]

The chess machine is Deep Blue. It was created by IBM (AKA Big Blue).

Re:Eliza

[Anonymous Coward]

Please, tell me more about confusing the other bots into submission. How often do you confusing the other bots into submission?

Could this have been predicted?

[The Clockwork Troll]

At the end of an age marked by wealth-making products and services with style and no substance, is it that surprising that one of the most popular recreations has become seeing who can bullshit most deftly?

Perhaps the winning program could be reconfigured to create business plans?

Lots of problems tho

[nokilli]

Whenever my poker bot goes "all in", my mudd bot somehow gets the idea that it's time to start slaying all of the other players.

I think what is required here is clear and concise rules on what kind of weaponry the bots get to wield.

Also, I don't think bots should get to wear sunglasses.

Um... pokerbot will always win

[Capt'n Hector]

Over the long run that is. A pokerbot:

1. Has a perfect poker face,
2. Can count cards,
3. Can compute probability,
4. Has no emotions, so it won't get stressed or tired,
5. And will always make the right move probability-wise.

Re:Um... pokerbot will always win

[kraada]

Clearly you don't understand Texas Hold-'em. There is no card counting required, beyond to 6 (two in your hand, 4 on the table). Any player who is so bad that he can't read the board isn't going to be a challenge to anybody decent.

Since decks are made out of 52 cards, and you get two of them, it gets very easily to calculate probabilities for a human (mostly involves multiplying by 2).

Finally, making the probabalistic move every time will not do as well, because if you do that you would absolutely never bluff. A bot to be good in the long run must bluff, otherwise it is far too predictable and you can gain too much information from its bets and raises.

To give a quick example: If there's 100$ in the pot, and the bot bets 10$, I need to believe I'll win 1/11 times in order to justify my call. If I know that the bot never bluffs and only bets there when he's best, I can fold every time and save 10$. If the bot bluffs 1/11 times though, I suddenly have an actually complicated decision. And note if I fold those complicated decisions every time I lose more money, because he is betting more hands and I am folding each time he bets.

So no, straight up probabilities simply won't cut it.

(For more information, see Sklansky's Theory of Poker.)

Re:Bot Training

[Anonymous Coward]

Anyone who thinks the best poker stategies are dictated by statistics has no idea how to play poker. That won't make much of a poker playing strategy. The trick is representing stength and guessing what kind of hand your opponent has despite what they are representing.

Against real players the primary way of determining this is through the unconscious betting patterns almost every player has. Bots with some AI could do well at this. Bots against other bots is potentially an even more difficult problem.

What I fail to see is why anyone who had a well functioning bot would enter this kind of contest. There is far more money to be made without getting yourself the undue notoriety of this sort of success.

Re:What's The Catch...?

[wmajik]

The "catch" is that unlike house games, the casino is only in the business of taking a percentage of the winnings in poker, known as the rake. In poker, since you are playing against your peers, if you are able to achieve a winning percentage greater than the house rake (commonly around 5-10%), then you are still a profitable poker player.

In the poker world, the common standard for a profitable, solid player is to earn two big bets per hour, which covers both the casino rake and tip. In a $3/$6 texas hold'em limit game for example, the big bet is $6, which equals a $12/hr wage for a solid player. Online, where you not only do not have to pay a tip to the dealer, but also generally pay a lower rake and play about 150% more hands per hour than in a brick and mortar casino, it's very well possible to win nearly twice what you would by playing online.

Thus, the only "catch" here is that by creating a successful poker bot that can play as well as a solid human, it may very well upheave the online poker industry as a whole. After all, if you could spawn near unlimited instances of an application that could pull in a meager $2/hr playing the $0.50/$1 low limit tables, that still means an insane amount of money. Whether or not it's legal.. that's another issue.

What would a program want $100000 for?

[MooseGuy529]

a 100,000 dollar cash prize for the winning program

What would a computer program do with $100,000? Build a cluster to run itself on?

Re:What would a program want $100000 for?

[Daffy Duck]

I am now telling the computer exactly what it can do with a lifetime supply of chocolate.

Re:What would a program want $100000 for?

[Epistax]

What would a computer program do with $100,000?

two chicks at the same time

Re:What would a program want $100000 for?

[teneighty]

Wouldn't that be two chips at the same time?

This is incredibly difficult

[cpeikert]

Don't let the posters who say "just enumerate all possibilities" fool you.

The hardest part of playing poker is "reading" your opponents' hands -- learning how they tend to play, and inferring what cards they are likely to hold, whether they are bluffing or slow-playing, etc.

It may be easy to read a poker bot's style of play, but reading good human players is extremely difficult. So even if a certain bot crushes the competition in this tournament, it may not do so well against humans.

Re:overlords

[NanoGator]

"I for one welcome our new poker playing robot overlords..."

SHUUUUUUUUUUUUUUUUUUUUUUUUUUUUT UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUP!!!!!!"

Sorry, man. I promised myself I'd do that after I had heard that joke for the millionth time.

The keys are the algorithms...

[Londovir]

As someone else pointed out, the main key to how successful bots will become in poker is the algorithms that conclude if a hand is "bet worthy". Obviously probability is easy for a bot to calculate; my high school students calculate card hand odds in my statistics class for homework. You can go even further by calculating rudimentary risk-reward odds to determine if the potential cash payoff for this current pot, combined with the probability you have for making your hand, combined with the probability the flop gives your opponents better hands than your own, etc.

The key, clearly, is the way your program "behaves" in response to opponent betting. You could code a program that only plays based on the probability of achieving a winning hand in a statistical sense. (IE, if my pocket has a 75% chance of becoming a hand that will beat 65% of all possible hands, then play it regardless) That wouldn't obviously play that well, since the bot won't consider opponent betting. However, if your bot regards opponent betting, it will easily become susceptible to power bluffing if the algorithm doesn't guard against it. (Hence, you have routines like poorly written cell phone games where you just have to come out of the blocks betting like mad and you'll 90% of the time bluff the bot out of the hand)

I wouldn't be surprised if some of the more ingenious bots would be a medley of pure probability, observed opponent behavior (for trend matching with a fixed opponent), and a database of "real life" situations. If I were to design a bot for poker, and had the resources, I'd be sorely tempted to first host an online poker website and take a ton of samples from actual, online play. You have the advantage (right now, at least) of being able to record everyone's exact hands (at every stage of the hand) as well as everyone's betting. You could distill that into a form of database where you could try and match a bot's hand to a pre-existing condition case, and determine, along with your other ranking criteria, what a human player once did with that same hand, and whether that player won or not.

Londovir I could see bots taking over after awhile, but it's going to take some time...and even then, it should be entertaining to watch programmers trying to tweak their bot to beat another bot, sometimes without even knowing they are going up against another bot.

Hidden markov models

[G4from128k]

One approach to this is to assume that the other players are markov processes with unknown internal states (sorry for the PDF) . Gathering enough data (and probing the opponents with various betting strategies) helps estimate the internal patterns of the opponents. Humans are terrible at creating random patterns needed for perfect playing strategies. This approach can be used, for example, to create a hard-to-beat paper-rock-scissors game that quickly found the non-random patterns in human players.

Poker is Hard

[w1z7ard]

I did a little bit of work recently at UofA with the poker group.

Poker is a hard problem. The game tree is huge for even heads up limit (~ 10^18 leaf nodes). Ring games (3-10 players) are intractable via any game theoretic methods. The only feasible possibilities are searching parts of the game tree through intelligent sampling methods, and perhaps abstracting the game down a bit.

Work has focused on both solving abstracted versions of the game and exploiting opponent weaknesses. A publication concerning most recent methods involving bayesian best response will be available soon at the following link:

http://www.cs.ualberta.ca/~finnegan/publications/p ublications.html

Just in case any one was wondering, calculating your raw chances of winning, dubbed "7 card roll out strength" is no problem at all once you harness the versatility of the gnu poker-eval library located on sourceforge.

Poker bots ARE a real science

[wmajik]

There's been a lot of postings about this tournament being bunk due to a lot of misconceptions about the game of poker. As a successful poker player of quite a few years and also a geek, I do believe I have an informed opinion here when I say that A) poker is profitable B) poker bots can and have been created C) the effort to code a high level poker bot is incredibly, incredibly difficult.

A team at the University of Alberta has been working on with a poker research group that has been researching and coding poker bots for years. One look at their page should tell you that there is definitely some high level thinking and analysis required to develop a poker bot. More importantly, is that fact that they *have* delivered a bot called Poki Poker that has an impressive record at beating human opponents in 1 vs 1 heads-up matches. Brian Alspach, Professor Emeritus of Mathematics and Statistics at Simon Fraser University has also contributed numerous publications to the field, giving credence to the fact that there is a genuine science behind creating an AI that can play good poker.

So, before anyone else spouts off about poker being a game of chance or poker bots being mindless hundred line pieces of code, please do your research. A lot of people have worked very hard on this subject to simply have it dismissed as beneath them. Just ask yourself this: If you could create a poker bot so easily, one that could generate at the very least, a poker bot that made $2/hr playing the low limit games, what would stop you from launching thousands of these bots upon the online world? Because unlike a human, you can replicate a bot innumerable times, which in this case would be the equivalent of finding the goose that lays golden eggs. If you understand this, you may begin to understand why there is so much interest in the creation of poker bots..

Schedule of events...

[MagicDude]

The Tournament will begin at 9:00 AM, and the grand prize will be presented at 9:03 AM.

Re:Bot Training

[vslashg]

And likewise, any poker player who thinks poker is only about representing false strength and putting your opponent on a hand has no idea how to play poker. The real trick is a balanced approach. The best poker players are great at both reading hands and psychological warfare, but you had also better believe they know exactly what odds the pot is offering and whether finishing a draw is a positive or negative play.

If you disagree, you're more than welcome to join our weekly game.

Re:Heres hoping this doesnt ruin online poker

[bcrowell]

There was an article on this topic recently on kuro5hin, although it was focused more on human-human play, with the possibility of bots discussed as a possible reason why humans might think the game was rigged against them.

It may be that current bots can beat some of the worse human players, but it's not clear how many of the human players are that bad, and it's not clear how good the companies that run the servers are at detecting bot behavior.

One thing I'm still wondering about is human-human collusion. It's a big concern in breathe-the-same-air games between humans who don't know each other. Not sure about online poker, however -- do you get thrown in a table with randomly chosen players, none of whom you're likely to know? What about collusion between bots? E.g., you could be the only player at the table, not realizing you're playing against 6 bots, each of which knows what cards the others have.

here's a list of the things...

[isny]

Here's a list of the things that a poker playing robot needs to know:
1. When to hold 'em
2. When to fold 'em
3. When to walk away
4. When to run
5. Never count your money: When you're sitting at the table, there'll be time enough for counting, when the dealings done.

Re:overlords

[PakProtector]

Pssh. You're too young. Just wait till you're asked to take the oath for when you hear it for the billionth time.

AI AI CTHULHU FTHAGN!

Re:That's not the point of bluffing.

[spasmatik]

Please come sit at my table. Your expert strategy would be a wonder to watch.

Re:What's The Catch...?

[Illserve]

The hard part in cashing in from poker bots online (which must exist somewhere, writing a statistically playing bot that just waits for nuts isn't that hard, and lousy players will fork over money to this thing) is evading detection.

If you just simply spawn 1000 bots, as easy as that seems, you'll be detected easily, and your online assets siezed. At the very least you'd need 100 or more IP's, and probably some variance in reaction times, mouse movements, etc.

that's the difficult part, because online casinos have alot of money to lose if players get spooked by the fear of bots. So they'll be trying *hard* to detect you.