Slashdot Mirror


CMU Researchers Reveal How Their AI Beat The World's Top Poker Players (triblive.com)

2017 began with an AI named "Libratus" defeating four of the world's best poker players. Now the AI's creators reveal how exactly they did it. An anonymous reader quotes the Pittsburgh Tribune-Review: First, the AI made the game easier to understand. There are 10**161 potential outcomes in the game of poker -- that's a one followed by 161 zeros, potential outcomes in a game of poker. Libratus grouped similar hands, like a King-high flush and a Queen-high flush, and similar bet sizes to cut down that number. Libratus then created a detailed strategy for how it would play the early rounds of the game and a less-refined strategy for the final rounds. As the game nears the end, Libratus refined the second strategy based on how the game had gone.

A third strategy was at work as well. In real-time, Libratus created another model based on how its play stacked up against the play of the humans. If the humans did something unexpected to Libratus, the AI accounted for it and built it into the strategy. Instead of trying to exploit weaknesses in the play of the human, Libratus focused on improving its play.

The AI was created by a computer science professor at Carnegie Mellon University and his Ph.D. student, who argue in a new paper that "The techniques that we developed are largely domain independent and can thus be applied to other strategic imperfect-information interactions, including non-recreational applications."

"Due to the ubiquity of hidden information in real-world strategic interactions, we believe the paradigm introduced in Libratus will be critical to the future growth and widespread application of AI."

36 comments

  1. Cool by Anonymous Coward · · Score: 0

    Time to reload my poker stars account and go against this thing

  2. So, AI is just software? by thinkwaitfast · · Score: 1

    I knew it

    1. Re:So, AI is just software? by Anonymous Coward · · Score: 0

      Exactly. It isn't magic. Just a bunch of algorithms.

    2. Re:So, AI is just software? by sfcat · · Score: 1
      That was one of the worst ML "papers" I've ever seen. I'm not sure who this guy is, but the paper's didn't use the standard ML jargon for any of the techniques they used. If CMU wasn't attached (disclaimer, I have a CS degree from CMU), I might claim that this is probably a Mechanical Turk. The system seems to use a mix of game theory, alpha-beta pruning and RL techniques (specifically Actor-Critic) but none of the "correct" descriptions or references to those things are present in the article. I guess because its for Science and not a real academic journal, maybe they just didn't want to scare anyone but the paper used other complex terms in place of the ones real ML researchers would use. Also, the claims of general purpose use fall very flat due to the immense amount of domain logic explained in the paper. This is very sad as advances in RL have recently shown the ability to learn in generic domains without code modifications (which is the bogus claim the paper makes) but papers about those systems use the same terms as real ML papers.

      Also, this guy's title at CMU isn't that of a usual CS Prof. He is some kind of associated researcher but that seems to be attached to a different dept at a different school inside of CMU (as in not CMU's CS school but instead associated with CIT). This entire thing seems fishy...

      --
      "Those that start by burning books, will end by burning men."
    3. Re: So, AI is just software? by Anonymous Coward · · Score: 0

      Nice try, but the Libratus authors did an AMA in Reddit's r/machinelearning, and none of the people there (some of who are top researchers with Google and Facebook) had the same objections as you. If it was a "mechanical Turk" as you imagine, it would still be damn impressive, since where would you find a Turker able to beat the world's top heads up no limit Texas Hold'em pairs? With that good a margin?

      Nope, calling you out. It seems like a trend, this, to try to come off as an expert by shitting on actual experts in that particular style. Don't know what's the point of it, unless you are practicing misinforming people about science or something.

    4. Re:So, AI is just software? by Anonymous Coward · · Score: 0

      "but the paper's didn't use the standard ML jargon"

      Hm. If you write paper's, why did you write it correctly the first time? Or why didn't you write technique's, seem's, description's, reference's, thing's, gues's...

      Or why did you write "because its for Science"? There you DID need an apostrophe. But no capital letter for "science".

      " I have a CS degree from CMU"

      Oh, a software retard. Can't expect you to know details of a language.

  3. Fake News by Anonymous Coward · · Score: 0, Insightful

    I have it on good authority that President Trump is the best poker player and would kick this computers figurative behind. Internal polls have repeatedly said so but the progressive liberal lamestream media refuses to report this!

    #MAGA

    1. Re:Fake News by Anonymous Coward · · Score: 0

      zzzzzzzzzz

    2. Re:Fake News by Anne+Thwacks · · Score: 1
      The sooner governments are replaced by algorithms like this the better. Even pathetic AI beats Real Stupidity.

      No, Wait...

      Fake Stupidity may be harder to beat!

      --
      Sent from my ASR33 using ASCII
  4. Game tree analysis by Anonymous Coward · · Score: 0

    Sounds like class game tree analysis. Work out all future outcomes, then create a strategy that avoids the worst outcomes and selects the most likely paths instead. Because of the multi-player nature and randomness of the playing cards, the AI player will be forced down unfavorable paths.

    1. Re: Game tree analysis by Anonymous Coward · · Score: 0

      Nope, not game tree analysis. Multiplayer is a harder case in general, but not so much in this type of Poker they say, since in most cases most people should quickly fold reducing it to the two players thinking they might have the best hand.

  5. Well that's the end of online poker. by Mal-2 · · Score: 4, Interesting

    It was one thing when bots could beat up on donkeys, but when even the best human players can't win it means only bots will be left standing. That doesn't mean humans are totally out of the loop, someone still has to be standing by to talk to the admin when questioned about their human status -- for now. That too will probably fall before long.

    The micro-stakes tables will probably remain largely human because there's very little to lose (or gain) down there, but for high-stakes games this signals a rapidly approaching end.

    --
    How is the Riemann zeta function like Trump rallies? Both have an endless number of trivial zeros.
    1. Re:Well that's the end of online poker. by mentil · · Score: 1

      Anyone who plays high-stakes online poker can afford to physically go somewhere and do it in person. Of course, then people will have a bone-conduction speaker hidden somewhere on their person, receiving instructions from an AI. Until the hosts start doing bug sweeps.

      --
      Corruption is convincing someone that the selfless ideal is the same as their selfish ideal.
    2. Re:Well that's the end of online poker. by Anonymous Coward · · Score: 0

      I'd imagine high-level tournaments will be held in SCIFs before long

  6. Not comparable by nospam007 · · Score: 2

    It's way harder if you know you may end sleeping in your car for a couple of years if you lose.

    1. Re:Not comparable by belthize · · Score: 1

      No it isn't, it's exactly as hard. The second you start having thoughts about losing you're dead money. That's kind of the whole point of poker.

  7. The Dumbest Computer Can Beat the Smartest Person by rtb61 · · Score: 1

    It is kind of silly. Reality is the least capable computer can beat the smartest person all of the time, as long as it is not a race. The computer can keep going and going and going, long after the person gives up https://www.youtube.com/watch?....

    --
    Chaos - everything, everywhere, everywhen
  8. It cheated ... by 140Mandak262Jamuna · · Score: 2

    By counting cards. The bouncers will have it thrown out in a minute.

    --
    sed -e 's/Chuck Norris/Rajnikant/g' joke > fact
    1. Re:It cheated ... by Anonymous Coward · · Score: 0

      That's the stupidest thing I've ever seen modded up on Slashdot.

      You can't count cards in poker. In Texas Holdem you can only ever see a maximum of 7 cards face-up at any time before the showdown.

      Counting cards is most frequently done playing Blackjack, where it is possible in some circumstances to observe most of the deck being dealt out over several hands. A computer is very good at calculating whether or not the remaining deck favors the player or the dealer.

      Card counting Blackjack is now impossible at most casinos, due to such counter-measures as auto-shufflers, or the use of several decks which are reshuffled some way before the last cards are reached.

      There is no comparison between Blackjack and Texas Holdem, they are totally different games.

    2. Re:It cheated ... by DNS-and-BIND · · Score: 1

      Counting cards is in blackjack. Not enough cards are seen in Texas Hold'em to make a difference. The deck is reshuffled every hand, anyway, negating the advantages of card counting. Sad to see basic ignorance like this modded up.

      --
      Shutting down free speech with violence isn't fighting fascism. It IS fascism!
  9. So? by fluffernutter · · Score: 3, Insightful

    It kind of defeats the point of AI if you preload it with all kinds of statistics a human wouldn't have access to while playing a game. The point of AI is to make a computer think and learn like a human, not to prove that a computer can beat a human. We already know computers are better at calculations than humans.

    --
    Laws are rules for the court, but merely a bottom bar to hit for life. Think beyond laws in your actions always.
    1. Re:So? by hey! · · Score: 2

      The point of AI is to make a computer think and learn like a human, not to prove that a computer can beat a human.

      Well, that's one possible point of AI. Other possible goals would be to generate better results, or good-enough results but much more cheaply. Yes, there have always been people who have hoped to shed light on human intelligence by creating machine intelligence, but by in large they haven't had as much to show for their efforts as people trying to improve on people in some way (performance or cost).

      --
      Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
    2. Re:So? by mentil · · Score: 2

      You're kidding, right? Read any 'how to win at blackjack/poker/gambling' book and, right after the rules, it will give all of the precise odds for all of the means of winning. Amateur gamblers are expected to memorize ALL of these odds, this is Gambling 101. IIRC, it's allowed to bring a 'cheat sheet' of all of these odds into a casino, as well. In this case, humans know a bunch of statistical information regarding the game. That said, this AI did improve its strategy over time, which could be called 'intelligence'. It doesn't have to precisely mimic human learning in order to be 'intelligent', they weren't calling it 'Artificial Human Intelligence' AFAIK.

      --
      Corruption is convincing someone that the selfless ideal is the same as their selfish ideal.
    3. Re: So? by Anonymous Coward · · Score: 0

      The point of AI is to lead to a bitcoin-like bubble where a bunch of charlatans get rich from throngs of people throwing money after bullshit. The amount of regular algorithms I've seen passed off as AI lately is an astounding example of marketing bullshit.

      As to this, so what? The game of poker is largely psychological--reading your opponents, bluffing, etc. It's not a simple probability game. When you introduce technology to it you make it boring and predictable and basically not worth playing.

    4. Re:So? by Registered+Coward+v2 · · Score: 1

      It kind of defeats the point of AI if you preload it with all kinds of statistics a human wouldn't have access to while playing a game. The point of AI is to make a computer think and learn like a human, not to prove that a computer can beat a human. We already know computers are better at calculations than humans.

      In the context you are talking about I always though AI was the wrong term also since all it i really do is many more complex computations father than a human can but not really coining up with any cognitive thought. While that is useful as an adjunct to human problem solving it really isn't intelligence. Years ago when I was in high school I attendee series of classes taught by some researchers on pattern recognition; which is really what this is except with a lot more computational power.

      --
      I'm a consultant - I convert gibberish into cash-flow.
    5. Re:So? by Kjella · · Score: 2

      I feel it's been more the opposite, people claiming machines can never do this or that until they have human intelligence. And since we don't really know what that is or how to build it, we can't have it. And then we have AI - but since so many people seem to get their panties in a bunch about that term I'll just say adaptive algorithms - that do it better anyway. And despite it's encroaching on more and more human jobs we're clinging to our own unique abilities as important.

      If I'd like to eat a hamburger, is there any part of its creation I don't think can be done by a robot if I extrapolate? I think a robot can cook it. I think a robot can get the ingredients to cook it. I think robots can grow the crops and vegetables, grind the flour, bake the bread, raise the cattle and so on. I think a robot housekeeper can wash my clothes, iron my shirts, do the dishes, vacuum the floors, dust the shelves and so on. The building I live in, I think could be built by robots. The excavation, foundation work, carpentry, plumbing, wiring, HVAC, painting, floors and tiles and doors and whatnot.

      Pretty much every practical aspect of my life I think could be potentially created by or done by robots. And that possibly includes a sex bot to replicate the physical experience of having sex, it should certainly beat jerking off. So what does that leave for humans? Creativity, deep thought, comprehension, philosophy and emotional connection. Which sounds like pretty big outs, until you start looking at them in more detail. For example, creativity are often variations on a theme and just because it's unique to me doesn't mean it's totally unique. For example in a lifetime I'll eat 80*365 = ~30k dinners. There's probably way more than 30k recipes out there and there's no harm in eating a great meal twice. I don't actually need a chef to come up with new meals for me to have a new meal.

      Deep thought maybe, to the degree that you don't have algorithms smashing you on speed or parallelism like playing chess or Go. The computer does a thousand simulations refining its solution in the time you do one. Comprehension yes, like what do these business specs actually mean. But we're working on digital assistants that maybe one day will evolve into rapid prototyping that'll do that kind of bouncing back and forth until people get what they want. Philosophy yes, though you can probably fake it pretty well with a textbook since nobody has real answers. Emotional connection yes, if so many weren't happy with faux emotion. Like a sex bot should in theory fake it equally well as an escort/prostitute.

      Overall, it's getting pretty hard to see a niche that is like guaranteed forever "human" and where we're not just tweaking some kind of meta-meta-meta-parameters of what an algorithm is doing. Sure it's a long way from where we are now to there but the principal hurdles have turned out not to be so principled after all, mostly just practical. There's tons of people working on automating all these little niche tasks and put together they are automating pretty much all of society. Turns out what humans do is taking one extremely unique and flexible brain and using it to do pretty many not that complex, limited tasks that can be automated. Basically, our mind is overkill and "AI" is sufficient.

      --
      Live today, because you never know what tomorrow brings
    6. Re:So? by Anonymous Coward · · Score: 0

      It kind of defeats the point of AI if you preload it with all kinds of statistics a human wouldn't have access to while playing a game. The point of AI is to make a computer think and learn like a human, not to prove that a computer can beat a human. We already know computers are better at calculations than humans.

      Just...no. for a couple of reasons.

      First, humans had access to the same stats; if not, how did the AI get access to them? Magic?

      Second, AI is not about teaching a computer to think and learn like a human. Humans do not have a single succesful model of either cognition or learning; whatever is being transferred to the AI cannot be labeled that.

      What AIs do is make decisions based on incomplete knowledge. A guy named John Nash wrote a rather popular two page paper about it 68 years ago. Besides kickstarting AI research, it influenced economic and political theories that have had serious real world effects that we are still feeling today. Kissenger's insane MAD doctrine was the direct result of that paper.
      is heading.

    7. Re:So? by fluffernutter · · Score: 1

      >>> First, humans had access to the same stats; if not, how did the AI get access to them? Magic?
      Humans figured out the stats by hand and fed them to the machine, thus leaving the machine nothing to do but calculations. An AI machine should be expected to figure out these stays (and determine their usefulness) on its own.

      >>> Second, AI is not about teaching a computer to think and learn like a human.
      Well we already have AI then. Why are we still studying it?

      >>> What AIs do is make decisions based on incomplete knowledge.
      No, that's a challenging computer problem but not the commonly held opinion of what AI is.

      --
      Laws are rules for the court, but merely a bottom bar to hit for life. Think beyond laws in your actions always.
    8. Re:So? by fluffernutter · · Score: 1

      So if the stats are so easy to remember, why don't humans do as good as the computer? Because it is harder for a human to use them effectively than you are making it sound. But he ability for a computer to use them effectively is no surprise.

      --
      Laws are rules for the court, but merely a bottom bar to hit for life. Think beyond laws in your actions always.
    9. Re:So? by fluffernutter · · Score: 1

      So what is left for humans are the areas of the economy that very few people are willing to pay for. It's already fairly common knowledge that a general arts degree ain't going to cut it. Nothing about AI is going to make that a better career; if anything, since there will be greater concentration of wealth, less people will be able to afford the arts and the rise in ticket prices won't cut it. But I digress.
      Automated driving needs real AI to be truly as safe is being marketed. It's that simple. Sure some jobs can be done better with 'a better algorithm' but that's just an algorithm design issue, don't call it AI. Many human replacing things will need strong AI. You just hold the common belief that 'most' things can be done with these kinds of algorithms. I think you are drastically overlooking the intricacies of how complicated most problems are, it's the human thing to do.

      --
      Laws are rules for the court, but merely a bottom bar to hit for life. Think beyond laws in your actions always.
  10. counting cards in poker?? by Joe_Dragon · · Score: 1

    In poker the players will just kick you ass in the parking lot if you cheat them

    1. Re:counting cards in poker?? by Anonymous Coward · · Score: 0

      In the old West, sometimes it was a shoosting.

  11. Translation by Anonymous Coward · · Score: 0

    ... ubiquity of hidden information in real-world strategic interactions ...

    Translation: Computers can remember (or calculate) precise details that a human can't.

    This is the 1950s version of "androids in 20 years": If one stuffs enough details and scripts (for using them in the real world) into a machine, we will have an artificial life-form.

  12. Don't be afraid of AI's... by Anonymous Coward · · Score: 0

    A book that everyone should read is "Artificial Intelligence for Dummies" by John Paul Mueller and Luca Massaron. Preorder your copy today.

    1. Re:Don't be afraid of AI's... by Anonymous Coward · · Score: 0

      Chris I was hoping you'd learn your lesson ... Oh well.

  13. Only win in an ultra simplified tournament by Blue23 · · Score: 2

    I followed this as it was happening. This is NOT about bots being able to beat human players. It's about bots being able to beat human players in the simplest possible space that doesn't mimic 99% of actual poker play.

    It was only heads-up with 1 human a time, not vs. a table. After every round the money was reset so it never had to play from low amount of chips, or have to try to bully with it's chip advantage. The amount of chips vs. the big blind was a very large stack in the first place even before it reset every hand, so the blinds were statistically little more than noise in the amount that was going back and forth.

    Don't get me wrong, this is really interesting and great strides. But this is far from a bot being able to play at a full table and having to deal with a few bad hands taking it out of the place where it's betting is suited for. (If you have less of a stack, you have less of an upside so draw hands aren't worth as much.) Or to have someone with a larger stack push it beyond it's acceptable betting and make it fold because it can.

    --
    LITTLE GIRL: But which cookie will you eat FIRST? C. MONSTER: Me think you have misconception of cookie-eating process.