Slashdot Mirror
Poker Pros Win Against AI, But Experts Peg Match As Statistical Draw
hypnosec writes with some positive news for Skynet watchers, in that humans still have at least a slight lead against the AIs who might one day imprison us in energy-harvesting goo tanks, or at least beating us in Las Vegas. The two-day poker showdown involving four of the world's top (human) players and a Carnegie Mellon University AI program called Claudico saw the professionals win, after several days of heads-up no-limit Texas Hold'em. "Despite the win, the poker players' $732,713 collective lead over Claudico wasn't quite large enough to attain statistical significance, experts have said. This means that the results can't be accepted as scientifically reliable thereby indicating that the "Brains Vs. Artificial Intelligence" competition effectively ended in a statistical tie." On the other hand, the computers sure got over what looked like a rout by the humans.
Halfway through the competition, the four human pros had a cumulative lead of 626,892 chips. Though much could change in the week remaining, a lead of around 600,000 chips is considered statistically significant.
or
"Despite the win, the poker players' $732,713 collective lead over Claudico wasn't quite large enough to attain statistical significance"
Maybe it means a 600k chip lead per individual head-to-head. Or it could mean FUCK THS COMPUTARZ ROOL
I think this is the optimal outcome for the scientists. They can show: 1) We have done something (i.e. the poker bot is not too far from the best human players so our time has not been wasted). 2) There is more to be done (i.e. give us more money to look at this).
Also, I do think it is a quite impressive outcome.
The losing team of every competition likes to say things like 'statistical draw'.
Playing blackjack is applied statistics. Playing poker is AI.
Peter predicted that you would "deliberately forget" creation 2000 years ago...
Pretty much any algorithm can be considered Applied Statistics... and even our own existence if you want to go down that rabbit hole.
Personally, when I gamble and end up about 3/4 of a million dollars in the hole, I assume that I lost.
Great minds think alike; fools seldom differ.
Then can I have the $732,713?? I mean really, isn't this the very point of AI? To be less than just a statistical automaton and actually be able to beat a human at real life in something? AI should be about winning the money, that's it.
Laws are rules for the court, but merely a bottom bar to hit for life. Think beyond laws in your actions always.
So what you're saying is that it's AI if we call it AI.
In this case, yes, it is. All they gave it as inputs were the rules to the game. The AI had to make its own determinations after that for what the optimal strategy was, how it would go about achieving it, and how it should respond to the individual characteristics of each of its opponents.
Alternatively, ask yourself if playing a game takes intelligence at all. We could argue that all that the pros are doing is making their best estimation of the statistical likelihood that they'll win a hand, then betting accordingly along an optimal path that they've cultivated through experience. It's really no different than what this AI was doing either, it would seem.
Any AI problem that is solved, is no longer AI.
Personally, when I gamble and end up about 3/4 of a million dollars in the hole, I assume that I lost.
That sounds more like a conclusion than an assumption.
But the question isn't "Who won?" It is: "On the basis of this result what can we say about who will win next time?"
I don't know what kind of measures they used, and there are a couple of links in this discussion to papers pointing out how problematic p-values are, but it is perfectly possible for the weaker competitor to win any given competition. All it requires is that the width of the performance distributions be large enough to give significant overlap between the players.
People who don't understand statistics are baffled by this. They see individual instances, but statistics is about distributions. We can, by measuring instances, make judgements about the distributions they are drawn from, and knowing about the distributions we can make predictions about future instances.
In the present case, it appears that the observed distribution of performance was such that it wasn't possible to distinguish clearly between the case where the computer is slightly better than the humans but the humans got lucky, and the case where the humans are definitely better than the computer.
Blasphemy is a human right. Blasphemophobia kills.
Welcome to most poker. 75-85% of hands you fold at a full table without putting any money in. Another big chunk you fold to a bet either pre or post flop. Most of your money in any session is won or lost in a few big hands.
I still have more fans than freaks. WTF is wrong with you people?
There is a slightly better description here:
http://www.smh.com.au/world/man-bests-machine-in-twoweek-poker-tournament-20150509-ggxwuz.html
All they gave it as inputs were the rules to the game. The AI had to make its own determinations after that for what the optimal strategy was,...
Yes, that was where the original question was going (despite someone thinking it was trolling and modding me down).
When the domain is constrained by a set of rules and the programmers (not really "the AI") have built a decision tree on how to act based on current conditions, it makes me wonder what is meant by intelligence. Obviously it takes some intelligence for a person to be good at the game; but it also takes some intelligence to multiply two numbers together. I don't think anyone claims multiplication is AI.
A few years ago "Expert Systems" were all the rage. We used to joke that the defining characteristic of an expert system was whether or not it could execute "if" statements. So, to repeat the question, at what point does "if" statements and decision trees become Intelligence?
Alternatively, ask yourself if playing a game takes intelligence at all. We could argue that all that the pros are doing is making their best estimation of the statistical likelihood that they'll win a hand, then betting accordingly along an optimal path that they've cultivated through experience.
Perhaps that's true.
It's really no different than what this AI was doing either, it would seem.
Except that human players somehow manage to make their estimates through an advanced "higher-level" intuition, while TFA says: "Claudico [the AI] sets its own strategy, Brown noted, and that strategy occupies about two terabytes of data -- far more than the CMU team could analyze."
And here comes the problem with a single definition of "intelligence." If you believe (as I do) that "intelligence" fundamentally requires a level of adaptability, as well as abiilty to alter one's own behavior on the basis of new data, can this AI do anything like that? It seems it cannot, because TFA later talks about how the next steps will be to spend time analyzing the 80,000 hands of pro poker playing they have on file and "improving the algorithms" which the AI uses to generate its 2 TB strategy. This is a tremendous achievement that should not be underestimated, but ultimately we're still looking at stats from a "black box" and tweaking some of its inputs until its output stats are better.
Meanwhile, you ask whether playing a game takes "any intelligence at all." If it does not, then why are these very, very smart people taking months and years to create a program that can beat humans? The answer, to me, lies in the fundamental nature of true "intelligence," which requires levels of abstraction, concept formation, and higher-level adaptability that doesn't seem to be part of most AI algorithms in any meaningful way.
Everyone's always talking about how the "goalposts" are supposedly moved for AI all the time. I have the same goalposts that AI researchers declared back in the 1950s in terms of concept formation and adaptability.
When they have created an AI that plays poker at this high of a level and can actually TWEAK ITSELF to beat human champions consistently, THEN I'll begin to consider whether it's actually "intelligent." Give any of these human poker champions a perfect memory and the ability to create a 2 TB optimally-derived strategy that he/she could tweak however, and that human player would clearly blow the pants off of any other player in the world, as well as any current "AI." Yet somehow humans manage to do well without all of that perfect access and recall to huge quantities of information... and it's in that higher-level sorting and adaptability that the secret to true "intelligence" lies.
Or, get this AI to the point that it can beat any human poker players in the world, and then let it learn to play another card game on a high level without any human assistance or tweaking of algorithms. Then I'll believe it's closer to "intelligence." Or let this poker-playing AI figure out how to make me a decent sandwich BY ITSELF or with some basic instructions from me. Or whatever. The specific task is irrelevant, but one point of intelligence should be the ability to develop NEW SKILLS independently, rather than requiring human tinkering and tweaking with algorithms. It's great to have 2 TB of optimally-generated perfect strategy to win against humans, and it's a fascinating and awesome experiment -- and I COMPLETELY applaud the efforts of this team. But, to me, it doesn't qualify yet as anything close to "intelligence" just because the generated strategy happens to be too long for humans to spend time parsing.
the reason people can multi-table so much is because you rarely have to do anything. mostly you are just folding until you find hands or positions you want to play from, multi tabling reduces the boredom and stops you from playing in hands you shouldn't. It is not uncommon to go 10-20 hands in a row on a table where all you are doing is folding. multi tabling is not hard.
Yet somehow humans manage to do well without all of that perfect access and recall to huge quantities of information.
Humans also have access and recall to huge quantities of information. It may not be perfect, but in the case of professional poker players, it's close enough.
Look, it is two thousand freaking fifteen. This is an article from some site called "Techie News" being re-reported at Slashdot. Can we please get a little ridicule of this supposed binary concept of "statistical significance" ? It would take us one or two sentences to tell us the actual numbers involved--the expected value, expected deviation, margin of error, confidence level, etc.
And then when all's said and done, if indeed the level of significance was too low (e.g. p too high), maybe we could get a Bayesian or two in here to criticize the traditional 5% value is being arbitrary and tell us all a little about the Frequentist vs. Bayesian rivalry in statistics that persists to this day? (Obligatory XKCDs: https://xkcd.com/1132/ , https://xkcd.com/882/)
Obviously you lost, but the question is why. Was your expectation negative or did you get unlucky?
Comment removed based on user account deletion
the reason people can multi-table so much is because you rarely have to do anything. mostly you are just folding until you find hands or positions you want to play from, multi tabling reduces the boredom and stops you from playing in hands you shouldn't. It is not uncommon to go 10-20 hands in a row on a table where all you are doing is folding. multi tabling is not hard.
Yeah sure, you are right and multi-tabling works great if you are a level above your competition. Where multi-tabling falls down though is studying your opponents style and play. Necessary to give you an edge against players who are of a similar skill to you.
in online poker that isn't hard. There are a heap of programs that track the betting behaviours and stats of your opponents so you can know as much as possible without having to monitor the table constantly. online poker by its very nature is far more limited in the information you have and as such is not to difficult to track.
Awesome. I want to play poker with these guys... we can just draw right at the beginning, and they'll give me 700 grand, right?
I completely agree with everything you said in response to me. The only thing I feel a need to address is this:
Meanwhile, you ask whether playing a game takes "any intelligence at all." If it does not [...]
I was trying to call attention to a double-standard he had set up. I took it as a given that we would all agree that games take intelligence to be played, and that as a result we would recognize the invalidity in his assertion that this is nothing more than applied statistics, since that assertion could be applied equally as well to natural intelligence.
As you, I think that what we've seen on display here is impressive, but it falls short of what we'd typically think of as "intelligence", though that word has enough different meanings to it that we need to be careful how we're using it. We're still nowhere close to achieving strong AI, and I have serious doubts that we ever will. My Computer Science research advisor in grad school once tried to explain what he viewed as the impossibility of it by pointing out that we're incapable of ever fully conceiving the entirety of our own brains and holding that idea in our mind (i.e. holding onto all of the details of a brain would take more than one brain of storage), so the notion that we could ever replicate the intricacies of our own brains is nonsensical. I'm not sure I agree with him (after all, the burden to conceive of a brain can be shared among many people, each of whom only needs to know their part), and I'll admit that I'm likely not doing justice to his ideas since I'm probably explaining them poorly, but it was definitely an interesting idea to hear, particularly since he was a through-and-through atheist, and I had only ever heard such ideas espoused by religious folks up to that time.
http://poker.srv.ualberta.ca/