Slashdot Mirror
OpenAI's Bots Defeated Former Pro E-Sports Players At Dota 2 (theverge.com)
On August 5th, OpenAI's bots defeated four former professional Dota 2 players in two of the three matches played. "There were a few conditions to make the game manageable for the AI, such as a narrower pool of 18 Dota heroes to choose from (instead of the full 100+) and item delivery couriers that are invincible," reports The Verge. "But those simplifications did little to detract from just how impressive an achievement [the] win was." From the report: The OpenAI Five triumphed in convincing fashion in the first game, not allowing the human players to even destroy one of their defensive towers. The humans recovered a little in game two, conquering one tower, but they still got demolished. And finally, in a game three played purely for pride, the humans managed to squeeze out a win. What stands out when you watch the matches is the apparent intelligence of the AI's decisions and the inhuman absence of any indecision. The typical Dota 2 game, even on the professional tier, involves quite a bit of equivocation about whether to engage in a fight, try and shift it to a more favorable battleground, or run away from it completely. The OpenAI team just doesn't need the processing time that humans require, which made its play appear unnatural -- but only in the speed and crispness of the decision-making, not in the content of those decisions.
Those are hardly professional players, that is a team of casters who never practice together. It is doubtful that any of them could get onto even a second tier team today (well, moon, but then again maybe not). Still an impressive benchmark for AI and top tier pros are sure to get beaten sooner or later too, but what is the point of misrepresenting what actually happened?
When all you have is a hammer, every problem starts to look like a thumb.
It should be noted that the bots are cheating and have a different level of access to the information in the game that players do not have. e.g. they have no need to select players to know inventories, precise health, mana etc. In addition to perfect possible information they also have perfect control input.
They also make a big point about how the draft is considered to be one of the more difficult parts of dota and how impressive it is their bots can draft... but restrict the pool from the normal 115 to 18 and eliminate bans rendering it entirely unlike dota drafting. They also restricted 2 items, no summons/illusions, 5 invulnerable couriers (instead of a team having 1, which can be killed) and no scanning (a time-limited radar like ability).
Don't get suckered into the news hype cycle until the bots use a screen with USB input and play the actual game instead of an arbitrary one.
For games 1 and 2 the AI got to draft it's own choice of 5 heroes out of the allowed 18 in an alternating picking sequence with the human opponents as is traditional. When each game started, the AI announced that it had a 95% confidence that it would win based solely on the heroes it had choosen against the human's choices.
For game 3 the audience picked 5 "random" heroes that had little to no synergy as a DotA team lineup and not good options for each of a DotA team's "roles" (pushing, ganking, tanking, supporting, etc). At the start of game 3 the AI announced that it calculated only a 2.9% chance that it would win against the human team with the heroes it was given.
Even with the severely gimped hero choices, it did an admirable job at trying to make it work by doing some unusual strategies. The professional casters even commented on how the AI was seemingly doing all the right things that it could have done given its poor position and that it actually did better in the match than anyone had expected it to do given the heroes that it was.