Slashdot Mirror
Can DeepMind's AI Really Beat Human Starcraft II Champions? (arstechnica.com)
Google acquired DeepMind for $500 million in 2014, and its AI programs later beat the world's best player in Go, as well as the top AI chess programs. But when its AlphaStar system beat two top Starcraft II players -- was it cheating?
Long-time Slashdot reader AmiMoJo quotes BoingBoing: It claimed the AI was limited to what human players can physically do, putting its achievement in the realm of strategic analysis rather than finger twitchery. But there's a problem: it was often tracked clicking with superhuman speed and efficiency.
Aleksi Pietikainen writes "It is deeply unsatisfying to have prominent members of this research project make claims of human-like mechanical limitations when the agent is very obviously breaking them and winning its games specifically because it is demonstrating superhuman execution."
"It wasn't an entirely fair fight," argues Ars Technica, noting the limitations DeepMind placed on its AI "seem to imply that AlphaStar could take 50 actions in a single second or 15 actions per second for three seconds." And in addition, "This API may allow the software to glean more information... " After playing back some of AlphaZero's back-to-back 5-0 victories over StarCraft pros, the company staged a final live match between AlphaStar and [top Starcraft II player Grzegorz "MaNa"] Komincz. This match used a new version of AlphaStar with an important new limitation: it was forced to use a camera view that tried to simulate the limitations of the human StarCraft interface. The new interface only allowed AlphaStar to see a small portion of the battlefield at once, and it could only issue orders to units that were in its current field of view....
We don't know exactly why Komincz won this game after losing the previous five. It doesn't seem like the limitation of the camera view directly explains AlphaStar's inability to respond effectively to the drop attack from the Warp Prism. But a reasonable conjecture is that the limitations of the camera view degraded AlphaStar's performance across the board, preventing it from producing units quite as effectively or managing its troops with quite the same deadly precision in the opening minutes.
Long-time Slashdot reader AmiMoJo quotes BoingBoing: It claimed the AI was limited to what human players can physically do, putting its achievement in the realm of strategic analysis rather than finger twitchery. But there's a problem: it was often tracked clicking with superhuman speed and efficiency.
Aleksi Pietikainen writes "It is deeply unsatisfying to have prominent members of this research project make claims of human-like mechanical limitations when the agent is very obviously breaking them and winning its games specifically because it is demonstrating superhuman execution."
"It wasn't an entirely fair fight," argues Ars Technica, noting the limitations DeepMind placed on its AI "seem to imply that AlphaStar could take 50 actions in a single second or 15 actions per second for three seconds." And in addition, "This API may allow the software to glean more information... " After playing back some of AlphaZero's back-to-back 5-0 victories over StarCraft pros, the company staged a final live match between AlphaStar and [top Starcraft II player Grzegorz "MaNa"] Komincz. This match used a new version of AlphaStar with an important new limitation: it was forced to use a camera view that tried to simulate the limitations of the human StarCraft interface. The new interface only allowed AlphaStar to see a small portion of the battlefield at once, and it could only issue orders to units that were in its current field of view....
We don't know exactly why Komincz won this game after losing the previous five. It doesn't seem like the limitation of the camera view directly explains AlphaStar's inability to respond effectively to the drop attack from the Warp Prism. But a reasonable conjecture is that the limitations of the camera view degraded AlphaStar's performance across the board, preventing it from producing units quite as effectively or managing its troops with quite the same deadly precision in the opening minutes.
We don't know exactly why Komincz won this game after losing the previous five
You could know if you'd watch the games. In the first set, DeepMind won with inhumanly superior micro. It was really cool, but computers have been better at micro for a long time. Speed and precision are things computers are good at, that's why we have aimbots.
In the second set, the human readjusted, and thought of strategies that would defend against the superior micro (by building more powerful units), while taking advantage of the computer's weaknesses (poor knowledge of army compositions, weak knowledge of positioning, and seemingly no object permanence: once enemy units are out of view, it has no idea where they are or if they exist).
"First they came for the slanderers and i said nothing."
bizarre time controls that removed stockfish's edge in time management
AlphaZero got the same time control.
stockfish didn't get its opening books
AlphaZero didn't get an opening book either.
nor did it get endgame tablebases
Neither did AlphaZero. Also note that in many of the games, Stockfish was basically lost in the early middlegame.
only 1GB when it should've had 64GB or more
That's the only legitimate concern, but the whole argument is stupid nitpicking nevertheless. This is like a race between a horse and the first model car. The exact conditions and outcome are secondary to the proof of validity of general principles. AlphaZero was just the first iteration of a new development. The fact that it came close to Stockfish at all is enough to show that this approach has merit. Sure, the SF setup was not optimal, but it wasn't completely crippled either.