Slashdot Mirror
Humans Are Still Better Than AI at StarCraft (technologyreview.com)
29-year-old professional StarCraft player Song Byung-gu won 4-0 in the world's first contest between AI systems and professional human players, writes MIT Technology Review. An anonymous reader quotes their report:
One of the bots, dubbed "CherryPi," was developed by Facebook's AI research lab. The other bots came from Australia, Norway, and Korea. The contest took place at Sejong University in Seoul, Korea, which has hosted annual StarCraft AI competitions since 2010. Those previous events matched AI systems against each other (rather than against humans) and were organized, in part, by the Institute of Electrical and Electronics Engineers (IEEE), a U.S.-based engineering association.
Though it has not attracted as much global scrutiny as the March 2016 tournament between Alphabet's AlphaGo bot and a human Go champion, the recent Sejong competition is significant because the AI research community considers StarCraft a particularly difficult game for bots to master. Following AlphaGo's lopsided victory over Lee Sedol last year, and other AI achievements in chess and Atari video games, attention shifted to whether bots could also defeat humans in real-time games such as StarCraft... Executives at Alphabet's AI-focused division, DeepMind, have hinted that they are interested in organizing such a competition in the future.
The event wouldn't be much of a contest if it were held now. During the Sejong competition, Song, who ranks among the best StarCraft players globally, trounced all four bots involved in less than 27 minutes total. (The longest match lasted about 10 and a half minutes; the shortest, just four and a half.) That was true even though the bots were able to move much faster and control multiple tasks at the same time. At one point, the StarCraft bot developed in Norway was completing 19,000 actions per minute. Most professional StarCraft players can't make more than a few hundred moves a minute.
Though it has not attracted as much global scrutiny as the March 2016 tournament between Alphabet's AlphaGo bot and a human Go champion, the recent Sejong competition is significant because the AI research community considers StarCraft a particularly difficult game for bots to master. Following AlphaGo's lopsided victory over Lee Sedol last year, and other AI achievements in chess and Atari video games, attention shifted to whether bots could also defeat humans in real-time games such as StarCraft... Executives at Alphabet's AI-focused division, DeepMind, have hinted that they are interested in organizing such a competition in the future.
The event wouldn't be much of a contest if it were held now. During the Sejong competition, Song, who ranks among the best StarCraft players globally, trounced all four bots involved in less than 27 minutes total. (The longest match lasted about 10 and a half minutes; the shortest, just four and a half.) That was true even though the bots were able to move much faster and control multiple tasks at the same time. At one point, the StarCraft bot developed in Norway was completing 19,000 actions per minute. Most professional StarCraft players can't make more than a few hundred moves a minute.
It used to be news when software beat humans at yet another game. Now it is news when we find a game that humans can still win.
Probably the most popular RTS game right now. It's also pretty well balanced between playable races as I understand it.
#DeleteFacebook
Mostly because it was a regular game at the wcg for more then a decade. Having professional players, leagues and channels devoted to it for longer than that. And about 10 years ago, it got hacked so a software could read the user variables and move the units, so the bot developing began before the machine defeated human on go, this is just the next natural step.
Here's some general thoughts on what makes SC difficult for an AI.
Starcraft has areas of focus called 'macro' and 'micro'. Macro is base building, selecting which tech tree to advance down, upgrading, building your economy. Micro is controlling small groups or individual units and their position on the map and how they engage with enemy units. For a very rough idea, alternative terms would be macro = strategy, micro = tactics.
Someone who is excellent at macro can be destroyed by someone who is lousy at macro and excellent at micro, and vice versa. The top players are excellent at both macro and micro.
Scouting what your opponent is doing is a bit part of both your macro and micro strategies but in different ways. If you see the enemy has built unit factory X, then you better counter it by building Y (macro). If it's in map location Z, then you better make sure you get your units to position A to head them off (micro).
There are 3 races in SC so 6 possible match-ups include the 'mirror' matchups. If both players are random, then each player must scout their enemy to initially learn their race. Then there are known build-orders, so if you scout your enemy at time 2 minutes, and see X and Y, then you can conclude they are (probably) using strategy A. But if you get that exact same set of information at time 4 minutes, then concluding they are using strategy A may be a mistake.
Humans know these build orders (like expert players memorising important chess gambits), but the AI likely doesn't, so has to brute force everything. Brute-forcing is possible in chess, and thought impossible in Go. So Starcraft is an extension in this same area.
It's probably not just Starcraft. But since Starcraft is such a popular RTS, and is very mature as an esport, it makes a good metric for AI systems.
I'd bet the best computer bot would also get its ass kicked at Civilization by any competent player. And that's probably true of any other game with a complex enough ruleset which doesn't rely on sheer mechanical fitness (such as an FPS aimbot). Additionally, these are games with limited information about the state of the world. That is, your view is limited to your own units, and clouded by a fog-of-war system. In some sense, that's similar to poker, except that the world is too complex to take advantage of brute-force computational abilities in any way. So, you're left with the cleverness of your strategies, and computer AIs apparently aren't quite there yet.
Anyone who has ever played Civilization at higher than average difficulty settings knows the built-in AI cheats like crazy (or at least, as of a few versions ago - haven't played the newer versions). So, even the game developers apparently have a tough time programming an agent to play as competently as a human.
Irony: Agile development has too much intertia to be abandoned now.
The game actually measures 'Effective Actions per Minute' and disregards spam clicks.
A 'casual player' is considered to be about 50 apm. 'proficient' players are about 150.
Selecting a building, selecting a unit, and training a unit would be three actions. Selecting a unit and commanding it to move would be two actions.
At that point A.I. will completely own any non A.I. foe in star craft.
Because at that point properly configured A.I. will have been effectively playing for 200 years.
As with Go, it won't use existing strategies- humans will learn new strategies from playing against it.
And it won't cheat by playing 19,000 actions per turn.
She was like chocolate when she drank... semi-sweet at first and then increasingly bitter.
I've played StarCraft competitively (note: not professionally) and follow the professional scene (yes there's still a pro scene). There's a special depth to Starcraft that other games lack.
Most of the ranks below top level are all about macro/micro mechanics. Macro (economy, building units, ensuring supply cap doesn't ever get hit) is easy for AI to do. Micro is more challenging but still a task that's better for computers than humans. Top level players are already so good at both of these that there's really only diminishing returns left for computers to gain an edge over them in. They're even so good as to know exactly if they'll win a fight between each unit; almost like they're subconsciously calculating the end game of any engagement automatically. You can even see them know exactly how many attacks it will take. It's freakishly superhuman.
They also have developed some sort of insane intuition. You can watch them play against each other and move units to locations or build defenses in places only seconds before they need them despite having zero knowledge of the impending threat (such as building turrets right when there's about to be a drop on your resource line). There's instances where the tactics become so deep that they manipulate "thinking ahead" to double or triple bluffs to create openings. To beat these players, you need to have a deep understanding of human motivations. Classic tricks such as hold position lurkers spell doom for computer opponents who need to understand where their enemy might be laying a trap for them, /if they have gone lurkers/. For all you know they're using stop lurkers to deceive you into thinking lurker/ling/hydra is their strategy, while they're in position to wipe out your workers with muta micro.
It's simply not a game where you can calculate the odds and win every game, because you lack sufficient information to calculate any meaningful conclusions.
For some reason I am always amazed to find out there are people who can make enough money to live on by playing video games.
And yes, I'm old.
#DeleteChrome
Yes, human players can still beat second tier AIs from Facebook and universities.
But if you turned the AlphaGo Zero team on it it would dominate it in a couple months max. AGZ learned, from scratch, how to beat every human on the planet at Go every single time in 3 days.
Go team "Smelly bags of mostly water!" Way to humiliate those dry silicates!
Why can't we go back to using jumpers to configure slot adapter cards? Why? I say!
> I'm genuinely wondering, what makes Starcraft stand out?
Nothing. It's the other way around.
Traditional games like Chess and Go stand out by having massively fewer degrees of freedom than almost any video or even board game.
Not too diminish the advancements in AI, but games with relatively few choices each round are perfect for computers. In StarCraft (and in most video games), there are probably thousands of possible choices each frame.
Even if they got a Alpha Go like research budget, don't expect the Civilization 7 AI to start playing a game that's challenging to higher-level human play.
I’d take offense at your comment, but it’s time for my nap. So maybe later, if I remember.
#DeleteChrome