Humans Are Still Better Than AI at StarCraft (technologyreview.com)

29-year-old professional StarCraft player Song Byung-gu won 4-0 in the world's first contest between AI systems and professional human players, writes MIT Technology Review. An anonymous reader quotes their report: One of the bots, dubbed "CherryPi," was developed by Facebook's AI research lab. The other bots came from Australia, Norway, and Korea. The contest took place at Sejong University in Seoul, Korea, which has hosted annual StarCraft AI competitions since 2010. Those previous events matched AI systems against each other (rather than against humans) and were organized, in part, by the Institute of Electrical and Electronics Engineers (IEEE), a U.S.-based engineering association.

Though it has not attracted as much global scrutiny as the March 2016 tournament between Alphabet's AlphaGo bot and a human Go champion, the recent Sejong competition is significant because the AI research community considers StarCraft a particularly difficult game for bots to master. Following AlphaGo's lopsided victory over Lee Sedol last year, and other AI achievements in chess and Atari video games, attention shifted to whether bots could also defeat humans in real-time games such as StarCraft... Executives at Alphabet's AI-focused division, DeepMind, have hinted that they are interested in organizing such a competition in the future.

The event wouldn't be much of a contest if it were held now. During the Sejong competition, Song, who ranks among the best StarCraft players globally, trounced all four bots involved in less than 27 minutes total. (The longest match lasted about 10 and a half minutes; the shortest, just four and a half.) That was true even though the bots were able to move much faster and control multiple tasks at the same time. At one point, the StarCraft bot developed in Norway was completing 19,000 actions per minute. Most professional StarCraft players can't make more than a few hundred moves a minute.

Comments

The tables have turned

[ShanghaiBill]

It used to be news when software beat humans at yet another game. Now it is news when we find a game that humans can still win.

Re:What's special about Starcraft?

[Lanthanide]

Here's some general thoughts on what makes SC difficult for an AI.

Starcraft has areas of focus called 'macro' and 'micro'. Macro is base building, selecting which tech tree to advance down, upgrading, building your economy. Micro is controlling small groups or individual units and their position on the map and how they engage with enemy units. For a very rough idea, alternative terms would be macro = strategy, micro = tactics.

Someone who is excellent at macro can be destroyed by someone who is lousy at macro and excellent at micro, and vice versa. The top players are excellent at both macro and micro.

Scouting what your opponent is doing is a bit part of both your macro and micro strategies but in different ways. If you see the enemy has built unit factory X, then you better counter it by building Y (macro). If it's in map location Z, then you better make sure you get your units to position A to head them off (micro).

There are 3 races in SC so 6 possible match-ups include the 'mirror' matchups. If both players are random, then each player must scout their enemy to initially learn their race. Then there are known build-orders, so if you scout your enemy at time 2 minutes, and see X and Y, then you can conclude they are (probably) using strategy A. But if you get that exact same set of information at time 4 minutes, then concluding they are using strategy A may be a mistake.

Humans know these build orders (like expert players memorising important chess gambits), but the AI likely doesn't, so has to brute force everything. Brute-forcing is possible in chess, and thought impossible in Go. So Starcraft is an extension in this same area.

Re:What's special about Starcraft?

[Njorthbiatr]

I've played StarCraft competitively (note: not professionally) and follow the professional scene (yes there's still a pro scene). There's a special depth to Starcraft that other games lack.

Most of the ranks below top level are all about macro/micro mechanics. Macro (economy, building units, ensuring supply cap doesn't ever get hit) is easy for AI to do. Micro is more challenging but still a task that's better for computers than humans. Top level players are already so good at both of these that there's really only diminishing returns left for computers to gain an edge over them in. They're even so good as to know exactly if they'll win a fight between each unit; almost like they're subconsciously calculating the end game of any engagement automatically. You can even see them know exactly how many attacks it will take. It's freakishly superhuman.

They also have developed some sort of insane intuition. You can watch them play against each other and move units to locations or build defenses in places only seconds before they need them despite having zero knowledge of the impending threat (such as building turrets right when there's about to be a drop on your resource line). There's instances where the tactics become so deep that they manipulate "thinking ahead" to double or triple bluffs to create openings. To beat these players, you need to have a deep understanding of human motivations. Classic tricks such as hold position lurkers spell doom for computer opponents who need to understand where their enemy might be laying a trap for them, /if they have gone lurkers/. For all you know they're using stop lurkers to deceive you into thinking lurker/ling/hydra is their strategy, while they're in position to wipe out your workers with muta micro.

It's simply not a game where you can calculate the odds and win every game, because you lack sufficient information to calculate any meaningful conclusions.

One thing constantly surprises me w/ these stories

[93 Escort Wagon]

For some reason I am always amazed to find out there are people who can make enough money to live on by playing video games.

And yes, I'm old.

That's only because Deepmind wasn't playing

[Sarusa]

Yes, human players can still beat second tier AIs from Facebook and universities.

But if you turned the AlphaGo Zero team on it it would dominate it in a couple months max. AGZ learned, from scratch, how to beat every human on the planet at Go every single time in 3 days.

Re:That's only because Deepmind wasn't playing

[guruevi]

Not really, Go and chess are full-knowledge games, you know at all points, where your opponent currently is, what his moves have been and where he can go. Not to say, there are only a very small numbers of paths you can take at any point in time.

StarCraft is a partial-knowledge game, you have to intuit where your opponent may be popping out what his intentions are based on very limited amounts of data and counter their strategy accordingly. There are also various ways of winning the game, you can starve your opponent, you can just go out and destroy him with superior force, annoy him continuously or simply execute a fast counterattack when their forces are away from the base in an offensive maneuver and most likely a combination of those things will win you the game. You can't just "guess" a solution because 90% of the times you will guess wrong, the game develops very differently based on tech trees your opponent chooses and choosing your own tech tree is a constant back and forth of trying to one-up your opponent.

This is really the worst situations for AI. There are no "common situations" as you have in chess or Go that you can just hard-code ideal responses to. AI's are still very poor at pattern recognition if the patterns aren't fully visible.

Woot!

[ma1wrbu5tr]

Go team "Smelly bags of mostly water!" Way to humiliate those dry silicates!