How a Computer Game Is Reinventing the Science of Expertise

An anonymous reader writes "Cognitive scientists at Simon Fraser University and UCSD are beginning to use StarCraft 2 replays to study the development of expertise and the cognitive mechanisms of multitasking. Unlike similar expertise studies in chess that consider roughly a dozen players, these studies include thousands of players of all skill levels — providing an unprecedented amount of data on how players move from 'chumps to champions.'"

Comments

any other studies?

[wierd_w]

Like a study on the mass exodus of players doing ladder play after the koreans find the tournament?

Development of Shortcuts

[Anonymous Coward]

Ninety percent of the players in this tournament are not superhuman multitaskers. After watching enough of their first-person video streams, you see that most players can't react to novel situations. They just learn "build an army before the X minute mark", or "counterattack with fast ground units when his army moves out," executing the same limited skill set game after game. What this study will probably show is the rapid development of mechanical skills from low-to-mid level play, followed by the gradual acquisition of timings and strategic instincts.

Re:Development of Shortcuts

[wisnoskij]

Well of course a novel play would slow them down. The reason they are so fast is because they no longer thing about the normal stuff, it is ingrained like a champion chess player. But that does not mean that they are unable to cope with the never seen before.

Re:Development of Shortcuts

[khallow]

And that would be good to know. It's worth noting here that the basic learning curve model is that in a doubling of the time that someone does an activity, then their efficient gets a certain fraction better. This was used to describe improvements in manufacture.

For example, one such rule of thumb is that doubling the number of widgets produced in a manufacturing process, results in a 5-15% reduction in the marginal cost per widget. I gather this was used like Moore's Law as a sort of self-fulfilling prediction. If the process didn't improve by the desired amount then you spent more to achieve the target improvement (or you lowered expectations by reducing the rate of improvement).

But such theory doesn't take into account qualitative differences. As you imply in your example above, in Starcraft, there's three crude, abstract regimes of play. Logistics is the management of production and resource harvesting, tactics is short term movement and use of military resources particularly in fights, while strategy is the long term planning of the game. You might also make a four category of physiological/mental adaptations to the peculiar Starcraft environment such as image recognition, muscle memory, acting while under stress, adaptive behaviors (such as scanning), etc.

This gives the researchers ways to refine ideas about learning. It may well be that in the future, that developers of a process will not only be able to set reasonable expectations for improvement of the process, but also relatively simple metrics that highlight the most likely areas for improvement.

As a fellow cognitive scientist...

[Anonymous Coward]

Let me be the first to explain "why Starcraft 2?"

The answer lies in the oft-cited measure of player skill at the game: actions per minute. This is an unprecedented numerical measure of expertise that lends itself well to the study of "expertise" -- a term which means something different in the study of the brain than it does to the everyday person. Expertise is nothing less than a figurative rewiring of your brain in order to better excel at a chosen repetitive task. You can check out Wikipedia if you want to read more about it.

Re:As a fellow cognitive scientist...

[ahabswhale]

As a daily SC2 ladder player, I can definitely state that APM has little to do with skill. It's a useful metric to determine how well a skilled player can execute his strategy but that's it. It doesn't, in any way, indicate whether a player can dynamically adjust his play to beat his opponent. It just means that if he can come up with a sufficient strategy, that he will probably be able to make it happen (assuming, of course, his opponent doesn't throw a wrench into the works). So, it scares me that anyone doing research in this field would put so much weight into APM.

Re:As a fellow cognitive scientist...

[Anonymous Coward]

Actually, APM has a lot to do with skill. But it's important to differentiate between raw APM and effective APM.

Raw APM is literally how many clicks and keystrokes you execute per minute. Effective APM is how many of those keystrokes were meaningful (repeatedly clicking the same move command over and over has no effect on the game and is thus not meaningful). The trick here is that the game obviously measures raw APM, and it's difficult for the game to discern exactly what separates meaningful clicks from meaningless ones, although Blizzard recently implemented an algorithm to try. If someone is just spamming the keyboard and mouse, they may have a high raw APM, but their effective APM is still low.

Anyway, effective APM measures your ability to multitask and effectively control your army and base. It doesn't have anything to do with high-level strategy, except that more APM allows you to execute on strategies more effectively; indeed, there are many strategies that are more or less impossible without having a high APM. It's also not a perfect measure, since it doesn't calculate how intelligent each move was in the context of the given strategy. But, it's better than nothing, and you can't play at a pro or even semi-pro level without being able to get your APM high.

Re:What you do is get every gamer to wear a helmet

[masternerdguy]

The problem is the number of cells in the brain. It took years to map the brain of a damn worm, your brain is orders of magnitude larger and has even more orders of magnitude in connections. Besides, I don't think we have a scanner that can read activity on such a small scale yet. You'd only be able to map their brain after killing them and putting them under a scope.

Day traders start taking notes

[earthsmurf]

RTS games are the most applicable to a wide range of real world systems & strategies than say FPS games or any other type of game. It's not only emergency systems that it compares to, but also at a high level to many different business types & strategies as well. Techniques such as knowing when to scout, expand, and attack all require precise timing just like in the business world, maybe even for the day trader. I wonder if successful day traders have a high APM?

APM is a distorted metric, many actions useless

[perpenso]

As a fellow cognitive scientist let me be the first to explain "why Starcraft 2?" The answer lies in the oft-cited measure of player skill at the game: actions per minute. This is an unprecedented numerical measure of expertise that lends itself well to the study of "expertise" -- a term which means something different in the study of the brain than it does to the everyday person. Expertise is nothing less than a figurative rewiring of your brain in order to better excel at a chosen repetitive task.

APM is a distorted metric. It does not distinguish between a meaningful action, a redundant action, a nervous "twitch" (i.e. multiple clicks rather than one), etc.

Furthermore it contains an additional distortion. Since it is a metric that players are evaluated by, and/or used in silly "pissing contests", it can be intentionally distorted. Why click on that point on the ground once when you can click on it five time rapidly? APM focused players often are manically clicking on empty ground issuing no unit orders when they have nothing to do for a second or two, they have rewired their brain to have them do "something" even if there is nothing useful to be done.

For the programmers reading along, Think of APM as the LOC (Lines of Code) of the Starcraft world. Both metrics can be meaningful in an idealized setting, but such is not the setting of most real world events.

"micromanagement, not strategy"

[Nova Express]

“Amateurs talk about strategy, professionals talk about logistics.” - Carl von Clausewitz

Now, what's Starcraft about again?

Re:"micromanagement, not strategy"

[Derkec]

How did that work out for him in Russia?

Re:"micromanagement, not strategy"

[ultranova]

How did that work out for him in Russia?

He reached Moscow. So I'd say it worked for him far better than most "conquer Russia" strategies have.