Slashdot Mirror
Learning the Scientific Method From Games
Wired is running a story about a research paper out of the University of Wisconsin-Madison which discusses how some games get players to do scientific research without them explicitly realizing it. The paper itself is also available. Quoting:
"... we examine the scientific habits of mind and dispositions that characterize online discussion forums of the massively multiplayer online game World of Warcraft. Eighty-six percent of the forum discussions were posts engaged in 'social knowledge construction' rather than social banter. Over half of the posts evidenced systems based on reason, one in ten evidenced model-based reasoning, and 65% displayed an evaluative epistemology in which knowledge is treated as an open-ended process of evaluation and argument."
Har har har! I is made smarterer by playing the WoWs!
These days most scientists have to work hard to keep their funding: publishing, making sure that they don't say things that offend their benefactors, etc.
If you don't believe me, go listen to the CMU podcast about Bill Gates. They suck up to him so badly it is embarrassing just to listen to it. So much for what was once a leading Computer Science department.
As much as we might dislike it, that is the New Scientific Method.
Engineering is the art of compromise.
...to the summary was: What? I didn't catch a word of that.
Thankfully, TFA is a bit better:
While I have the highest respect for my esteemed colleagues in Madison, I find myself disagreeing with Steinkuehler's conclusions. These kids are not practicing science, they're practicing being human beings. And as human beings, we find new and inventive ways to meet a challenge whenever one is presented to us.
All one needs to do is look back through history. Aliens didn't build the pyramids; humans did. Humans who were given the seemingly impossible task rose to the challenge and made it happen. The Flavian Amphitheatre (aka the Coliseum) didn't just appear when someone pulled the plug on a drain and the water swirled around. Humans wanted a better place to host their blood sport. So they devised a new method. Trains didn't start moving themselves. Humans had a problem of not enough labor. So they devised a solution.
Which isn't to say that these many engineering feats were devoid of what we today think of as "science". However, it is important to remember that the scientific process (i.e. the thing that separates "science" from simply "effort") is a formalized process that vets the actual facts from the statistical noise. If you are not following the formalized methodology, you are not performing "science".
Which isn't to say that I don't think these kids deserve mad props. They used their brains and were rewarded for it. Which is something to be proud of in a comfortable modern society that makes it all too easy to turn one's brain to the "off" position. :-)
Javascript + Nintendo DSi = DSiCade
Games undeniably require considerable evaluation and analysis to play at a high level. Strategies have to be formulated according to a model, and honed by tests against the gameworld, unknowns have to be tested, etc, etc. However, I'm not sure that what goes on in games is very much like science, except perhaps at the level of small scale mechanics(testing and similar).
With all the games I've ever played(not all of them certainly; but a fair few), I've never escaped the sense that I'm attacking a constructed puzzle, that was built by somebody with the explicit purpose of being a game. Games just reek of design. Some are better than others; but all of them are, to a noticable extent, a process of reverse engineering somebody's carefully designed puzzle.
Very few games even rise to the level of having a degree of unintended emergent behavior, rather than strictly scripted design, and what does emerge frequently derives from the humans in a multiplayer game, not the game itself. Most games are also orders of magnitude less complex than even fairly simple natural systems. Find an object in a game? It almost definitely has a purpose.
I agree that there is an overlap between the skills needed to dissect a game's workings, and the skills needed to study the world; but the epistemology of dissecting a game and the epistemology of studying the world seem significantly distinct.
Since games are probably more relevant than my amateur whinings about epistemology, does anybody have examples of games that seem particularly "natural", not in the sense of visually appealing or having accurate physics(though those are nice); but in the sense of feeling as though they hadn't been engineered in every detail?
It certainly isn't perfect; but I'd say Dwarf Fortress had some of that feel for me.
The WoW forum scientific method
Define the question
Gather information and resources (observe)
Form hypothesis
Perform experiment and collect dat.... Post to the forum with junk data (personal anecdotes)
Analyze dat.... get flamed by trolls incessantly to 5 pages
Interpret data and draw conclusions that serve as a starting point for new hypothe.... Blizzard buffs rogues (no matter what the subject was).
Publish resu.... You have wasted your time, just like when you play the game.
Rinse and repeat.
VLC FOR MAC IS DYING! IF YOU DEVELOP, PLEASE SAVE IT!!
The new Mythbusters scientific method:
1. Find urban legend
2. Wonder out loud if urban legend is true
3. Dream up inadequate set of tests. Bonus points if the tests only work for a specific subset of what you're trying to proove.
4. Get special effects team to engineer something wacky to apply tests in 3.
5. Blow something up
6. Comment on how awesome it is and congratulate self on coolness
7. Come up with off hand summary of whether myth is busted. Bonus points for ignoring or manipulating facts to suit.
8. Profit!!!
These posts express my own personal views, not those of my employer
Actually, while I'm not on the WoW forums at all, I can tell you that in some games they _are_ doing proper science.
The scientific method isn't about publishing in the sanctified journals and getting grants. It merely says that you have to make falsifiable predictions, and go with the theory that, in this order, (A) better explains the available data, and (B) if they're equal in the former aspect, pick the simplest.
And a lot of reverse engineering a game does just that: it devises experiments, measures some data, and makes falsifiable predictions. And in all cases I've seen, the simplest theory _is_ the one picked.
One example was COH before the game devs decided to actually show you the numbers. People did devise experiments to basically measure a lot of data, and solved the equations to come to the endurance (mana) costs of powers. And made falsifiable predictions.
E.g., one such experiment was to figure out a mix of powers which drains your endurance to zero, measure over how much time. Given enough such equations (at least one per variable), you can calculate the costs of each. And you can make the predictions for another set of powers. Or in reverse, how long it takes to recharge X points of mana, if you have powers A, B and C turned on. And again, the experiments were done, and available to everyone, to try to falsify the theory based on those predictions.
The theory also passed Occam's Razor with flying colours, in that it assumed the minimum possible: that each power only has a given cost per second, without any other interdependencies. E.g., if power A costs 0.21 endurance/sec, it always does so, regardless of whether you have powers B and C also active, and regardless of who you are or what you're doing. And again, even that assumption was falsifiable and supported by experimental data.
Now someone may argue that reverse-engineering a game is hardly a _serious_ scientific domain, or that it doesn't exactly benefit humanity in the same way as the LHC's reverse-engineering hadrons does. Fairy 'nuff. But nevertheless the scientific method was applied. Quite literally.
A polar bear is a cartesian bear after a coordinate transform.
In any game like this you'll always find a minority that will do all the theory crafting in order to maximize some aspect of the game. For World of Warcraft people usually refer to the elitist-jerks forums that specializes in this. However I'd say far less than 1% really ever does any concrete critical thinking about game aspects in order to improve play.
I myself have actually gone and done the math on some things, and I actually got into a sort of "discussion" with another person over it. The conclusions I came up with were easily testable and easily verifiable, yet they refused to believe unless they saw "proof" of it somewhere on the web. (For people who don't play WoW, or a rogue this is going to be a crappy example).
I play an assasination rogue, the staple of which is a move called "mutilate" - an attack with both daggers that awards combo points. However if there is a crit, it will award 3. I couldn't find any information how this actually worked so I started thinking about it myself. I found that each dagger had a chance to do a critical strike independently, and that if only one of them would crit I would get the extra combo point. So what are the chances of me getting the extra point? I just read up on simple probability and found it was a non mutually exclusive probability event and plugged in my critical chance (given on the character stat page). I can say however that out of the 15k people on my server, I'm probably the ONLY one who has bothered to figure this out. And also that not only will people not believe you and do the (scientific) tests themselves, but will blindly follow some random page on the web - even if it's wrong.
But that's the world we live in.