Does Mathematical Tuning Make Games Better?

simoniker writes "What do game designers need to know about statistics? Age Of Empires DS designer Tyler Sigman focuses on statistical topics that he believes should be understood by game designers, in a new article. His reasoning: 'In the game I just finished, we recorded data from play sessions and then set challenge levels in the game based upon the mean and standard deviation values from those recorded data. We set Medium difficulty to be equal to the mean values, Easy difficulty to be equal to the mean minus a certain amount of standard deviations, and then Hard difficulty equal to the mean plus a certain amount of standard deviations.' Would all games be better if they were tuned mathematically?"

Comments

Leave out "Mathematical"

[AKAImBatman]

"Tuning" makes games better. Period. End of story.

Since tuning is all about improving the feel of the game to the humans who will interface with it, it all depends upon the creator for how he wishes to accomplish this. In this case, the creator was looking for sweet spots that he was able to find through mathematical manipulation of sampled data. In other cases, using math to tune the results might give the game a clinical feel; something that's generally bad for video games. (Unless you're playing Trauma Center. :-P)

So the question is pretty much moot. Creating a good game is an art form, but even art can benefit from a few structural calculations. :)

Re:Leave out "Mathematical"

[Wandering Wombat]

That's architecture... the blending of form and spirit, with the cold hard numbers that make it possible. A painting can be done with ANYTHING, and still fulfill it's role. If a building looks the way you want, but you forgot to carry some threes, you're probably going to kill some people, and go to jail for neglicence.

More games need that kind of accountability. :)

Re:Leave out "Mathematical"

[lostboy2]

I'll hop on the agreement bandwagon. Mathematics alone does not make for a better game.

Case in point: I once had a cribbage game where you could play against the computer and set different levels of difficulty. I quickly discovered that "Expert" level just meant that the computer got better hands more often -- it had nothing to do with the quality of the computer's strategy. After getting lousy hands several games in a row while the computer consistently drew hands like 4-5-5-6, I simply stopped playing. While "Expert" level was certainly harder, it was also not fun to play.

So, while TFA has a point about statistics being important for game design, that's not much more profound than saying that vision is important for driving cars well.

Re:Leave out "Mathematical"

[SimHacker]

BlackEmperor writes: "Is the intelligence of an object being tuned (cool)" or is the object just being given a x% production/whatever boost (sucks).

"Intelligence" generally does not have a "knob" that you can simply tune up and down. How smart a character acts is an emergent behavior that depends on many other factors in the system, themselves which include many tunable knobs like "x% whatever boost", and complex dynamic behavior scripted into the code.

That said, you can still add more layers to tune the higher level behavior of the AI, which randomize the emergent behavior and diminish the effect of the "intelligence".

For example, The Sims uses a complex dynamically tuned algorithm to figure out what action each character does next, based on scoring "advertisements": It asks all the objects in the house to enable and score all their action "advertisements", as the advertisement applies to a particular Sim at the current time. For example: the fridge's "fix dinner" advertisement is enabled in the evening; the toilet's "use toilet" action gets a higher score the fuller your bladder is; the bed side you usually sleep in remembers the relationship and raises the score, which increases when your energy is low, so it's more likely you will go back and sleep in the same place every time you're tired.

That algorithm produces a list of all possible actions, which is sorted by score. But if perfect Sims always performed the first action with the highest score, it would have made the game frustrating and un-challenging. Because if you didn't interfere with the Sims lives, they would automatically always do the right thing all the time, without they player's help. They didn't need you, because they were theoretically smart enough to live the optimal most efficient life. (I think that's what BlackEmperor means by "a clinical feel".) Anything the player told the perfect Sims to do would only make their lives worse off. The game was more fun and engaging if player intervention was required to help the Sims be happier.

So instead of choosing the first action on the list, the Sims choose randomly from the "n" top scoring actions. So "n" is a positive whole number that can be increased to "dumb down" their automymous behavior, and roughly controls how "whimsical" (or "stupid") they are.

Of course that's all assuming the ideal virtual world in which all the advertisements are truthful, mathematically correct, in your best interest, internally and externally consistent, without any contradictions, and perfectly fair and balanced. Which of course they're not, because there are many false advertisements, exagerated scores, distorted curves, and arbitrary tweaks, hacks, quirks and ironies in the code (especially stuff written by twisted players like SimSlice), that make simulated life more interesting. So tuning "n" to different values has a non-linear complex effect over their "intelligence", and it's quite coarse with a small range of useful values.

So of course you have to use lots of play testing to figure out the best value for "n", not pure mathematics.

-Don

developers

[the dark hero]

I always figured that there was some sort of mathematical tuning in videogames. I mean, there has to be a better way of balancing a game than just plugging in numbers by trial and error. Maybe its that i've played too many RPGs where math is an obvious factor, but every punch or every bullet has a numerical value right? It only makes sense to me that there would have to be some kind of number crucher on the dev team.

Re:developers

[paeanblack]

It's not trial and error. It's a binary search algorithm [wikipedia.org] that executed within O(log n) time. :P

Think of it like turning a knob back and forth, getting closer to the setting you feel is best.

That method will only deliver a local maximum of a polynomial function. If your game has any complexity at all, your proposed method is even less useful than trial and error.

You call that "mathematical modelling"?!

[6350']

We set Medium difficulty to be equal to the mean values, Easy difficulty to be equal to the mean minus a certain amount of standard deviations, and then Hard difficulty equal to the mean plus a certain amount of standard deviations
Wow. If this is "mathematical modelling", then me swapping the coffee mugs out for wine glasses in my kitchen cubbard would be "advanced sphere packing analysis and optimization".

Game tuning as more art than science. The goal is not to create an interestingly distrubuted difficulty curve, but to create an "easy", "medium", and "hard" amount of enjoyable challenge. Huge amounts of time can be (and frequently are) wasted focusing too-strongly on a "cool" and intriguing difficulty model that some under-experienced junior designer is all fired up about, instead of keeping the focus tightly and solely on the how the game actually feels.

Re:You call that "mathematical modelling"?!

[SnowZero]

The sad thing is the article teaches people the tools but not when to apply them. For example, it introduces the normal distribution, but no tests to check the normal assumption on a dataset are given. It also ignores the incredibly important subject of outliers. So in the end, he balances his game using a mean and variance, when its quite likely that a median and percentiles would have been better. Oh well.

I am constantly amazed at how much game programmers know about the mathematics and algorithms for computer graphics, and how little they know of everything else. And if you want to help them out by writing something, it is usually the case that a crap article or book on the subject has already been written by a game programmer.

I bet Easy isn't actually easy.

[onion2k]

I'm a bit of a fan of computer games. I've been playing them pretty close to my entire life. I'm 29 now and since the days of the Zx Spectrum I've probably played at least a couple of hours a week, often much more.

Unfortunately I suck at games. My coordination is all over the place. I have NO patience. I play games for a laugh, I don't want to invest a great deal of time learning a game or practising it. I want to pick it up, play for a while, and be entertained. As a rule I always play games on Easy because I don't want a challenge. I don't want to get frustrated playing the same level over and over. I want that feeling of progression like I'm getting somewhere. I can honestly say that if I get stuck for more than an hour in a game it gets turned off and never switched on again. I make a mental note not to buy a game from the same people again.

Easy is for people like me. Lazy, good-for-nothing "casual" players who have no skill to speak of and a life of some sort that means there isn't the time to learn perfection. I expect Easy to be easy. I very much doubt that "mean minus standard deviation" of some enthuiastic professional testers or Beta players is really going to be down at my level.

Please, for the love of Mario, when you're writing a game, sit your mother down in front of it for a few hours and tweak the difficulty of "Easy" to something she can cope with. That way I might buy your sequel.

Alternatively, give me God mode. :)

Re:I bet Easy isn't actually easy.

[Boronx]

Sid Meir Solved this problem years ago: Have a whole slew of difficulty levels from braindead to impossible. Somehow I doubt he needed much math for the easiest Pirates! level, where it's nearly impossible to lose in any situation, and in a few minutes play time you are god of the sea.

Sims Designer Chris Trottier on Tuned Emergence

[SimHacker]

Sims Designer Chris Trottier on Tuned Emergence and Design by Accretion

The Armchair Empire interviewed Chris Trottier, one of the designers of The Sims and The Sims Online. She touches on some important ideas, including "Tuned Emergence" and "Design by Accretion".

Chris' honest analysis of how and why "the gameplay didn't come together until the months before the ship" is right on the mark, and that's the secret to the success of games like The Sims and SimCity.

The essential element that was missing until the last minute was tuning: The approach to game design that Maxis brought to the table is called "Tuned Emergence" and "Design by Accretion". Before it was tuned, The Sims wasn't missing any structure or content, but it just wasn't balanced yet. But it's OK, because that's how it's supposed to work!

In justifying their approach to The Sims, Maxis had to explain to EA that SimCity 2000 was not fun until 6 weeks before it shipped. But EA was not comfortable with that approach, which went against every rule in their play book. It required Will Wright's tremendous stamina to convince EA not to cancel The Sims, because according to EA's formula, it would never work.

If a game isn't tuned, it's a drag, and you can't stand to play it for an hour. The Sims and SimCity were "designed by accretion": incrementally assembled together out of "a mass of separate components", like a planet forming out of a cloud of dust orbiting around star. They had to reach critical mass first, before they could even start down the road towards "Tuned Emergence", like life finally taking hold on the planet surface. Even then, they weren't fun until they were carefully tuned just before they shipped, like the renaissance of civilization suddenly developing science and technology. Before it was properly tuned, The Sims was called "the toilet game", for the obvious reason that there wasn't much else to do!

Here are some questions and answers from the interview with The Sims designer Chris Trottier:

[...]

Q: On paper, a game where you simulate daily life doesn't sound that interesting. Yet The Sims is really fun to play, so much so that it is now the biggest-selling PC game ever. Although any development team working with Will Wright has to feel confident in the product they are creating, has the unbelievable popularity of the franchise shocked even the development team?

A: Absolutely. When I was first assigned to The Sims, it was not-very-affectionately-known within the company as "the toilet game." Will Wright had tremendous stamina for the risk involved with trying something very new, but there were certainly a lot of head-scratchers both on the team and outside of it. In all honesty, the gameplay didn't start to really come together until a couple of months before ship. Being involved in that tuning process, and seeing the game take shape from what had previously been a mass of separate components, was one of the most powerful experiences of my career.

[...]

Q: What makes The Sims massively popular with female gamers, who traditionally don't make up a big number of gameplayers?

A: It's so hard to answer that question without making broad, sweeping statements that anyone of my gender would probably resent. But... I can say that there are several untraditional forms of gameplay in The Sims. For instance, there are many people who spend most of their time decorating and redecorating their homes. Since there's so much user-created content being posted on websites, they spend a lot of time collecting more looks to add to the game. There are also a lot of people who enjoy having a fantasy life where they get to call the shots... for good or for bad. I've heard a lot of stories

Re:Sims Designer Chris Trottier on Tuned Emergence

[drinkypoo]

While I really have to apologize to the author of the game for using it as an example, 2H4U (Too Hard for You) demonstrates exactly what happens if you throw all the elements in the pot, but don't take the time to balance and tune them

to be honest, I read the name and the premise and decided not to download. there is no reason for the premise to be exciting. it's more of a demonstration when you throw all the elements for one meal into one pot, and all the elements for another meal into another pot, and then accidentally combine the two pots.

Not to mention the inexplicable decision to name a game "too hard for you". Is it? Hell with it then. I'll go play some Gate 88.

my take

[dosboot]

I'm a mathematician and amateur game programmer. The problem I have with tuning is that you aren't paying attention to the actual game design when you make stupid changes like adjusting health/damage parameters. Games can be equally hard but not equally fun. If a boss (or level, or anything) is too hard then maybe the problem is with everything else in the game up to that point which did not prepare the player for that challenge. i.e., the player should have had opportunities to learn the techniques needed (which themselves can either logical techniques or twitch techniques). The same goes for something being too easy: you've in effect over prepared the player to beat X and need to add more depth to your game (different things to master) or make the game shorter.

You don't want to end up with a game that plays like a steady hike up the side of a foothill. These games are only 'hard' because you aren't stimulating the player to learn. A fun game has hills and valleys which in the end has the player standing on top of a mountain.