Slashdot Mirror
Does Mathematical Tuning Make Games Better?
simoniker writes "What do game designers need to know about statistics? Age Of Empires DS designer Tyler Sigman focuses on statistical topics that he believes should be understood by game designers, in a new article. His reasoning: 'In the game I just finished, we recorded data from play sessions and then set challenge levels in the game based upon the mean and standard deviation values from those recorded data. We set Medium difficulty to be equal to the mean values, Easy difficulty to be equal to the mean minus a certain amount of standard deviations, and then Hard difficulty equal to the mean plus a certain amount of standard deviations.' Would all games be better if they were tuned mathematically?"
"Tuning" makes games better. Period. End of story.
:-P)
:)
Since tuning is all about improving the feel of the game to the humans who will interface with it, it all depends upon the creator for how he wishes to accomplish this. In this case, the creator was looking for sweet spots that he was able to find through mathematical manipulation of sampled data. In other cases, using math to tune the results might give the game a clinical feel; something that's generally bad for video games. (Unless you're playing Trauma Center.
So the question is pretty much moot. Creating a good game is an art form, but even art can benefit from a few structural calculations.
Javascript + Nintendo DSi = DSiCade
I always figured that there was some sort of mathematical tuning in videogames. I mean, there has to be a better way of balancing a game than just plugging in numbers by trial and error. Maybe its that i've played too many RPGs where math is an obvious factor, but every punch or every bullet has a numerical value right? It only makes sense to me that there would have to be some kind of number crucher on the dev team.
You constantly struggle for self improvement - and it shows.
Hooray for bad Engrish on fortune cookies
Wow. If this is "mathematical modelling", then me swapping the coffee mugs out for wine glasses in my kitchen cubbard would be "advanced sphere packing analysis and optimization".
Game tuning as more art than science. The goal is not to create an interestingly distrubuted difficulty curve, but to create an "easy", "medium", and "hard" amount of enjoyable challenge. Huge amounts of time can be (and frequently are) wasted focusing too-strongly on a "cool" and intriguing difficulty model that some under-experienced junior designer is all fired up about, instead of keeping the focus tightly and solely on the how the game actually feels.
I'm a bit of a fan of computer games. I've been playing them pretty close to my entire life. I'm 29 now and since the days of the Zx Spectrum I've probably played at least a couple of hours a week, often much more.
:)
Unfortunately I suck at games. My coordination is all over the place. I have NO patience. I play games for a laugh, I don't want to invest a great deal of time learning a game or practising it. I want to pick it up, play for a while, and be entertained. As a rule I always play games on Easy because I don't want a challenge. I don't want to get frustrated playing the same level over and over. I want that feeling of progression like I'm getting somewhere. I can honestly say that if I get stuck for more than an hour in a game it gets turned off and never switched on again. I make a mental note not to buy a game from the same people again.
Easy is for people like me. Lazy, good-for-nothing "casual" players who have no skill to speak of and a life of some sort that means there isn't the time to learn perfection. I expect Easy to be easy. I very much doubt that "mean minus standard deviation" of some enthuiastic professional testers or Beta players is really going to be down at my level.
Please, for the love of Mario, when you're writing a game, sit your mother down in front of it for a few hours and tweak the difficulty of "Easy" to something she can cope with. That way I might buy your sequel.
Alternatively, give me God mode.
http://twitter.com/onion2k
It's always better if the answer to 2 + 2 = 4 instead of some null value that the programmer doesn't catch until after the game is released.
The article is essentially a fluff piece, but crammed between the useless paragraphs were occasional nuggets of practicality. The important thing to take away is that numbers are important in data models (which is what balancing a game involves.) Statistics is one way to quickly and abstractly summarize a lot of numbers. Read on for a boring, detail-oriented analysis:
While the article doesn't present it well, I think that the author probably is very good at tuning games. He doesn't come right out and say it, but it is important to target the statistics you are gathering to address some issue. For example, the example of "time to level" in an RPG shows consistent times with one outlying data point much lower than the others. The author leaves it as a hypothetical question, but this is a clear sign of an overpowered class or an exploit. However, if the data from this same example had instead been "damage done" or "quests attempted" or "distance walked" by players, it is doubtful that this same issue would have been noticed.
As a futher example, the statistics in designing a trick-taking card game would be different. The power of choosing trump could be modeled by how many tricks are taken with trump on average. The number of tricks per hand could change the accuracy (and relative risk) of bidding, so recording the average bid between design iterations could be useful.
I can still remember a number of games where an archer had an equal probability of defeating a tank. In the real world, the tank will almost always beat the archer. The mathmatics in the game should reflect reality (or the virtual reality). If a character is tougher or more powerful than another character, that ratio should be reflected in the results of the contest between the two characters. This means the more powerful character should have more than a 50-50 chance of victory, likewise the tougher character should have a lower probability of incuring damage. In this way, game play can be improved.
As I understand it, they are basically making 'Hard' be 'As hard as possible and still beatable based on previous user performance'. I would get bored with a game like that and stop playing it after not very long. Back in the day, I was pretty good at Starcraft (not as good as some of those disgusting fast Asian kids these days, but pretty good still.) Know how I got that good? Getting my ass handed to me over and over again, finally winning, and then designing an even more diabolically difficult level for myself. Lather, rinse, repeat. Same basic principle goes for just about any game I've invested time in and gotten good at...you get better by losing and learning from the mistakes that made you lose, not by just barely winning over and over again. That's just boring.
Unpleasantries.
Means and standard deviations are not exactly mind bending statistical exercises. I wonder if they accounted for different biases in the data from their "play sessions." Who was playing these games and what was their motivation for being there? Were they industry people? Developers? Testers? Beta Testers? and etc...
Sims Designer Chris Trottier on Tuned Emergence and Design by Accretion
The Armchair Empire interviewed Chris Trottier, one of the designers of The Sims and The Sims Online. She touches on some important ideas, including "Tuned Emergence" and "Design by Accretion".
Chris' honest analysis of how and why "the gameplay didn't come together until the months before the ship" is right on the mark, and that's the secret to the success of games like The Sims and SimCity.
The essential element that was missing until the last minute was tuning: The approach to game design that Maxis brought to the table is called "Tuned Emergence" and "Design by Accretion". Before it was tuned, The Sims wasn't missing any structure or content, but it just wasn't balanced yet. But it's OK, because that's how it's supposed to work!
In justifying their approach to The Sims, Maxis had to explain to EA that SimCity 2000 was not fun until 6 weeks before it shipped. But EA was not comfortable with that approach, which went against every rule in their play book. It required Will Wright's tremendous stamina to convince EA not to cancel The Sims, because according to EA's formula, it would never work.
If a game isn't tuned, it's a drag, and you can't stand to play it for an hour. The Sims and SimCity were "designed by accretion": incrementally assembled together out of "a mass of separate components", like a planet forming out of a cloud of dust orbiting around star. They had to reach critical mass first, before they could even start down the road towards "Tuned Emergence", like life finally taking hold on the planet surface. Even then, they weren't fun until they were carefully tuned just before they shipped, like the renaissance of civilization suddenly developing science and technology. Before it was properly tuned, The Sims was called "the toilet game", for the obvious reason that there wasn't much else to do!
Here are some questions and answers from the interview with The Sims designer Chris Trottier:
[...]
Q: On paper, a game where you simulate daily life doesn't sound that interesting. Yet The Sims is really fun to play, so much so that it is now the biggest-selling PC game ever. Although any development team working with Will Wright has to feel confident in the product they are creating, has the unbelievable popularity of the franchise shocked even the development team?
A: Absolutely. When I was first assigned to The Sims, it was not-very-affectionately-known within the company as "the toilet game." Will Wright had tremendous stamina for the risk involved with trying something very new, but there were certainly a lot of head-scratchers both on the team and outside of it. In all honesty, the gameplay didn't start to really come together until a couple of months before ship. Being involved in that tuning process, and seeing the game take shape from what had previously been a mass of separate components, was one of the most powerful experiences of my career.
[...]
Q: What makes The Sims massively popular with female gamers, who traditionally don't make up a big number of gameplayers?
A: It's so hard to answer that question without making broad, sweeping statements that anyone of my gender would probably resent. But... I can say that there are several untraditional forms of gameplay in The Sims. For instance, there are many people who spend most of their time decorating and redecorating their homes. Since there's so much user-created content being posted on websites, they spend a lot of time collecting more looks to add to the game. There are also a lot of people who enjoy having a fantasy life where they get to call the shots... for good or for bad. I've heard a lot of stories
Take a look and feel free: http://www.PieMenu.com
you are right in that a rocket launcher should easily outplay a machine gun... but your missing the purpose of the tuning. the tuning in an fps game wouldnt take place on making the guns do equal damage... but making the damage and the level design create an even playing field.
designers can balance a map out by varying the level design, so that one weapon can be the advantage in one area, while other areas are better suited by other weapons... giving one weapon the advantage at all times it not an enjoyable, nor is a game where no one has the advantage. another option is to break the key elements apart for example: lower shield plus good gun on one side of the map... with the better shield and health on the opposite side.
I've played games that have done this in the past and done it wrong.
;)).
Basically, the game "watched" me get better at playing a certain level of the game... unsuccessfully. But, apparently it saw that I was doing soooo well, that it decided to increase the difficulty without telling me. Which actually made me continue to fail to complete the level. Quite frustrating, not to mention annoying to have to keep an eye on the difficulty level so that it doesn't go beyond what you want.
IMO, Ratchet Deadlocked did it right. If it sees you having problems/too easy, Clank says something along the lines that if you're having problems or what a greater challenge, you can adjust the difficulty at load. If you don't and continue to perform in this way, it'll actually pop up a dialogue box asking you if you want to change the difficulty at the next death (haven't experienced the way it does it if you're too good
Basically, auto adjust can only be done correct for a certain amount of people, but will piss others off. Do it the Ratchet Deadlocked way. It'll give a hint to the player and leave the final decision up to him/her.
It doesn't appear that the developers of AOE DS tuned the game, mathematically or otherwise. The game frequently locks up on my DS, and there is a known bug regarding the length of your profile name which can cause a lockup.
These days with the hardware doing all of the calculations, how much tuning can you actually do?
I recall Thief had game modes called "Normal", "Hard" and "Difficult" (or names to that effect), and they weren't kidding. I really could have done with an "Easy".
I'm scared of numbers that can't be written as a fraction. It's an irrational fear.
I'm a mathematician and amateur game programmer. The problem I have with tuning is that you aren't paying attention to the actual game design when you make stupid changes like adjusting health/damage parameters. Games can be equally hard but not equally fun. If a boss (or level, or anything) is too hard then maybe the problem is with everything else in the game up to that point which did not prepare the player for that challenge. i.e., the player should have had opportunities to learn the techniques needed (which themselves can either logical techniques or twitch techniques). The same goes for something being too easy: you've in effect over prepared the player to beat X and need to add more depth to your game (different things to master) or make the game shorter.
You don't want to end up with a game that plays like a steady hike up the side of a foothill. These games are only 'hard' because you aren't stimulating the player to learn. A fun game has hills and valleys which in the end has the player standing on top of a mountain.
Dystopia, a Half-Life 2 mod, developed a statistics server that collected information on in-game beta testing they conducted with their world-wide fan base. They used e.g. rates of damage inflicted in the beta tests to balance the new weapons they implemented between beta 4 and their release candidate. There's a very good interview right at the top of the Dystopia main page that discusses the weapon balancing in some depth.
If mod developers relying on volunteers can use statistical analysis, its pretty much applicable to anyone.
remember megaman, those games frustrated me so much because the levels basically varied in difficulty by maybe 25% max, and they were all hard. and if you didnt do them in the right order then its going to be so hard youll punch your very own cat. now metroid on the otherhand was nicely balanced.
nothing can replace good and thorough testing, its obvious when a game was rushed out the door, like for instance running out of ammo, while playing conservatively. and then having to reload to a place 2 hours back and trying to conserve even more. god there is nothing that pisses me off more. that to me is the worst thing, artificial difficulty. platformers are usually worse in the this regard, with horrible control, or collision detection hampering your progress. or the loathsome camera that points in the totally opposite direction that you are looking at to avoid a wall! why do games still suffer from this.
anyway i think this statistical method is just fine for an rts, but for a more linear type game i think the old trial and error hands on method would result in a game that is more uniformly fun.
Wii sports has a rating for your user, and changes the difficulty based on your performance.
For example in Tennis it's become noticably harder at the 500 level, while at the 0 level the computer is trivial to beat.
I think satisfaction happens when the game is hard enough that failure is a realistic possibility, but you still tend to win more often. You can tune on the players performance, or a sample audience performance, it doesn't really matter.
Some hardcore games tune ultimate hardness with the intent of having only a handful of people ever suceed at it.
More complex modeling than means and standard deviations have gone into improving online games (unless the author was simplifying things by saying means and standard deviations).
For example both Trueskill and etpub use a Bayesian form of Arpad Elo's rating system to rank and match players.
I did some work modeling kills and wins in Enemy Territory that yielded interesting insights about map- and weapon- balance in that game.
At arena.net, there is at least one employee whose sole job is to model the association between skills winning probabilities in order to balance the skills across the classes.
I'd like to see an RPG that rated players and mobs based on statistical models and made sure the game always gave a near constant level of difficulty. RPGs present, I think, more difficult problem in this area than some of your standard first-person-shooters, etc., because of the variety you have in builds.
I'm a statistician / machine learning guy, so I am all for modeling aspects of games to improve them.
This is the dumbest ask slashdot I've ever seen. Fucking pathetic. I realize that's strong language, but honestly...
really it's just tuning through statistical means.
tuning == always important
tuning through statistical means == might work, assuming that statistical means yields something fun for the player.
music - http://www.subatomicglue.com
You could play Dune 2, in which you must give every single unit an order. So: Click the troop, click where you want him to go. Rinse and repeat for however many you have. I think they at least attack automatically...
Or, you could play Starcraft. Click+drag to select a squad -- up to 12 units, where some units (dropships, overlords) can carry other units inside them. Click where you want them to go, watch them attack anything they find (or run, depending on what mode you have them set to).
Or, you could play Natural Selection. Drop some things, click a guy, order him to go build them. Or, most of the time, you don't even have to do that -- just tell him over voice chat what you want him to do, if he's not doing it already -- you can play Natural Selection as an FPS or be the Commander and play it as an RTS, and the commander is ordering around other, real human players, who think it's an FPS.
Technically, yes, it takes away from the interactivity. But you still have to be there, still thinking of a strategy and where to go next, and it does take away the tedium. It has the computer doing exactly what you know you'd be doing anyway.
I'm not saying I want aimbots in an FPS -- after all, the whole point of an FPS is your reflexes and accuracy. I am saying, however, that clicking one troop at a time (in Dune) is not really a skill, certainly not the kind that makes the game fun. Zergling rushes are fun, but they'd be absolute hell in Dune.
So, similarly -- I don't actually mind, say, Final Fantasy X, where the game waits for you to choose what to do for every single player's move. But let's face it, the whole point of the interactivity is not tapping X to have Auron attack when you know he's going to attack every turn unless something different happens. The whole point of any of the gameplay is for you to make decisions like figuring out that a zombie boss can be killed with a couple of Pheonix Downs -- the bot won't do that for you -- or deciding whether to move in, or when to use your salvos of missiles.
As you say yourself:
Decision making. Not tedium.
Don't thank God, thank a doctor!
Alright, you do have some valid points.
One thing I should mention, some people do improve their games by making them easier -- or at least consistent in their difficulty. Jak & Daxter was mostly easy, partly because you could skip most of the game, by choosing easier alternatives -- it's a pretty open game; each area needs x number of Power Cells, and there are probably 2x or 3x quests you can do in that area which give you a Power Cell. But some parts were hard; for instance, the final boss is very difficult, and since he's a boss, you have to fight him anyway.
The sequel, Jak II, is the kind of thing you'd hate. Responding to what they thought gamers wanted, they made it more linear, gave it more of a plot, and made it MUCH harder, even MUCH darker of a theme. Worse, it was inconsistently hard -- you'd get relatively easy missions, then suddenly something impossibly hard, or things that were hard in different ways.
The final game in the trilogy, Jak 3, wasn't really any easier, but it had a much gentler progression -- you'd be getting the practice you need, and still progressing, as it got steadily harder, but it wouldn't really feel that much harder, because you'd be better by then.
I imagine you'd still give up on Jak 3, but I imagine it'd be much, much better for you. So, I'd suggest that it might be a bit of an overreaction for you to just give up on a developer because of that. One solution might be to simply have some good friends -- maybe a significant other, but in any case, someone who lives physically close to you -- who play games, so you can go to their place and play their games, so you can get an idea of which ones you might buy for yourself. Or, buy games and trade them -- if you buy a game you don't like, trade it with a friend for one you do.
In any case, why limit yourself to three difficulty levels? I have a friend who typically buys a game, opens it up, puts it on "hard", and finishes it in a day or two -- saving him money, too, as he just rents them, rather than buying them. My brother will play a level over and over again, till he finally beats it -- no matter how hard it is, he just memorizes a way through it. And there's you -- from what you're telling me, you're just lazy enough that most games should have cheats, although I'd much rather have there be an "easy" enough mode that you don't need them.
Most games that seem to have this figured out actually have four difficulty levels, at least. Quake 3 has five:
Halo (1 and 2) has four:
And Duke Nukem 3D also has four:
I think what you're really looking for, though, is games that don't have any difficulty settings -- but manage to be interesting enough for you, and easy enough for me. Pretty much any Zelda falls under this category -- if it ever gets too hard, you can just look up the solution. Ocarina of Time was a masterpiece. Or Final Fantasy games -- Final Fantasy X pretty much never requires coordination, only enough patience to cut through to the next save point. If you ever get stuck, you can run around a save point to gain experience -- it may be tedious, but it's not hard.
In general, these kinds of skills are transferable, too. I'm decent at an FPS -- pretty much _any_ FPS. I haven't played all of them, but they aren't different enough that it's actually hard to learn a new one. The coordination I get from playing a game on "Normal" (instead of "Easy" or "Godmode" all the time) -- it's still fun every step of the "practice", but I can pick up wildly different kinds of games relatively easily.
One final note: Keep in mind that the point of standard deviation here is that you're not the market, and my mother's not the market. The market generally can handle t
Don't thank God, thank a doctor!
How would you ever tune this game? I have been playing it for nearly five years, including all of its mods and new developments. Tuning it is nearly impossible because the tactics employed by new players are so vastly different from veterans that I cannot fathom how it could be done mathematically.
Nevertheless, if you coders want to go at it, its open source. Go to the Bear's Pit.
Clickety Click
in an environment ultimately based on 0/1, by flavor ? :D
errr....umm...*whooosh* *whoosh* Is this thing on ?
Statistics and math tuning should be selectively applied to add polish to the game play. If you try to balance everything out using math you tend to blur the lines of creativity. Especially in RTS games and good RPG games, interesting game play comes from the potential imbalances in the game that require tactics to overcome and choices to be made. Between which faction you choose, or whether to be a Paladin or Shaman in WoW (which has now has had it's uniqueness "balanced" away), it's the unbalanced choices that help people feel that their decisions make a difference in a game.
There's a difference between math and statistics (statistics uses some math and substansial amount of mathematical notation), and TFA seems to be all about statistics and probability. What you are suggesting is a negative feedback mechanism, and hasn't much to do with neither :) Not that I think that is a bad concept though.
There's a game which is tuned to make winners win more. It's the main reason I stopped playing. CS was the same. Game creators would do well to take a page out of this guy's book and try to analyse the larger trends that will occur over time with their games.
It's OK Bender, there's no such thing as 2.