BioWare On Tracking Player Feedback

simoniker writes "BioWare's QA director Phillip DeRosa has written a piece called 'Tracking Player Feedback To Improve Game Design' over at Gamasutra, which deals with how game developers can use statistics, even before a game is released, to improve gameplay. DeRosa "...explains how the Mass Effect creator has set up and executed code-based monitoring of key metrics to test, analyze, and refine its projects through playtesting." Is this approach sensible, or could it be more like movie producers 'pandering' to test audiences?"

Comments

No, because games are made FOR players.

[EWAdams]

Unlike movies, in which taking into account the opinions of test audiences is thought of as compromising artistic vision, video games are made for players to play interactively. It's not just their money that matters, it's their ability to play and have a good time. The best game designer in the world doesn't always get it right. Playtesting is not just done for marketing reasons; it's absolutely imperative if you want to make sure the game is as good as it can be.

Re:No, because games are made FOR players.

[AvitarX]

So games arn't art then?

Re:No, because games are made FOR players.

[clem]

It's worse than you think. Art isn't art.

Re:No, because games are made FOR players.

[blahplusplus]

"Playtesting is not just done for marketing reasons; it's absolutely imperative if you want to make sure the game is as good as it can be."

I would make the argument that actually games are NOT truly targetted FOR *players*, if we are speaking about advancing the art of game design and gameplay. Tonnes of mediocre games rake in a lot of money for many other reasons.

I'd say lots of playtesting now-a-days is geared towards dumbing down and making games easier, less interactive, more passive and more mediocre. One only has to look at modern MMO's and console RPG's to compare the basic battle mechanics in those games with a game like God of War or other RPG's whose battle systems have real-time or more interactive elements.

I've been gaming for a long time and games have been steadily declining towards rigidity (rigid by the books game mechanics with minor tweaks) or mediocrity where game mechanics are thrown out or dumbed down entirely for making it easier to insert eye candy or to make it so easy to "play" all a drooling moron would have to do would be to babysit the robotic avatar.

Re:No, because games are made FOR players.

[Das Modell]

I'd say lots of playtesting now-a-days is geared towards dumbing down and making games easier, less interactive, more passive and more mediocre.

Ah, of course. Playtesting is meant to make the game as bad as possible so it won't sell. Now it all makes sense.

One only has to look at modern MMO's and console RPG's to compare the basic battle mechanics in those games with a game like God of War or other RPG's whose battle systems have real-time or more interactive elements.

Console RPGs have been more or less the same for over two decades. Western RPGs tend to be stat-based games where the character's abilities are emphasized far more than the player's own skill. WoW's combat (which is real-time) requires more "interactivity" and skill than something like Baldur's Gate.

You're effectively trying to impose one genre's standards on another. You may as well demand that racing games should have more RTS elements and a better combat system.

Re:No, because games are made FOR players.

[n3tcat]

Sure, until you get a game company that doesn't know how to "Just say no" sometimes, and they try to fulfill every game player's wildest dreams and completely cock up the balance in the game mechanics. This does not affect single player games, but rather average multiplayer games.

MMO's are not completely fucked by this sort of thing because many random variables cause the ebb and flow of the community to shift. Standard multiplayer games have a much tighter interactive experience, and screwing with the way a person plays against 15 other people is far more likely to destroy their gaming experience than screwing up how they grind for 15 hours a week.

Re:No, because games are made FOR players.

[n00854180t]

WoW requires more skill than Baldur's Gate? You, sir, are not a gamer. I doubt you've even ever played Baldur's Gate. Mashing a number row in order like a crack-fiend three year old does not take "skill". Baldur's Gate is a game of strategy at its core, and definitely requires far more skill than WoW ever could. So, go Collect 10 Kobold Candles, and have fun pressing your number row in order while doing it, several hundred times. I'm sure you are very *skilled* at it.

Re:No, because games are made FOR players.

[Jaysyn]

No doubt. You're not going to kill a demi-lich by mashing buttons, I don't care how "skilled" you are.

Re:No, because games are made FOR players.

[Das Modell]

WoW requires more skill than Baldur's Gate? You, sir, are not a gamer. I doubt you've even ever played Baldur's Gate.

You're the one who's never played Baldur's Gate. You can pause BG at any time, and you don't have to perform combos etc. in realtime, or run and jump around your target (which is what people do in PvP).

Oh, and since we're throwing around random accusation, then I may as well claim that you've never used a computer in your life. Ha ha, I win!

Baldur's Gate is a game of strategy at its core, and definitely requires far more skill than WoW ever could.

It requires skill the same way Civilization or Dawn of War does. WoW is a realtime action RPG, not a strategy game. There's a difference, but for some reason an elite hardcore gamer like you hasn't figured it out yet.

So, go Collect 10 Kobold Candles, and have fun pressing your number row in order while doing it, several hundred times. I'm sure you are very *skilled* at it.

When did I say anything about collecting items? I never said collecting items requires any skill. If you think you can win in WoW by pressing your number keys in order then I'm guessing you haven't played it.

I could claim that BG doesn't require any skill because you can just win by trial and error. When I had to take down a lich in BG2 with one character, I just loaded and reloaded until I found the right approach and got very lucky. It had nothing to do with skill.

Re:No, because games are made FOR players.

[n00854180t]

Oh yes, because you obviously can't "win" in WoW by trial and error, despite the fact that there are no penalties for dying. And yes, the game is essentially pressing your hot bar keys in order, during every fight, ad infinitum. Apparently you've never played that, either, though.

Re:No, because games are made FOR players.

[Das Modell]

Oh yes, because you obviously can't "win" in WoW by trial and error, despite the fact that there are no penalties for dying.

Your equipment durability decreases and you have to run back to your corpse, which can often be very time-consuming and annoying. Resurrecting at a Spirit Healer will give you resurrection sickness that lasts up to 10 minutes, making you essentially unable to fight. If the entire party is wiped in a dungeon or raid it must be restarted. Preventing a wipe is one of the most important things to do while in a raid or dungeon. If there are no penalties for dying then a wipe shouldn't be a big deal, right?

In Baldur's Gate you just reload your last save and all is forgiven.

And yes, the game is essentially pressing your hot bar keys in order, during every fight, ad infinitum.

Yeah, that's exactly how dungeons, raids and PvP are conducted. Killing normal mobs is easy routine in WoW, but killing normal enemies in Baldur's Gate isn't that hard either.

Apparently you've never played that, either, though.

This is funny coming from someone who thinks death is of no consequence in WoW.

All new media pander

[spun]

One has to pander when one is new. It takes years before a media is mature enough not to pander.

Re:All new media pander

[nuzak]

Music's been around for a little while. You telling me it doesn't pander?

Hell, commercial art sluts like Thomas Kincaid could be called pandering.

Re:All new media pander

[JFMulder]

Music is very old. Maybe it's like Alzeihmer. Start like a baby, grow up and end up a babbling idiot, like a baby.

Re:All new media pander

[heinousjay]

The fact that you can identify sectors which (in your opinion, I should note) "pander" to what people want (what a sin!) doesn't indicate that it holds true across the entire industry.

Re:All new media pander

[spun]

Yeah, of course media that have been around a while still pander. Commercial writer sluts like Tom Clancy pander too. I'm not saying old media doesn't pander. But artists in those media at least have some non-pandering art to look back on when deciding whether to pander or, you know, actually make art.

Re:All new media pander

[nuzak]

The other difference between games and writing and most visual arts is that their requirement of being interactive means that they must primarily entertain. Add to it the high costs of production, and this means that game producers are always going to have to write what their consumers want. Still, even most "artistes" do that, it's just that their audience tends to like being challenged with new ideas, and their primary motive is to communicate, not profit (again, distinguished from the commercial whores)

Funny that you mention Clancy -- he certainly writes better game scripts than books. Or maybe it's because the only real comparison I have to Splinter Cell is Metal Gear Solid (nice games, but the writing makes anything else look good).

Re:All new media pander

[Anonymous Coward]

Music, painting, Literature, and other mature arts only have to pander to be commercially successful. But computer games are expensive to make by their very nature, so the kind of people who are *death* to art are built right into computer gaming.

It Depends

[RamblinLonghorn]

I think it depends on the size and scope of a test audience. If they're picking up a group of 15 year olds at the mall on a weekend and having them sit down to play the game for a half hour, than yes, it is definitely pandering to a certain audience (this conversely could be said if they pick a few college aged gamers who spend several hours at a time on the game). However, if they have a decent beta/playtest application and select a good cross section of who they believe will be playing the game, then I think it's sensible.

The ultimate goal has to be to create a game that the most people will find enjoyable, but we all know that "you can't please all the people all of the time."

Re:It Depends

[Anonymous Coward]

Hell, you can't even please ALL of the people, EVER, PERIOD. I think it's nice that they are shooting for a nice percentage though.

Re:BioWare Nothing But A Shell Of Their Former Sel

[RamblinLonghorn]

Baldur's Gate games were amazing, but saying they've gone downhill since is a bit of a stretch.

KotOR was awarded game of the year by a ton of publications, and is generally agreed to be one of the best XBox games ever.
Jade Empire was received well and agreed to be one of the best RPG's of the year.
Neverwinter Nights was also a franchise that was well received and supported by an active community.

MOD UP: Sad but ture

[Anonymous Coward]

Yeah, I hate to admit I really haven't enjoyed their work since BGII (except for KOTR).
BGII's expansion was by its very nature overpowered
NWN was a single PC game, versus the group game of BGII. And, it was buggy as all Hell until about 6-9 months after it was released.

I long for a good RPG.
Really I should just play BGII again.
Or maybe someday I'll finish Torment.

Screw 'em

[JamesRose]

Sorry, but a little less bioware player tracking, and EA Games corporate restructuring. Take us back to the days when Bullfrog was making kickass games and stuff.

Isn't it weird when something can be so far from its roots even when its so new.

Re:Screw 'em

[Telvin_3d]

You might be able to get away with that if this article was about any other company, but Bioware? This is the company that has an almost completely unblemished reputation for quality, well made games that immerse the player and invite re-play. Take us back to the days of 'Baldurs Gate' and 'KOTOR'? Please.

Re:Screw 'em

[Sabathius]

Bullfrog was great...but LookingGlass was the _best_. System Shock, Thief...ahhh. The good ol' days.

Re:BioWare Nothing But A Shell Of Their Former Sel

[empaler]

Seeing as I have played none of those games, my only counterpoint can be towards your argumentation:
OF COURSE there is going to be a "Game Of The Year". Being the top of the pile can just mean that the rest sucked worse. Like political elections.

Re:BioWare Nothing But A Shell Of Their Former Sel

[Anonymous Coward]

The only reason KotOR was hyped and won the usual old game of the year awards was because it was on the Xbox.

The same way mediocre games like Dead Rising, Lost Planet, Gears of War, etc. all get hyped. KotOR was mostly given a yawn in the pc world. That is the only opinion that really matters.

One just has to look at the raving Xbox fans at place like EGM and 1Up to see why anything on the Xbox gets reviewed high, given awards, etc.

Re:BioWare Nothing But A Shell Of Their Former Sel

[blahplusplus]

"Neverwinter Nights was also a franchise that was well received and supported by an active community."

The community was the only thing that saved NWN from total disaster, as a game it *SUCKED* and it sucked hard. I would like to say that bioware was never been truly a consistent top tier developer. Kotor was above average, but bioware is pretty hit and miss, since different dev teams for different games do not all produce equally.

Re:BioWare Nothing But A Shell Of Their Former Sel

[Anonymous Coward]

Neverwinter Nights was also a franchise that was well received and supported by an active community.


Neverwinter's community was generally die-hard tabletop RPG fans who wanted to recreate the experience online. They used the DM and builder tools to create single-and multi-player modules as well as persistant worlds that managaed to capture the pen-and-paper experience like no other computer game before or since. You could actually have a living, breathing DM generating custom content on the fly. You could create mini-MMORPGS like Arelith, Forgotten Realms:Cormyr and Layonara where a hundred players could adventure, craft and whatever else on their own and then go on custom and unique DM-driven adventures. Aside from certain MUDS (of course) there really hasn't been anything like it. And hell, it ran on Linux and Mac !

If Bioware was listening, and I mean really listening, they would realize that this was the reason the game was a success, not the lackluster single player campaign or modules (which were only playable for the "hey, I'm playing D&D!" factor, not the story themselves.) There's a real call for a new "online role playing game construction set", D&D or d20 rules or not, to replace it, and Bioware would be just the company to make it. They most likely won't, as they probably think it would hurt sales of their single player games or some future MMORPG, but there is a community and market of role players who are begging for a GM-created and driven experience instead of another linear 40-hour single player story or massive multiplayer grind and gank fest.

(And no, Neverwinter 2 just isn't cutting it as a replacement-- it's too bloated and only runs on high-end Windows systems, and the tools simply aren't there to create modules and worlds on the scale of the original.)

Re:BioWare Nothing But A Shell Of Their Former Sel

[eviltypeguy]

That is your opinion.

I for one sincerely enjoyed the original campaign, it wasn't the *best* I've ever played, but it was very enjoyable for me.

I think too many people say the community is what saved NWN.

Bioware's internal surveys and statistics always indicated that the singleplayer portion of NWN was far more popular.

It's one of the reasons they launched the Premium Modules program (which was very successful until Atari killed it).

Re:BioWare Nothing But A Shell Of Their Former Sel

[mlts]

I do admit that the campaigns in NWN were as not as detailed as the BG ones, but the third party modules and Bioware's excellent support for the game more than made up for it. Player modules like "Tortured Hearts" and such were top notch, not to mention the persistant worlds. NWN is the only game where I bought copies just to support Bioware because the extensibility of the game gave me a lot of hours of entertainment.

NWN2 is a disappointment for me, IMHO for third party modules. Its impossible to put as many areas into a NWN2 persistant world as a NWN1 one, just due to the much larger memory footprint. I'm hoping things will get better with the expansion.

Most games are not works of art.

[EWAdams]

Neither are most movies and most novels. They're light entertainment.

Video games are an art FORM, just as painting is an art FORM, but not every painting is a work of art, nor is every game.

Re:Most games are not works of art.

[lessthan]

Bah, I disagree. I think the best definition of art I've ever heard is "It is art because you say it is art. All that is left is to determine why it is art."

You're confusing "easy" with "mediocre."

[EWAdams]

There's nothing wrong with easy games. The market for them is substantially larger than it is for hard games, and that's why the industry is moving in that direction -- and about time, too. It has treated the less-skilled player with contempt and derision for far too long. You're an old-time hardcore gamer, so you think of easy games as bad ones, but the days when the industry would pander to the hardcore gamer's every whim are over. Don't worry, though, I'm sure a few companies will still make games for your little niche.

Re:You're confusing "easy" with "mediocre."

[blahplusplus]

"There's nothing wrong with easy games."

You're right but the truth is when a game becomes too easy it becomes boring, so there must be some kind of push-pull with the player or gaming becomes a pointless excercise (no risk, no reward). There is such a thing as too easy and too hard. The thing that differentiates games from movies is PASSIVITY and lack of interactivity (taking part). Many modern games are becoming more passive, and hence skewing towards what a game shouldn't be: Passive. We have movies and other forms of entertainment for those people.

"It has treated the less-skilled player with contempt and derision for far too long."

What the heck are you talking about? There have been many games who've tried to go beyond the hardcore crowd and didn't succeed financially or otherwise. MMORPG's in my opinion are the epitome of the mass market mindset: Computer controlled avatars you simply navigate and press menu buttons, everything else is handled automatically.

"You're an old-time hardcore gamer, so you think of easy games as bad ones."

Wrong again, hardcore is such a vague all-encompassing definition with no real specific meaning. I like games of all types because I like hardocore games doesn't mean I don't like "mass market" games. I am passionate about games, there's a difference between respecting someones enjoyment and wanting to understand enjoyment and fun as passion to create better games for different types of people. I analyze why different people enjoy games and what about them they enjoy, just because you enjoy something doesn't mean you are able to judge the quality of the overall development of gaming as a whole.

"but the days when the industry would pander to the hardcore gamer's every whim are over."

What? Zelda TWP and Halo 3, Final fantasy 12 are 'hardcore', in fact the hardcore base is what fueled gamings growth up to this point, it certainly wasn't the barbarian hordes that got gaming to this point.

"Don't worry, though, I'm sure a few companies will still make games for your little niche."

That "little niche" supported the likes of Civilization, Mario, Sonic, Halo, Half-life, etc, name any of histories greatest games and you will see "hardcore" written on it, and by "hardcore" I don't mean it in the sense of deriding other less experienced, casual gamers, etc, (since hardocre vs casual really is vague), I mean it in the sense of pushing the envelope or refining the quality of game development.

Game mechanics matter, I really truly wonder if Wii, PS3 or Xbox could survive without a hardcore audience (i.e. a market that actually BUYS games), and yes games like Halo are definitely hardcore even though they are percieved to be 'mainstream', the truth is the mechanics are still based off quality game design.

The truth is market economics and developing quality games to not mix well, period. You measure a games success by it's audience size or money it makes, this is of course incorrect way to judge value.

It's based on a MARKET VALUE SYSTEM, i.e. how much money can this item attract. While gaming is a 'business' the many people in the business are not in it just for the money.

An excellent example of the market value system can be seen with bill gates comments here, on why millions of people are allowed to die.

Bill gates said:

"So we began our work in the same way anyone here would begin it. We asked: "How could the world let these children die?"

The answer is simple, and harsh. The market did not reward saving the lives of these children, and governments did not subsidize it. So the children died because their mothers and their fathers had no power in the market and no voice in the system."

http://www.news.harvard.edu/gazette/2007/06.14/99- gates.html

Re:You're confusing "easy" with "mediocre."

[EWAdams]

Forgive me for misunderstanding the nature of your complaint, but I disagree with your clarification as well. Different players want different levels of activity. The entertainment world isn't divided into passive (films, TV) and active (games). There's a continuum, and some players want "interactivity-light" games. Demanding large amounts of interactivity is threatening and off-putting to a certain class of players, and developers are starting, finally, to cater for them as well. Others are looking into ambient games, with which the player can interact exactly as much, or as little, as he chooses.

Have a look at www.inanimatealice.com as an example of a game that begins with very little interactivity and gradually requires more from the player with each new chapter.

Bottom line: almost any statement of the form "interactive entertainment should be like THIS" is wrong.

Re:BioWare Nothing But A Shell Of Their Former Sel

[l3mr]

Actually, Bioware is working on this. It's called Dragon Age, will be based in a new, bioware-created world, and will have a full-featured editor from the start. It is not based on d20 but on a newly developed, probably more computer-friendly character+combat system.

Too bad it will take at least another year until its done.

Re:BioWare Nothing But A Shell Of Their Former Sel

[blahplusplus]

"That is your opinion."

As if this statement says ANYTHING about game quality. The Dev team even admitted the SP campaign was pure crap! That right there says quite a lot.

Yes thats right, since I actually own the piece of crap called NWN. It was totally mediocre to anyone who's played RPG's for the last 20 years, so I'd say my opinion has a bit more weight behind it then a D&D fanatic or some newbie gamer with no gaming history under their belt. How one experiences NWN is directly related to the amount of games they have played and what kind of gaming history they have.

I will respect bioware for their valiant attempt in NWN because they really made an attempt at making great tools for the mod community but it ended up making the game itself a mediocre pile of crap and no one with serious gaming acument can dispute this. I really WANTED to like NWN, lots of good things ended up in the game but the gmae really needed another 2 years in development, NWN was like a game who's potential was never truly realized, period.

"I for one sincerely enjoyed the original campaign, it wasn't the *best* I've ever played, but it was very enjoyable for me."

Then you are either:

1) Inexperienced gamer or
2) Have a very low thresh-hold for enjoyment

"I think too many people say the community is what saved NWN."

Give me a break, many people on the forums complained about the utter crappyness of the single player experience compared to their (biowares) previous games like Baldurs gate, BG2, and other infinity engine games not made by them like Torment and Icewind dale. The truth is NWN basically attracted all the really strange D&D fanatics & weirdos who didn't really care much for game quality. The truth is I compare the majority of the people who bought and liked NWN with the likes of people who bought and liked the matrix or 50 cent games, I have the same respect for them as I do say creationists. As hard as that may be to swallow, I know gaming in's and outs I have played nearly every game under the sun and I analyze games to find out what makes them percieved as "good" to different gaming demographics. The truth was the barrier to playing NWN was very low, since it took a page from MMO's with their automatically controlled gaming mechanics (i.e. the user just pushed menu buttons), WoW and NWN have the exact same mindless interface for the drooling hordes, which I can understand from but the content simply could not make up for the borefest of crappy single player content.

And yes the community did a lot to save NWN, most of my enjoyment from NWN came from playing other peoples mod's and the single player and sp campaign was truly horrible. The fact that you had one single character you could truly control in the party which was handled automatically (robot avatar) made for a tenuous and boring game with little for the player to do then sit and drool through the SP campaign.

If you enjoyed NWN over those prior games, I have to say you are among the reasons why gaming is taking a nosedive in quality to serve the mass audience of mediocrity.

"Bioware's internal surveys and statistics always indicated that the singleplayer portion of NWN was far more popular."

Of course but because something is popular doesn't make it good. I'm not the only one that feels that gaming is backsliding into mediocrity.

http://www.gamespot.com/users/Cube_of_MooN/show_bl og_entry.php?topic_id=m-100-25099957

Why is pandering bad?

[Anonymous Coward]

The question of whether or not it's pandering like mainstream movies carries the implication that pandering to the target consumer is inherently bad. Why do so many of us hold this perception?

If you can make your product appeal to a greater base of people, why is that a bad thing? Sure, you might be "dumbing it down", but you're making it accessible and enjoyable to a greater number of people. This means more money for you, the developer/publisher/producer, and it means more people enjoyed whatever it was you were producing. Sure, there are hardcore fans or the intellectual elite who are upset that their favorite movie/game got "dumbed down" so the lowest common denominator could enjoy it, and while they might justifiably ignore or dislike the product as a result, the simple fact is that more people liked the change than didn't.

And we give directors/developers crap about this? Who are we to demand that they make it the way we prefer, rather than the way that will be preferred by a greater number of people?

Re:BioWare Nothing But A Shell Of Their Former Sel

[eviltypeguy]

"The Dev team even admitted the SP campaign was pure crap! That right there says quite a lot." The Dev team said no such thing as far as I am aware. Please provide your source for this statement. Then you are either: 1) Inexperienced gamer or 2) Have a very low thresh-hold for enjoyment Assumptions tend to get people in trouble. I have been gaming on the PC since the early 1980's. My threshold for enjoyment is "not very low" -- it's just that I happen to thoroughly enjoy the story and artwork in the original NWN. I have hundreds of PC game titles on my shelves, some good, some bad.

I never said I enjoyed NWN "over" prior games, just that I enjoyed it. Although there are some prior games I didn't enjoy as much. For example, I never could bring myself to finish Baldur's Gate I, even though I finished NWN I. Why? I just wanted to play it for the story and quests, I didn't want to control each individual member of my party. Your preference to do so, and insinuation that it is a inclination towards mediocrity is a opinion not shared by the general gaming public at best.

Before you accuse me of not understanding or being a true D&D fan, yes, I do have the Player's Handbook, etc.

Of course but because something is popular doesn't make it good. I'm not the only one that feels that gaming is backsliding into mediocrity.

Businesses exist to make money. Game developers make games. Therefore it follows that they make games their public wants to play because that makes money. If people want to play games that folks like you see a mediocre, then more power to the game developers for making them. Life is too short to take things that seriously. Fun is fun. There's a reason that Nintendo is so successful right now, and its because they aren't catering to Ivory Tower gamers like you.