Lies, Damned Lies, And Gaming Statistics

Thanks to the IGDA for pointing to a recent Chris Crawford-authored piece discussing the unreliability of statistics that reveal an ever-more diverse gaming public. The piece cites a recent Entertainment Software Association study and suggests that, because they don't reveal their methods or sources, "...the ESA results are unscientific. We can't place any confidence in them because we have no idea what they really mean. It doesn't matter how representative the sample is or how large it is or even if the researchers wore white lab coats." The ever-controversial Crawford goes on to argue that "Games people are smarting about the tawdry, unhealthy image that their industry suffers", and then offers a way out: "What's especially sad about this is that the solution to these problems is obvious: start making respectable games."

Comments

It's nice to see people complaing

[M3wThr33]

Numbers are critical to a game's success and in the past the industry has got the shaft in tracking sales for the most part. The industry-dominating NPD TRSTS don't even include Wal-Mart. Who's fault is that? No idea, but we really need to start modeling Japan's method. Each week they have it down pat.

Aye, there's the rub.

[fuzzdawg]

If you look at the most popular games of all time, (I have in mind Tetris, Final Fantasy, Unreal Tournament, GTA, Pac-Man, The Sims, WarCraft, Mario, well, you get the idea . . .) they are all fun. That's the point. But they are all respectable in their own right.

The reason violent and brutal games are fun is because games involve conflict, and the most basic type of conflict is physical violence/confrontation. It is also the easiest type of conflict to understand, and resolve. (Guy is in my way, kill him)

It is much easier to make a game where you have to kill all your enemies, as opposed to make them loyal and win their friendship and love. And besides, most of us do the more complex task in our daily lives anyway.

As far as comparing books and film to video games, the problem is interactivity. A person doesn't make choices in the two proven genres, but games are participatory. The future of game development is going to be intresting, because current game sales have been based on improved technology; not improved gameplay. (ex: UT v UT2003) The proven game genres are going to have to take a back-seat, and be modified to make some fresh ideas. Who would have thought games such as The Sims would have been so successful? What the game business needs is innovation, not this repetitive stagnation.

How that is going to come about has yet to be seen -- I don't have any ideas just yet.

Re:Aye, there's the rub.

[PainKilleR-CE]

If you look at the most popular games of all time, (I have in mind Tetris, Final Fantasy, Unreal Tournament, GTA, Pac-Man, The Sims, WarCraft, Mario, well, you get the idea . . .) they are all fun. That's the point. But they are all respectable in their own right.

Almost half of those games don't even rank in the top 20, while games that do (Half-Life, Myst) aren't there. Mario accounts for about 25% of the top 20, though.

It is much easier to make a game where you have to kill all your enemies, as opposed to make them loyal and win their friendship and love. And besides, most of us do the more complex task in our daily lives anyway.

Civilization (and most of it's sequels and derivatives) can be played this way. Obviously, though, it has a more niche appeal than the others. Of course, a handful of the top selling games involve no conflict at all, such as Tetris and Myst.

As far as comparing books and film to video games, the problem is interactivity. A person doesn't make choices in the two proven genres, but games are participatory.

This is also why games tend to get hit by controversy and that controversy tends to stick better than it ever did with music, movies, and books. This is also why controversy stuck with pen & paper RPGs. Still, it has little to do with whether or not they can be considered art, as that's a matter of the people working on them treating them as such.

The future of game development is going to be intresting, because current game sales have been based on improved technology; not improved gameplay. (ex: UT v UT2003)

As far as I knew, UT outsold UT2003, and Unreal outsold UT. Half-Life outsold Quake 3 by quite a bit. Super Mario Bros 1 outsold all other Mario games (though SMB3 is the best selling game never officially bundled with hardware). Doom 3 might be the exception, if it does well, because it has a better name in the mainstream than Quake. There are far more non-gamers that played Doom than Quake.

The proven game genres are going to have to take a back-seat, and be modified to make some fresh ideas. Who would have thought games such as The Sims would have been so successful? What the game business needs is innovation, not this repetitive stagnation.

As with anything else, the stagnation will remain. Movies certainly suffer from it, music suffers greatly, and books slightly less so, but still to a degree. Occasionally there are those rare games that surpass the current genres and/or bring in mass appeal. I am much happier to see Half-Life on the 'top selling games of all time' charts than Myst or the Sims, because it shows that an existing genre can appeal to the masses with some effort on the part of developers. Too many people are complacent to keep doing the same old things, to try to develop something cheaply that will bring in quick cash. The Sims is actually a solid example of this, as once the actual game took off they kept turning out expansion packs that required very little change to the game itself, and brought in tons of cash. Of course, Valve is nearly as guilty with Half-Life, turning out a dozen or so versions of the game over the past 4 years and selling mods developed by fans of the game (though, of course, those fans made a pretty penny in most cases, too).

People tend to go too far into the mindset of 'we need something completely new' instead of realizing that we just need people to tell new stories and create technology that tells the stories better. Even Half-Life's story was fairly derivative of the rest of the genre, with only a few twists and changes, but the technology they developed to tell the story made it a best seller. Of course, Quake 3 was considered better from a technological standpoint, but it's technology was focused around flashy eye-candy rather than story-telling, which is why it came nowhere near the sales numbers.

On the other hand, what are the SMB games really about? Those are the best-selling ga

Surveys of your friends

[ElleyKitten]

This is why you can't take surveys of your friends. If I did the same thing, I'ld get the reverse results, because almost everyone I know plays games more than they read books or go to the movies. Granted, maybe the people I know are wierd since most of them never see movies, but no wierder than his friends, many of which said 'never' to the video game question.

He does make a good pointabout how the ESA survey is flawed, but then he tries to make his survey sound better, and then he falsely accuses the game industry of being 'tawdry'.

For every Betty Bigtits there's a Mickey Mouse [movie]

There are plenty of Mickey Mouse games, way too many I'ld say. Go to Gamestop, look at the PS2 shelf, and count how many Disney games there are. Then count how many "Betty Bigtits" there are. There's one that comes close, BMX XXX, and it's probably not on the shelf anymore because it sucks. It was the game equivalant of "Gigli" for its popularity. There's no other games (Right now, the Lesuire Suit Larry series and the Atati porn games count as much as a 20 year old porno) that come close in terms of 'tawdryness'. Ok, there's Dead or Alive beach volleyball, but there's no nudity, and, unlike any porn movie, it provides fun besides just looking at the girls (girl who played it for months talking her!).

Finally he talks about how the game industry doesn't have enough independent games. While this is a problem, it can't be solved just by telling the game companies to support independent people. Where a book, or even a movie can be made by someone on their own, a game, a full-fledged mainstream competitive game, cannot be. Anyone can write a book, and, to a lesser extent, make a movie that could be competitive (watch "Clerks" sometime, you can tell that the guy had no money when he made it, but it's still a movie you'ld rent from Blockbuster like any other, unlike an independent game you'ld pick up because it's cheap). Untill any random person can make a game, there will be no good independant games.

There is a pretty big indie games scene.

[illumen]

Very interesting article :)

"""The games biz has nothing like this; we have hobbyists who slap together amateur efforts, and we have big-time people who develop the mainstream products. There's nothing in between, which is one reason why the games industry is so creatively stagnant."""

I do not think that is so true. There are a few independant groups which make wildly popular games. eg Doom, snood, various other 'indies'. Over the last few years the indie game scene has really taken off. I think these people make games for a lot of people not into the mainstream games. For example many of them make games which will run on many computers, not just the latest ones.

A game like snood, for example has gotten very wide public attention(In the multi millions). I've meet a number of non gamer types who have played it. The same with the doom/quake series of games. People who didn't normally play games at that time gave it a go because it was so interesting.

Indie game developers can also afford to make games which do not sell lots of copies, because they are running more efficient businesses. For example instead of making 3-5% of each copy sold they can make 70-90% of each copy sold, and generally make games with much smaller budgets. They rarely do not sell as many copies as the publisher distributed.

Take a look at the igf.com, dexterity.com/forums/. From there you will find lots of indie games, and evidence of such a scene. Check out gametunnel.com, indiegammer.com for reviews of indie games. There are really good original games in there, but also a number of crap games(like other indie, and publisher funded entertainment areas). Also check out ludumdare.com where there are occasionally 48 hour game making comps. Some very interesting ideas have come from there.

Something like the Harvey Keitel effect*1 in pulp fiction is happening in other indie games. One occurance is where one/some of the makers of ufo XCOM are selling a game: Laser Squad Nemesis, http://www.lasersquadnemesis.com/.

*1 Harvey Keitel effect - Having a big name giving more reputation, expertise and funds to an almost unknown group, drawing more attention to it. Like what happened in pulp fiction with Harvey Keitel. Because people knew he was on board, other people agreed to work with them.

Have fun!
holepit.com

Re:There is a pretty big indie games scene.

[tigermonkey]

I do not think that is so true. There are a few independant groups which make wildly popular games. eg Doom, snood, various other 'indies'. Over the last few years the indie game scene has really taken off. I think these people make games for a lot of people not into the mainstream games.

Yeah, but wildly popular in terms of those who have access to the 'Net, and know where to look. How many indie games do you see sold in mainstream stores (like the big retailers) or even specialty stores (like EB or Gamestop)?

Most, if not all, of the games stocking the shelves are from established publishers, or from large corporations (primarily, Microsoft, but once upon a time, IBM did sell games); until indie games can be picked up with some frequency at some place like Target or EB, I don't think indie games will be able to assume a 'mainstream' presence, and I think this is what Mr. Crawford was talking about: making the smaller games more visible at retail level.

Using movies as an example, big studios do distribute smaller, more experimental films; there are also new studios which are willing to take a chance (IFC Films springs immediately to mind). The situation in video games is not the same: either a large publisher is publishing your game, or your game gets buried in a 'shovelware' pack.

Without a big game studio behind an indie game, I don't think the good indie games are going to get the kind of exposure that, say, an indie film or indie music group/album would get. You have whole channels devoted to promoting new, not necessarily mainstream music (MTV2 springs to mind) and new, not necessarily mainsteam film (IFC and Sundance, for example); in the gaming world, you're limited to something like G4...which will only talk to you if you're a big studio or a console developer (at least, I haven't seen a game from, say, Garage Games or Matrix Games on that channel yet). And I think there is enough of a game-playing public which doesn't log onto the Internet religiously which will miss out on the indie game scene, unless they somehow find out where to look for indie titles...

And if a big studio's PR/marketing efforts got behind some kind of clearinghouse for indie titles or some kind of collective of smaller game studios, I think that the gaming public-at-large would have a much better idea of where to start looking for indie games...as well as improving visibility on the retail level.

monkeynova

Errr... what?

[DarkZero]

This person seems to be witnessing a completely different reality than myself and all those around me. We've either discovered travel between parallel dimensions or one of us is nuts.

There's another consideration here: why are such polls so common in the games biz? It's not just a marketing slant; everybody in the games industry passionately believes this nonsense. Question this dogma and you'll be regaled with anecdotes of unlikely friends and relatives who are avid game players:

Who passionately believes this? Not only are these statistics the biggest joke on every video game message board, chat room, or website I've ever visited, but if I remember correctly, even gaming magazines like EGM or TV shows like X Play have laughed them off. They're nothing but marketing propaganda and I have yet to see anyone take them to heart outside of a couple of ignorant, overzealous bloggers.

What's especially sad about this is that the solution to these problems is obvious: start making respectable games. There is absolutely no technical or design reason why games have to be so tawdry. Sure, the primary market of adolescent males much prefers the violent rebellious stuff, and the industry can continue to make billions supplying this market. But the games industry needs to learn a lesson that the movie industry and the book industry learned long ago: you gotta push the noble fringe. Every year there are dozens of high-minded movies that get funding. Most lose money, but a few break even. Hollywood is willing to fund these efforts for three reasons:

1. They support the image of cinema as an art form worthy of respect. Sure, Hollywood makes tawdry movies; moviemakers are certainly not above showing some skin to boost sales. But while there are plenty of individually sleazy movies, Hollywood's output taken as a whole is a healthily heterogeneous collection. For every Betty Bigtits there's a Mickey Mouse; Koyaanisqatsi played just down the road from Debbie Does Dallas.

See, this is where we really get into that "parallel dimension" stuff. This man appears to be from a world where games like Grand Theft Auto make up the majority of the titles available. A world where there is no Ico, Mario Sunshine, Wind Waker, Jak & Daxter, Sly Cooper, Ratchet & Clank, Sonic Adventure (and soon Sonic Heroes), Smash Bros., Crash Band--- y'know, I'm gonna stop here, because with every platformer review I've read lately complaining about the "glut of platformers" in the vein of Ratchet & Clank and Wind Waker pushing out all of the other cutesy platformers that are worthy of being played, I could go on for several paragraphs.

This author is apparently also from a world where the top ten list for American game sales isn't frequented by sports titles and platformers, but rather all of these "tawdry" games. In short, a world where BMX XXX sold more than two copies and returned copies of Enter the Matrix aren't cozying up next to E.T.: The Extraterrestrial for the Atari 2600 in a New Mexico landfill. A world where the game that GTA3 dethroned as the king of the North American sales charts for months on end was another tawdry, bloody game, rather than Gran Turismo 3 A-Spec. And, finally, a world where no one has tried out unusual, nonviolent games like Disaster Report (Zettai Zetsumei Toshi) and watched them fail miserably along with Ico, Rez, and Ikaruga, or occasionally watched them soar, like Gran Turismo 3 A-Spec, Dance Dance Revolution, or this little unknown game that's only released a couple of expansions... what was it called again? Oh yeah, "The Sims". I heard that game sold a couple of copies.

I live in a world where there are not only plenty of respectable games, but at least as many of them as the bloody games. The only place where that differs is in the media, where nightly readers/viewers were once regaled with equally innacurate depictions of the comic book and movie industries, and where the only thing that will change that is the passage of time.

Re:Respectable Games?

[easychord]

I'm going to sound like horrible Nintendo fan saying this, but you could have bought a Gamecube instead. At least, Nintendo *try* to make games for all ages.

I don't see why you would have problems with games like Mario Sunshine, Pikmin or Super Monkey Ball.