Slashdot Mirror
Investigating Bias In Videogame Review Sites
jvm writes "We've all read comments that some videogame sites are allegedly biased for
or against some games, consoles, or companies. So, Curmudgeon Gamer has investigated whether bias can be seen in the review scores
over several games on each console. The review sites in question are GameSpot, GameSpy, and IGN, each of whom are compared to the game
review averages on GameRankings.
Additionally, a selection of review scores for crossplatform games are
examined. While solid conclusions are difficult to draw and improvements can
admittedly be made, perhaps people will find these results interesting to examine and
discuss."
I have to say, from the game reviewer's standpoint, that it's hard to give a game a bad rating.
1) You have advertisers. Typically, these people keep your site alive. They're expecting you to review their game.
2) You like games.
3) You get tons of swag, press passes, the royal treatment at trade shows, and a ridiculous amount of geek cred.
4) Did I mention that you like games?
5) You get to mention Sanya Thomas' butt in casual conversation. (see?)
6) It's a freaking game. What's not to like about a game? (well, unless it's Final Fantasy XI -- which I refuse to admit is actually a game)
Mystery@Warcry.com
Unless you were just born with the knowledge you'd have in 35 years but had no opinions, all reviews are gonna be a little biased, i mean cmon, reviewing games arent a mathematical thing, its giving your opinion.
You forgot the Editor... the position where the one with the Big Thumb can thump a candidate review back to draft status for adjustments. The EIC is responsible overall for a publication, and can be 'coached' to tell his team how to play.
It's mildly interesting to see that there may be an anti-GameCube bias in some reviews. However, what I would much rather see is the same methodology applied to other factors, like who the publisher is and who is advertising with a specific site. I'd never heard anything about specific-console bias, but I've darn well heard an earful about EA getting a free-ride on steaming piles like Earth & Beyond. I'd like to see whether the numbers back that up.
Bite the hand.
because you aren't doing anyone a favour.
If I buy a $50 game based on your review then I can never trust your opinion again.
Regular readers should notice that you never give bad reviews.
If you can't be honest then you are *just* advertising not "relying on advertising".
There are places where the networks are not touching,and there are places where they are-Boeing's Lori Gunter
Curmudgeon Gamer's examination is certainly an interesting one, but what it complete neglected was the trend of increasing stupidity in gaming reviews.
Perhaps the slow but steady rise of ludology means that we may yet see a rise in the quality of game reviews, but by and large the current level of intelligence in most game reviews is as low as the number of female companions most of the game reviewers have (and I mean that collectively). Reviews inevitably follow the "Background, Cool Trivia About Game, Graphics, Controls, Bad, Good" template. The size of the font of the scores at the of end review keeps getting bigger, as does the size of the screenshots. Basically, we're left with an industry wide standard of two paragraph GamePro-esque reviews that generally boil down to, "It rocks!" or "It sucks!" EGM's recent redesign is a fine example of this. There is little in the way of anaylsis, or the game's relation to the industry at large, or (heaven forbid) society itself. I think there are sites on the net that are the exception, such as GameCritics.com, and ironically, Penny-Arcade. Likewise, I think either Computer Games or CGW (I always get them mixed up) is making a conscious move in the other direction. Kudos to them, and shame on the rest of the reviewers. I suppose that online is, strangely enough, more "textual" than the magazines due to the lack of space confinement. But the actual content is about as low.
What I'd like to see is a complete abolishment of scores all together in a magazine or website. It would mean, yes, you'd actually have to read the review. That's not to say there's no place for the blurb-y Gamepro-style of reviews with a big thumbs up or thumbs down at the end of the review. There obviously is, as it's quite handy for a quick reassurance when you're in the line at EB almost ready to buy the game.
But there needs to be more peer reviewing, more intelligent discoursing, and more analysis ala ludology in the gaming industry. Not everything is the next best thing since, well, the last game the person reviewed. But some things most definitely are, and it's these that need extra attention. Not, contrary to popular belief, more screenshots.
that can be drawn is that game players are harsher than the professional critics. no duh. visit any official or psuedo official forum for a game, and chances are you'll encounter a rather vocal group complaining about problem X. never mind that 95% of the players don't encounter problem X, but the 5% that do, really want to shout about it. I imagine the same thing would happen with reviews. One is much more likely to bitch about a game that was less than satisfying than praise one that was enjoyable (as you get satisfaction from the game in the latter case, and are satisfied by bitching about it in the former). So the scores are biased in the downward direction. that isn't too surprising.
I forgot to mention Play magazine. I don't particularly always agree with their reviews; I think that Gunvalkyrie is difficult not because of content but because of poor control design, and so on and so forth. I feel that at times Play tries too hard to be "hardcore." But Play magazine provides a niche, instead of trying to be *THE* magazine for gamers as [insert game magazine name here] is so fond of doing. You have to admire that.
Likewise, I thought the reviews in the defunct NextGen were absolutely brilliant writing. I need to be careful in saying this, because they were notorious for their doublepaged screenshots, but they really did see games as an integral part of society. However, that's what we need in the gaming review industry: great, creative writing. It needs to be an art unto itself. I see a long but straight line between the rock music reviewing of the late sixties, early seventies, and the gaming industry reviewing of today. Rock music was coming of age as the game industry is very much doing today; it's growing beyond the size of its own knickers, so to speak. Likewise, both industries are highly release driven, and every new release in either rock or in gaming was the "next big thing." There's a general feeling that reviewers are "cool" now.
That said, every game reviewer, print or otherwise, should have to watch Almost Famous, and they should walk away with the words, "Be honest, and be ruthless" etched in their minds coupled with the insistence that "We are not cool. We are the uncool." If game reviewing wants to grow up, the reviewers themselves need to realize that the game designers and publishers need to hate reviewers. The reviewers need to be the enemy. That passion, combined with the belief that one's own writing and reviewing can be artistic in nature, is enough to revolutionize the gaming media. Will it happen? That's up to them.