Slashdot Mirror
Game Reviews Don't Matter, Study Finds
Next Generation has an article up looking at a report from SIG, on the correlation between game reviews and sales. Their findings indicate that, while reviews obviously do have some effect on games sold, there just isn't that much of a correlation. From the article: "He said he doubted that publishers and PRs would stop caring about review scores, especially as they matter a lot with consumers who compare games from the same sub-genre — say, basketball games. But he said that, as with last year's report, the report's findings are unlikely to be popular. 'We received a lot of attention but the stats do not lie,' he said."
I tend to read about games through, for example, their website, previews, beta-tester forums, etc. I make a decision about whether or not a game is worth my money on my own, go out and buy it, play it, then think "hmm...that was [fun/stupid]. I wonder what other people think about it." Then and only then do I go and read reviews.
Anyone else think the comments just weren't rendering right before they turned off ABP and saw ads?
Game reviews do matter - its called word of mouth. Terms such as "clickfest", "pushover" and "tedious" are negative aspects of a game that you should avoid.
If you want a list of things that a good review should look for, all you have to do is find lists of Cliches and reviews that make note of them. There are similar lists for strategy and action games - but common components among all such lists involve being railroaded through events outside of the players control (e.g. is captured by 3 units after taking out 2000 soldiers), or events that are obvious enough to be traps but the player is forced to go through them to advance the plot.
Let me tell you a story. There was a game I had been long anticipating, a game I was so psyched about. I was, I must admit, young and naive, I had never played a truly bad game before, I didn't think 'bad' games existed, other then the Edutainment games. I got the game the day it came out. The game was Superman 64. I have never bought a game (other then sequels and launch titles) without reading a review ever since.
Perhaps I'm in the minority, but I always check the reviews before buying a game, and that silly little number does impact me. I am much more likely to buy a game with a score above 9 from the magazine and a similar user score than a game that scores below 9 in both categories. I tend to rely more on the player review numbers though, and ultimately use them to make my final decision. Actual player comments are also very nice. The whiners and the fanboys are easy to filter out, and a lot of people leave good comments about core game features that will have an impact on me. If I see the same complaints or cheers about an aspect of the game from several people, I tend to take those opinions very seriously. Likewise if I read a dozen comments about the game crashing and two comments about how the game never crashes and is awesome, then I'm going to assume the game has some issues and I might wait for the first round of patches. (I guess I'm mostly referring to gamespot, because it happened to be the game site that I found first however long ago and I don't feel like following multiple sites)
Here's an embarassing thing to admit: if I've been looking forward to a game and it gets an 8.9, I actually feel disappointed and I might take a few weeks to make a decision. But if that game gets a 9.0, I'm much more inclined to run out and grab it. Yeah, it's dumb. It's just like I know that $49.99 is only a penny away from $50, but it feels a lot more like $40.
One thing I realized as I was typing this is that while I'll look at the points the reviewer gives a game, I rarely ever read a professional review. They're simply too wordy and glossy. I'd rather just read some player comments about how "it sucks ass that you can't ride the donkey in multiplayer mode" than to try to discern these tidbits from the professional review.
Sales are, at least in my experience, based on three factors:
Name of Game/Company
Hype
Popularity of genre
In that order.
There's quite a lot of really awesome games out there that didn't get any attention. Why? They lack at least 2 of the three factors.
Imagine something is labeled "Command & Conquer Final Chapter". Will it sell? You bet it will! It could be the last crappy ripoff, a repackaged C&C1 with new graphics for example (no, you don't need to tell me "it's already been done", I know, thank you very much). It will sell. It has the name. Imagine Halo 3. Quake 5. NHL 2006. Diablo 3. Does/would/will it sell? Yes. Why? Name.
Same with "games to movies". Fortunately, at least some movie studios do care these days who buys the license for the name to slap on a game. It used to be a surefire way to say a game sucks donkey bits if it had the name of a current movie. But they always were bestsellers. Even the crappiest of the lot.
Then there is hype. DNF will sell. No matter when it comes out and no matter if it is up to par. You could offer an empty box and it would sell. Other games have the "new feature" hype, whether that feature actually matters or not. Max Payne was hyped as "revolutionary" in the genre of shooters for its bullet time and the actually rendered bullets. Whether it really was, I leave to the reader.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
[ARGH. Lost my typing (emacs bindings don't *always* work in browser text boxes. Shortened version follows]
This was why the OLD Computer Gaming World avoided giving numeric/star ratings for so long.
Some people consider different things when buying. Some look at a tie-in/license (parents, younger kids, non-"gamers"). Some look at genre/subject: WWII, evil-alien-shooter, cute ponies, horror. Some look at whether text of reviews (is it monotonous, etc). Some look for eye-candy or cool tech; game is secondary (anyone remember "Trespasser"?). And some look at the numeric rating closely - but not many, I'll bet.
My guess is that of those who even look at reviews at all, most use the numeric rating just to group it into 3 bins: Run like the plague, it's ok to look at other factors, and (occasionally) look at it even if you normally wouldn't. And for those who find a game first then check reviews, the 3rd case is out, so it's basically only a "look for warning flag before pulling trigger" check.
Who buys "Barbies playhouse" because it got a 8.8 instead of "Kill nasty aliens with cool guns" which got a 7.8? No one. Some may buy Barbies playhouse because that the sort of game they like. And others might NOT buy KNAWCG when they normally would because it got a 4.8, or because if KNAWCG got a 7.8 and "Kill nasty Nazis with cool guns" got a 9.5. But the point is that subtleties of score have almost no impact, and even gross differences of score have only moderate impact except at the very extremes.