Slashdot Mirror
Game Reviews Don't Matter, Study Finds
Next Generation has an article up looking at a report from SIG, on the correlation between game reviews and sales. Their findings indicate that, while reviews obviously do have some effect on games sold, there just isn't that much of a correlation. From the article: "He said he doubted that publishers and PRs would stop caring about review scores, especially as they matter a lot with consumers who compare games from the same sub-genre — say, basketball games. But he said that, as with last year's report, the report's findings are unlikely to be popular. 'We received a lot of attention but the stats do not lie,' he said."
I tend to read about games through, for example, their website, previews, beta-tester forums, etc. I make a decision about whether or not a game is worth my money on my own, go out and buy it, play it, then think "hmm...that was [fun/stupid]. I wonder what other people think about it." Then and only then do I go and read reviews.
Anyone else think the comments just weren't rendering right before they turned off ABP and saw ads?
n/t
popularity is usually based on marketing, not on reviews.
sucky games (dirge of cerberus) do well with good marketing (commercials, ads, et. al.). good games with little/no marketing don't usually do nearly as well (not that they don't do well at all, just that they lose some of their potential).
a large enough portion of the market doesn't read reviews and bases their purchases off of the "coolness" factor of the game, instead of the quality. if a commercial or ad can make the game look cool, then they're all over it.
'We received a lot of attention but the stats do not lie,' he said."
What was that saying again? Something about lies, damn lies, and statistics?
DYWYPI?
The interesting thing here is that the study only looked at reviews versus sales.
It said nothing about reviews (written by magazines, etc) versus consumer opinions (or user reviews). It also said nothing about consumer opinions versus sales.
In my experience, reviews written by reviewers generally align pretty well with what consumers think of the game, while sales have little to do with either. In short, sales seem to have little to do with how good a game actually is. Sales seem to correlate more with things like movie and cartoon and brand tie-ins to a game, distribution methods, price point, and other such factors. All of this is really no big surprise, since the game industry has always successfully relied on churning out such drivel and it has obviously worked pretty well for them most of the time.
A more interesting thing to study would be what percentage of sales are purchases made by people who know nothing about games and won't be playing the games themselves... such as parents and grandparents choosing games as gifts for kids, etc. I bet they make up more than 50% of sales.
Remember when the Atari era went bust and the bottom of the video games market completely dropped out? My theory is that it was because the industry stopped creating any good-quality games, having realized from experience that they could just produce well-branded crap and rely on all those gullible non-gamer sales. I think the problem is that when the market floods with crap, the gamers (who ultimately receive those games from the purchasers) completely lose interest in games and stop asking their parents to buy them more. So then the purchasers stop buying completely.
In other words, a sufficient minority of titles must continue to be of good quality for the industry to sustain itself, but once that sufficient minority is met, the rest can be crap and the industry can thrive off the crap. The industry then foolishly thinks all it needs to produce is the crap, which kills demand completely, which kills the whole industry.
Moderator hint: a comment is neither "Flamebait" nor "Troll" if it is true.
Game reviews do matter - its called word of mouth. Terms such as "clickfest", "pushover" and "tedious" are negative aspects of a game that you should avoid.
If you want a list of things that a good review should look for, all you have to do is find lists of Cliches and reviews that make note of them. There are similar lists for strategy and action games - but common components among all such lists involve being railroaded through events outside of the players control (e.g. is captured by 3 units after taking out 2000 soldiers), or events that are obvious enough to be traps but the player is forced to go through them to advance the plot.
Newsflash: Can't predict the future based on one criteria.
Hell, the article doesn't even say that there isn't a correlation between good reviews and good games, just that it's not a reliable predictor of sales. Well duh. Maybe things like number of consoles in the market for that game, or marketing, or whether its a sequal, or the price, or whether it's released in May or during the holiday season, might all play a role.
We expect that the same game with good reviews will perform better than that same game with poor reviews. The article confirms that expectation, while trying to sound like it's conclusion is surprising. It's not.
Some people will buy a Pokemon game no matter how bad the review is. This is obvious. Doing a study that confirms it doesn't change that it's obvious and your study is just an excuse to fill some pages under the guise of 'news'.
paintball
Let me tell you a story. There was a game I had been long anticipating, a game I was so psyched about. I was, I must admit, young and naive, I had never played a truly bad game before, I didn't think 'bad' games existed, other then the Edutainment games. I got the game the day it came out. The game was Superman 64. I have never bought a game (other then sequels and launch titles) without reading a review ever since.
which is what the study concludes, and what should be pretty obvious if you read lots of game reviews. Game ratings are basically random numbers between 8 and 10, and where it falls in that range seems to be largely divorced from the content of the review. How many times have you read a review that said something like "the gameplay was fun for the first few levels, but quickly became monotonous and boring" but gave the game a 9.5, or one that said "despite a few minor flaws, this game is all around a lot of fun" and gave it an 8? When I'm out looking for a game, I think I'm going to weigh "monotonous gameplay" a lot more than "Overall Score: 9, Excellent!".
I can understand them using the game rating, as it's the only obvious number you can apply to a game review and do correlations with. However just having a number doesn't mean it actually represents something, and I'm not surprised that game sales don't correlate well with a number that is basically pulled from the reviewer's ass.
The enemies of Democracy are
Perhaps I'm in the minority, but I always check the reviews before buying a game, and that silly little number does impact me. I am much more likely to buy a game with a score above 9 from the magazine and a similar user score than a game that scores below 9 in both categories. I tend to rely more on the player review numbers though, and ultimately use them to make my final decision. Actual player comments are also very nice. The whiners and the fanboys are easy to filter out, and a lot of people leave good comments about core game features that will have an impact on me. If I see the same complaints or cheers about an aspect of the game from several people, I tend to take those opinions very seriously. Likewise if I read a dozen comments about the game crashing and two comments about how the game never crashes and is awesome, then I'm going to assume the game has some issues and I might wait for the first round of patches. (I guess I'm mostly referring to gamespot, because it happened to be the game site that I found first however long ago and I don't feel like following multiple sites)
Here's an embarassing thing to admit: if I've been looking forward to a game and it gets an 8.9, I actually feel disappointed and I might take a few weeks to make a decision. But if that game gets a 9.0, I'm much more inclined to run out and grab it. Yeah, it's dumb. It's just like I know that $49.99 is only a penny away from $50, but it feels a lot more like $40.
One thing I realized as I was typing this is that while I'll look at the points the reviewer gives a game, I rarely ever read a professional review. They're simply too wordy and glossy. I'd rather just read some player comments about how "it sucks ass that you can't ride the donkey in multiplayer mode" than to try to discern these tidbits from the professional review.
Sales are, at least in my experience, based on three factors:
Name of Game/Company
Hype
Popularity of genre
In that order.
There's quite a lot of really awesome games out there that didn't get any attention. Why? They lack at least 2 of the three factors.
Imagine something is labeled "Command & Conquer Final Chapter". Will it sell? You bet it will! It could be the last crappy ripoff, a repackaged C&C1 with new graphics for example (no, you don't need to tell me "it's already been done", I know, thank you very much). It will sell. It has the name. Imagine Halo 3. Quake 5. NHL 2006. Diablo 3. Does/would/will it sell? Yes. Why? Name.
Same with "games to movies". Fortunately, at least some movie studios do care these days who buys the license for the name to slap on a game. It used to be a surefire way to say a game sucks donkey bits if it had the name of a current movie. But they always were bestsellers. Even the crappiest of the lot.
Then there is hype. DNF will sell. No matter when it comes out and no matter if it is up to par. You could offer an empty box and it would sell. Other games have the "new feature" hype, whether that feature actually matters or not. Max Payne was hyped as "revolutionary" in the genre of shooters for its bullet time and the actually rendered bullets. Whether it really was, I leave to the reader.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.