Game Reviews Don't Matter, Study Finds

Next Generation has an article up looking at a report from SIG, on the correlation between game reviews and sales. Their findings indicate that, while reviews obviously do have some effect on games sold, there just isn't that much of a correlation. From the article: "He said he doubted that publishers and PRs would stop caring about review scores, especially as they matter a lot with consumers who compare games from the same sub-genre — say, basketball games. But he said that, as with last year's report, the report's findings are unlikely to be popular. 'We received a lot of attention but the stats do not lie,' he said."

Comments

I read reviews as an afterthought

[Enoxice]

I tend to read about games through, for example, their website, previews, beta-tester forums, etc. I make a decision about whether or not a game is worth my money on my own, go out and buy it, play it, then think "hmm...that was [fun/stupid]. I wonder what other people think about it." Then and only then do I go and read reviews.

marketing

[aleksiel]

popularity is usually based on marketing, not on reviews.

sucky games (dirge of cerberus) do well with good marketing (commercials, ads, et. al.). good games with little/no marketing don't usually do nearly as well (not that they don't do well at all, just that they lose some of their potential).

a large enough portion of the market doesn't read reviews and bases their purchases off of the "coolness" factor of the game, instead of the quality. if a commercial or ad can make the game look cool, then they're all over it.

Sales != A good quality game

[c0d3h4x0r]

The interesting thing here is that the study only looked at reviews versus sales.

It said nothing about reviews (written by magazines, etc) versus consumer opinions (or user reviews). It also said nothing about consumer opinions versus sales.

In my experience, reviews written by reviewers generally align pretty well with what consumers think of the game, while sales have little to do with either. In short, sales seem to have little to do with how good a game actually is. Sales seem to correlate more with things like movie and cartoon and brand tie-ins to a game, distribution methods, price point, and other such factors. All of this is really no big surprise, since the game industry has always successfully relied on churning out such drivel and it has obviously worked pretty well for them most of the time.

A more interesting thing to study would be what percentage of sales are purchases made by people who know nothing about games and won't be playing the games themselves... such as parents and grandparents choosing games as gifts for kids, etc. I bet they make up more than 50% of sales.

Remember when the Atari era went bust and the bottom of the video games market completely dropped out? My theory is that it was because the industry stopped creating any good-quality games, having realized from experience that they could just produce well-branded crap and rely on all those gullible non-gamer sales. I think the problem is that when the market floods with crap, the gamers (who ultimately receive those games from the purchasers) completely lose interest in games and stop asking their parents to buy them more. So then the purchasers stop buying completely.

In other words, a sufficient minority of titles must continue to be of good quality for the industry to sustain itself, but once that sufficient minority is met, the rest can be crap and the industry can thrive off the crap. The industry then foolishly thinks all it needs to produce is the crap, which kills demand completely, which kills the whole industry.

Incorrect

[Sigma 7]

Game reviews do matter - its called word of mouth. Terms such as "clickfest", "pushover" and "tedious" are negative aspects of a game that you should avoid.

If you want a list of things that a good review should look for, all you have to do is find lists of Cliches and reviews that make note of them. There are similar lists for strategy and action games - but common components among all such lists involve being railroaded through events outside of the players control (e.g. is captured by 3 units after taking out 2000 soldiers), or events that are obvious enough to be traps but the player is forced to go through them to advance the plot.

Well duh.

[raehl]

Newsflash: Can't predict the future based on one criteria.

Hell, the article doesn't even say that there isn't a correlation between good reviews and good games, just that it's not a reliable predictor of sales. Well duh. Maybe things like number of consoles in the market for that game, or marketing, or whether its a sequal, or the price, or whether it's released in May or during the holiday season, might all play a role.

We expect that the same game with good reviews will perform better than that same game with poor reviews. The article confirms that expectation, while trying to sound like it's conclusion is surprising. It's not.

Some people will buy a Pokemon game no matter how bad the review is. This is obvious. Doing a study that confirms it doesn't change that it's obvious and your study is just an excuse to fill some pages under the guise of 'news'.

Game review RATINGS don't matter

[Chris Burke]

which is what the study concludes, and what should be pretty obvious if you read lots of game reviews. Game ratings are basically random numbers between 8 and 10, and where it falls in that range seems to be largely divorced from the content of the review. How many times have you read a review that said something like "the gameplay was fun for the first few levels, but quickly became monotonous and boring" but gave the game a 9.5, or one that said "despite a few minor flaws, this game is all around a lot of fun" and gave it an 8? When I'm out looking for a game, I think I'm going to weigh "monotonous gameplay" a lot more than "Overall Score: 9, Excellent!".

I can understand them using the game rating, as it's the only obvious number you can apply to a game review and do correlations with. However just having a number doesn't mean it actually represents something, and I'm not surprised that game sales don't correlate well with a number that is basically pulled from the reviewer's ass.