Game Reviews Don't Matter, Study Finds

Next Generation has an article up looking at a report from SIG, on the correlation between game reviews and sales. Their findings indicate that, while reviews obviously do have some effect on games sold, there just isn't that much of a correlation. From the article: "He said he doubted that publishers and PRs would stop caring about review scores, especially as they matter a lot with consumers who compare games from the same sub-genre — say, basketball games. But he said that, as with last year's report, the report's findings are unlikely to be popular. 'We received a lot of attention but the stats do not lie,' he said."

Comments

Pre-release hype

[daranz]

Seems to me that for every major game release out there, there's tons of pre-release hype generated by both the publisher, and the websites/magazines that end up reviewing the game. Screenshots, videos, interviews, all that serves to hype up the game. That hype often ends up being the reason for which people choose to buy the particular game. The whole pre-ordering system makes it even more common. Many people buy the game on the day it comes out, without paying too much attention to the reviews (many of which come later).

Seems like a "no shit" to me

[CaseM]

Especially when rubbish like Enter the Matrix sells tons of carts regardless of actual review scores.

There's a hardcore segment out there that scrutinizes every scrap of information, for sure, but most games are still purchased by Grandma and Grandpa for little Joey because of the title, genre, or franchise, not because of an aggregate score somewhere in cyberspace.

marketing

[aleksiel]

popularity is usually based on marketing, not on reviews.

sucky games (dirge of cerberus) do well with good marketing (commercials, ads, et. al.). good games with little/no marketing don't usually do nearly as well (not that they don't do well at all, just that they lose some of their potential).

a large enough portion of the market doesn't read reviews and bases their purchases off of the "coolness" factor of the game, instead of the quality. if a commercial or ad can make the game look cool, then they're all over it.

They do not lie

[dcapel]

'We received a lot of attention but the stats do not lie,' he said."

What was that saying again? Something about lies, damn lies, and statistics?

Judging from website comments, I have to disagree

[Rifter13]

A lower than expected or higher than expected review from a major game site, can generate hundreds of posts on Evil Avatar, or Gamespy, or bluesnews. I think that the need for reviews are not as great as they used to be, before the internet. I am sure that a lot of people get an idea of what a game is about, in previews. The secret to reading reviews, is to find a reviewer that likes the same games you do. :-)

Sales != A good quality game

[c0d3h4x0r]

The interesting thing here is that the study only looked at reviews versus sales.

It said nothing about reviews (written by magazines, etc) versus consumer opinions (or user reviews). It also said nothing about consumer opinions versus sales.

In my experience, reviews written by reviewers generally align pretty well with what consumers think of the game, while sales have little to do with either. In short, sales seem to have little to do with how good a game actually is. Sales seem to correlate more with things like movie and cartoon and brand tie-ins to a game, distribution methods, price point, and other such factors. All of this is really no big surprise, since the game industry has always successfully relied on churning out such drivel and it has obviously worked pretty well for them most of the time.

A more interesting thing to study would be what percentage of sales are purchases made by people who know nothing about games and won't be playing the games themselves... such as parents and grandparents choosing games as gifts for kids, etc. I bet they make up more than 50% of sales.

Remember when the Atari era went bust and the bottom of the video games market completely dropped out? My theory is that it was because the industry stopped creating any good-quality games, having realized from experience that they could just produce well-branded crap and rely on all those gullible non-gamer sales. I think the problem is that when the market floods with crap, the gamers (who ultimately receive those games from the purchasers) completely lose interest in games and stop asking their parents to buy them more. So then the purchasers stop buying completely.

In other words, a sufficient minority of titles must continue to be of good quality for the industry to sustain itself, but once that sufficient minority is met, the rest can be crap and the industry can thrive off the crap. The industry then foolishly thinks all it needs to produce is the crap, which kills demand completely, which kills the whole industry.

Well duh.

[raehl]

Newsflash: Can't predict the future based on one criteria.

Hell, the article doesn't even say that there isn't a correlation between good reviews and good games, just that it's not a reliable predictor of sales. Well duh. Maybe things like number of consoles in the market for that game, or marketing, or whether its a sequal, or the price, or whether it's released in May or during the holiday season, might all play a role.

We expect that the same game with good reviews will perform better than that same game with poor reviews. The article confirms that expectation, while trying to sound like it's conclusion is surprising. It's not.

Some people will buy a Pokemon game no matter how bad the review is. This is obvious. Doing a study that confirms it doesn't change that it's obvious and your study is just an excuse to fill some pages under the guise of 'news'.

Game review RATINGS don't matter

[Chris Burke]

which is what the study concludes, and what should be pretty obvious if you read lots of game reviews. Game ratings are basically random numbers between 8 and 10, and where it falls in that range seems to be largely divorced from the content of the review. How many times have you read a review that said something like "the gameplay was fun for the first few levels, but quickly became monotonous and boring" but gave the game a 9.5, or one that said "despite a few minor flaws, this game is all around a lot of fun" and gave it an 8? When I'm out looking for a game, I think I'm going to weigh "monotonous gameplay" a lot more than "Overall Score: 9, Excellent!".

I can understand them using the game rating, as it's the only obvious number you can apply to a game review and do correlations with. However just having a number doesn't mean it actually represents something, and I'm not surprised that game sales don't correlate well with a number that is basically pulled from the reviewer's ass.

OT:A more interesting statistic to find out is...

[kn0tw0rk]

How many copies of a game get traded in as the days/weeks/months pass since the release date. As that is more of an indicator of wether the game sucks or is worth holding onto, IMO. Few copies of the game in the 2nd hand section gets my attention and gets me asking questions to the EB staff.

My buying habits are erratic, sometime I go by reviews, or word of mouth, or a continuing franchise that I've bought previous versions of. But wether I keep hold of a game or trade it in is based on wether I enjoyed playing the game.