Slashdot Mirror
Review Scores the "Least Important Factor" When Buying Games
A recent report from a games industry analyst suggests that among a number of factors leading to the purchase of a video game — such as price, graphics and word of mouth — the game's aggregated review score is the least important measure. Analyst Doug Creutz said, "We believe that while Metacritic scores may be correlated to game quality and word of mouth, and thus somewhat predictive of title performance, they are unlikely in and of themselves to drive or undermine the success of a game. We note this, in part, because of persistent rumors that some game developers have been jawboning game reviewers into giving their games higher critical review scores. We believe the publishers are better served by spending their time on the development process than by 'grade-grubbing' after the fact."
When I personally buy a game, I look at gamefaqs user reviews instead of Metacritic. When looking at the main page of a game on gamefaqs, the first two averaged review numbers are exceptionally useful to me. They seem to give a very strong feel of what the general reaction to a game is - anything under a 7 is probably not worth my money. Also, user reviewers seem to me to play the games more thoroughly than someone who does reviews for a job, and game depth/replayability is a big point for me. Although, if I think about it, I generally buy games for Nintendo DS - price is pretty uniform and graphics can only get so good. In order to look up the game at all I had to have heard about it from my friends or some sites, so my experience doesn't really contradict his research at all.
If you ask people if they are willing to pay more for quality 90% will answer yes. However when the moment supreme is there to purchase for example a new notebook 80% will go for the cheapest and don't care about long term stuff like quality. I think there is a good chance this survey works the same; People SAY they are not influenced by reviews because 'Hey, I'm an original, I don't let anybody influence me'.
>But we don't do we ?
There is a reason for that. It's a lot of hard work and cost bringing a product to market and generally, the real dogs are killed long before they hit the shelves. I've been reviewing hardware/software for 20 odd years now and I can only remember giving a score of less than 4 a handful of times. Equally, 9 & 10 is rare (for me). The vast majority of stuff is 'good enough' and merits 7 or 8 out of 10. TBH, I get really frustrated by constantly dishing out 7s and 8s and the few times something has turned up for review that's truly bad, I'm been delighted as it gives me a chance to have a real opinion.
I want a list of atrocities done in your name - Recoil
This is certainlhy an issue with US based sites/magazines. Over in the UK it's less driven by advertising spend (in my experience, at least). I've given fairly bad reviews to a few products and I still get new stuff from them to look at. Equally, I've had software from the US where they've asked outright if the review will be looked upon favourably if they advertisise with us. They seemed amazed that I was adamant advertising and editorial don't talk to each other. They can't, otherwise the whole point of reviewing is null and void.
There is possibly an argument that because some firms let you keep the kit (sometimes quite expensive kit) and others always want it sent back, that this could affect your scoring but I try hard not to fall into that trap. That said, I often request review items I actually have a need for and this can actively work against it if it doesn't do what I'd hoped.
I want a list of atrocities done in your name - Recoil
I find that when the spread is large, the extreme ends tend to point to piss-poor reviews that I can safely ignore. If there's a lot in either end, however, some of those are probably worth paying attention to. Most of the time, I look at a few "trusted" sites in the middle of the pack, such as IGN and GameSpot. GS invariably gives a lower score than IGN to any game, so I end up looking at that most of the time.
Finally, if a game gets mostly high scores, the low score reviews tend to be informative, and vice-versa.
So any time I buy a game or browse around a store, Metacritic is the first place I check, and then combine that information with the price of the game to arrive at a decision. So far the only times I've missed is when I didn't check Metacritic carefully.
True confidence comes not from realising you are as good as your peers, but that your peers are as bad as you are.
This phenomenon is not unique to Games. I watched a TV Rerun of the Matrix last night and remembered that some Newspaper reviews were very harsh on it. That I would watch it again after all these years and loosing count of how many times I have seen it suggests that the reviewer has standards incompatible with my own. That is not the worse case however.
There was the Mag Innovision letter to the editor after it's 17" monitor received the worst ranking in a roundup of 17" monitors. The Editor's choice award went to a Gateway 2000 monitor. The point of the complaint letter? "This is the same monitor, we just print different labels on the ones we ship to Gateway 2000." Or words to that effect.
So as a general rule I have very little use for published reviews of any product. Word of mouth, and personal trials work best. Also it's good to know what advertisers are allowed to lie about.
"This POS is the best on the market" -: Allowable lie.
"This overpriced crap is great value for money" -: Acceptable lie.
"This 500GB drive holds more data than 750GB of data without using compression" -: dangerous ground.
--= Isn't it surprising how badly I spell ?