Slashdot Mirror
Videogame Reviews - Playing With Numbers?
Thanks to NTSC-uk for its editorial discussing the possible confusion in using numbered rating schemes for videogame reviews. The author rhapsodizes: "No number can possibly capture the striking vision of the sun setting over Hyrule Field or the ingenious brilliance of Metal Gear Solid's interactive references to reality", before going on to conclude: "Treated as numbers with a defined value, they will always be looked down upon as having deficiencies. Yet when you read them as you would a word and open it up to your own interpretation, they begin to fully deliver the explanatory potential that is locked within." Do you think numbered ratings have an important place at the end of game reviews?
I think that number scores are important for what they are: A distillation of opinion. That's all it is, and all it should be treated as. If you want the justification of the opinion, then that is what the review is for. The way I think about is like this: A random art critic can say 'Van Gogh is the greatest artist, ever.', and that is like a numeric opinion. If I want to know why he thinks that, then I'll read further. If I hate Van Gogh, then I'll be curious about how his opinion could be so different to mine, and if I love him, well, to be honest I probably won't care too much to read a gushing review of the man's work.
I think that the review method in Zzap was the best I have ever seen. They would give it a % rating for several categories and an overall rating, then if they thought it was worth buying by most people, it got a Silver medal. If you didn't like that type of game then maybe you still wouldn't like it, but the very best games got a Gold medal. These ones should be at least looked at by everyone. The best bit though, was the boxes that had the other members of staff commenting on the game. This meant that you got three or four different opinions on each game. That made their review method the best in my opinion.
You're absolutely right. Rotten Tomatoes recognizes the problem that you raise, explicitly, when they determine if a game got enough 'good' reviews to be rated "Fresh".
From the FAQ:
Why is the cutoff for a Fresh Tomato so much higher for individual game reviews?
Although most publishers rate games on a 1-10 scale, it is a rarity for a game to get a score below 6. Because game reviews are mostly positive (a very high majority fall in the 7-10 range), the cutoff for a Fresh Tomato is raised to 8/10. This higher cutoff actually produces a wider spread of Tomatometer scores that is equivalent to movies; otherwise, almost all games are recommended!
The problem is that the bar being set this high has become a defacto standard. Some review site or magazine that starts doing what you suggest (and you're absolutely right, they should) will stand out as a sore thumb and as a company that routinely gives low scores. Which means that companies will stop sending them review copies to play. Which means that they can't compete (especially if they're a magazine) with the other reviewers.
I have a lot of opinions about Cyborgs and Architects
A discerning gamer should never base their game impression (and ultimate decide on whether to buy... or sadly, pirate) solely on these singular values because they abstract away all the qualitative properties of a game. That said, *-star ratings and final numbers /10 or /100 or percentages are all there to give a very quick and summative value on a product.
As someone mentioned earlier, many people want a general impression of what they're about to read. Personally, I like how sites like Gamespot and Gamespy throw the rating right up front, whereas a place like Firingsquad with its insightful yet girthy reviews requires navigating through a drop-down list to check out the "final verdict." I suspect most would rather spend time reading and learning about a "4-star" game than a "1-star" one.
Of course, that leads to the perceived notion that there is some grand quantitative scale when you see something like 79 and 81 / 100. Is the 81 game really better than the 79 reviewed on another site? Ultimately it's up to the reader. It's sometimes good to have bias -- if you're a hardcore genre or platform player, you may be more inclined to accept the given idiosyncrasies (i.e. directed linear levels vs. free-roaming, checkpoint saves vs. save anywhere, etc.).
These are ordinal values at heart, and should not be compared at interval levels.
Now with respect to that article, the author makes a good point about reinforcing the qualitative, descriptive muses of the reviewer. However, it's often preferential to give different abstraction levels of your information to pull in a greater volume of readers. The rating/percentage is a good start. It's doubtful that many readers will engage a lengthy game review (no matter how elegantly written) without having a hint of the final mark. Why read eight pages if it's a really crusty game? Conversely, why do that with a game that's already known from other sources to be great? Just a quick check to verify assumptions, and you're off to go get it. Game reviewers are not supposed to write elaborate and astounding essays for which its effect will fail if abstracted into a single value. They are supposed to aid in (and perhaps entertain) the decision to acquire a game for which the player will ultimately decide whether or not it is of good or sufficient quality.
It's necessary to have and utilize both a summative value and a qualitative review. Relying exclusively on a single value leads to game misconceptions, while a written piece alone cannot realistically convey your information to all but the committed (or bored) readers.
-Victor Chow (Elder_MMHS)
Numbers don't always suck though.
There are plenty of sites that average review scores. Others have already pointed out GameTab and Rotten Tomatoes. There is also GameStats, Gaming Chart and Game Rankings
Game Rankings in particular is good because they include a "difference" listing for each site to compare how far their reviews are from the average of each game.
For example, you can see that the average PSX Nation review is 8.5% higher than the average.