Videogame Reviews - Playing With Numbers?

Thanks to NTSC-uk for its editorial discussing the possible confusion in using numbered rating schemes for videogame reviews. The author rhapsodizes: "No number can possibly capture the striking vision of the sun setting over Hyrule Field or the ingenious brilliance of Metal Gear Solid's interactive references to reality", before going on to conclude: "Treated as numbers with a defined value, they will always be looked down upon as having deficiencies. Yet when you read them as you would a word and open it up to your own interpretation, they begin to fully deliver the explanatory potential that is locked within." Do you think numbered ratings have an important place at the end of game reviews?

Comments

Of course I think they have their place!

[tgrotvedt]

I think this person has missed the point of number ratings; they don't puppourt to sum up the entire game in a digit.

Hint: That's what those weird-looking "paragraphs of descriptive text" are for.

What a non-issue

[Anonymous Coward]

We should do away with the star system for hotels (how can one number possibly describe a whole hotel?). We should do away with film genres (how can one word describe a whole film?). We should do away with races (how can one word describe a whole person?). We should do away with ages (how can one number describe the experiences of a whole lifetime?).

What a stupid article. It's not like the whole review is being replaced with a number. It's something to quickly glance at to see if it's a load of shit or a really good game. If there's a review of a game I've never heard of before, and it's got 2/10, then I'm not going to waste my time reading it.

Numbers suck

[Masa]

I think that in the reviews, the writers comments and experiences are the most important aspect. Rating with numbers or percents is dangerous, because it seems to be a rule, that all games are rated between 80% to 100% and if any game receives any lower rating, it is automatically labelled as a bad game even if the game is billiant and the lower rating is given only by techincal reasons (bugs etc.). Also, it's tempting to compare numbers between different reviews even if there isn't any common rule set between different gaming magazines for giving these ratings (so the comparing is actually pointless). Numerical ratings are too subjective to be taken as a meter for the quality of the game (idea, storyline) itself.

Re:Numbers suck

[Knetzar]

How is one to rate things then? If one person says the graphics are beautiful, and another says that they are gorgeous, how does that improve things? 9 and 10 are a lot easier to understand then english.

6=bad - inefficient.

[Anonymous Coward]

We have ten or eleven values in a 10 point system, yet 7 often means average quality and anything below that is bad. Great games always get 9 or 10. That leaves the numbers 0-6 for bad, 7-8 for more average and 9-10 for great games. That means we can differentiate between 7 flavours of bad games but only 2-3 (depending on your interpretation of 8) different levels of great. However, as the article mentions, people aren't interested in bad or lower-average games, so you'd need one, maybe two different descriptions (bad and REALLY bad) for those games. I mean, who cares whether Teletubbies Adventure is better or worse than Jar Jar's Fun Games? People care for comparisons like Zelda vs. Metroid (to name a close example), which might get problematic if you consider both games a ten (yet most people would agree that one is better than the other, but not on a scale that would justify a full point in a ten point system). Sure, you can use a percentage score or other concepts to split the range up further, but then you still have 60%-70% nearly useless possible ratings. Also, such fine differentiation (especially single percents) often isn't possible and numbers are assigned within a certain range with some arbitrartity (93, 94 or 95? Roll a dice).

Note that average game here doesn't refer to statistical average, but perceived average.

they are very useful

[hak1du]

The purpose of the summary ratings is not to let you pick "the best" item, it is to draw your attention to the outliers. For example, if a game (or car or whatever) from a company you have never heard of receives a really high overall rating, you might take the time to look at it. And if a game (or car or whatever) that you just assumed was good and were going to buy based on its brand receives a really low rating, you might check the review and maybe investigate alternatives.

It's simple...

[Weirdofreak]

Just take a hint from /., and instead of giving each game a discriptive number, give it a word as well - if it works for /. comments, it'll work for anything!

-1, bandwagoniser, for example, or +5, mind-bending.

Conserving Space

[DarkZero]

The greatest thing about number scores in reviews is that they let reviewers get their entire message out, even in a confined space. For instance, a lot of the magazines reviews for Prince of Persia started with a small blurb about how the reviewer absolutely LOVED the game, but the rest of it was usually dominated by the warnings that the reviewer felt were very important for every player to know: the camera has some problems, the fighting can be repetitive at some points, the difficulty is very uneven, it's as short as some GBA games, etc. Taken as a whole, the review is very negative; 80% of it is a list of complaints. But taken as a whole and with a score attached, the reviewer is allowed to use his very limited writing space to give the reader a head's up about the game's short-comings while still stressing how wonderful the game really is.

Magazine reviews don't often have the luxury of including a "bottom line" sentence like this one, let alone one that's in a separate paragraph (like this one!), so the number score really helps them sum up their view on a certain game without forcing their opinions of a more obscure game off the magazine's review section and into the ass-end of their website. It allows them to cover both the super-hyped AAA games like Metal Gear Solid: The Twin Snakes or Splinter Cell: Pandora Tomorrow and mostly unknown cult hits like Disgaea in the same issue without charging more than $6 an issue.

On a separate note, it also helps when comparing games. If you generally agree with a reviewer and follow them over a period of time, in the same way that many people follow Roger Ebert's movie column and TV show, then you can use their scores to compare different games that they've reviewed. For instance, if you loved Devil May Cry and Shinobi and a certain reviewer gave them both a 9.0, then it's worth looking at the scores to see how that reviewer sizes them up against a new game that you want to buy, especially something similar like Crimson Sea 2 or Samurai Warriors. He could really like those games and find very little to complain about when reviewing them, but still give them a 7.0, meaning that there's nothing wrong with them, but that they're not really top tier games, either.

And really, it's not as if having a number there hurts the review in some way.

Re:Zzap!64 had it right

[JFMulder]

Gamespot had this rating system for years now... Even the gold medal thingy.

Yes, unfortunately.

[SuperRob]

As someone who writes for a videogame website, we had this exact same debate. Numbers are ultimately meaningless and arbitrary, for all the reasons that the original poster said.

However, we use them anyway. Why? Because as much as we hate to admit it, there are a LOT of people out there who simply WON'T READ THE GODDAMNED REVIEW. In order to do most games justice, you have to write a great deal, and these Ritalin cases simply can't sit still long enough to read it all.

I experimented with this briefly, and the result was a flood of e-mails from people asking, "So, what did you THINK of the game?" No matter how plain I made my opinion, or how basic my vocabulary was, it was clear that they simply weren't READING the review.

We have our point ratings there basically as a way to shut those people up.

Re:Yes, unfortunately.

[fallingdown]

The fact that people don't read the review has very little to do with Ritalin. I personally don't read them because it's bad writing.

Here's how they usually go:

Paragraph one - a brief introduction usually involving some bad puns using the game's name, characters or developers.

Paragraph two through twenty - an overwrought synopsis of the game's plot and back-story, usually cut and pasted from the press release or another review. Sometimes both.

Paragraphs twenty-one through forty-five - A careful consideration of the game's graphics and frame-rate in which the reviewer compares it to the frame-rates he usually gets while playing a five year old first person shooter down at the local Lanwerks. Usually a reference or two is made to the reviewer's expensive, over clocked computer system and a plug for the latest NVIDIA or ATI product. This usually comes with a thinly veiled request for more free products from NVIDIA or ATI.

Half of Paragraph forty-six - a comparison between the product being reviewed and the game-play of some other game in a different genre made in the previous decade. This serves as the "meat" of the review and in the reviewer?s mind gives him ?street-cred.?

2nd half of Paragraph forty-six through paragraph two-hundred and fifty ? Now that the reviewer has revealed to the world, his clear and encyclopedic knowledge of all things game he tells us what he would have done differently had he been allowed to design the game. This usually includes some ranting about not being allowed to rebind his keys and further rebuffs of the games graphics and framerates.

Paragraph two-hundred and fifty one - The reviewer makes reference to the bad puns in paragraph one, inserts a poorly considered sentence about the "meaning" of the game and it's place in pantheon of games that have come before it and then wraps the whole thing up with another bad pun.

Then he gives the game a 9.

I only had to read 5 or 6 billion of those reviews before I figured out I don?t have to read them at all. In short, what passes for game criticism isn?t criticism at all. It?s a thinly veiled ad for the product and the hardware that runs it. It?s also bad writing.

It sounds like you?re a game reviewer. I?m sure your reviews are great. Sorry, no offense. Looks good on you.

Re:Numbers in ratings are needed

[SuperRob]

"However, it is a proven fact that people rating pretty much anything (academic performance, produce quality, car safety) become wildly inconsistent when rating on a scale above 3-point-something."

I can't get over how entirely wrong this is. Proven Fact? Cite your source.

It's not the depth of the rating system that makes it inconsistent ... it's an inconsistent set of values and the application of those values that causes the ratings systems to fall apart. In order for arbitrary numbers to work, you need to have a list of criteria for how to assign the numbers. Are we starting at a total of 100 or 10 and working BACKWARDS (essentially subtracting points for flaws)? This would cause most games to get a relatively high score. Do we start at the low end and add points for positives? You could end up with a lot of low rated games. How do you decide how many points each good or bad trait is worth? Ah, there's the catch.

You need to now have a list of those, and make sure that your reviews consistently evaluate those traits. Dock a point for a bad camera here, you need to dock it for every game with a bad camera. What if the game only has a half-assed camera, compared to the completely unusable camera in this other game? This is exactly where it all starts to fall apart. You can not be consistent when dealing with something as subjective as ART. It's not the number system that causes the problem. It's the HUMAN factor.

You also have to take into account that each REVIEWER is different. Many people absolutely hated the camera in Kingdom Hearts, but I never had a problem with it. I would rate the game higher, but the game is still widely accepted as brilliant.

FWIW, my site recently got some complaints about how we are GameCube biased. Note that we are a GameCube specific website. The specific complaint alledged that we gave "perfect 10" scores to F-Zero GX, Soul Calibur 2, Super Smash Bros. Melee, Super Mario Sunshine, Metroid Prime, and Zelda: Wind Waker. (These are also the ONLY games to be given 10's at the time of that complaint)

The complaintant ignored the fact that we give MULTIPLE reviews for most games, and that a score of 10 is not PERFECT on our scale, because nothing can be perfect. The only game on that list to be given 10's from EVERY reviewer was Wind Waker.

He also ignored the reviewer slant ... some reviewers just like certain types of games better than others. That's again why we have multiple reviews. We want people to find a reviewer who has tastes similar to theirs, because that's the best barometer for choosing what you'll like ... find someone like YOU.