Deconstructing Game Review Structure

Recently there has been a lot of division on the topic of game review structure. Kotaku has an interesting summary of recent commentary, including a piece by GameSetWatch's Simon Parkin and the Taipei Gamer blog. "Except, of course, video games don't work in the same way as toasters or digital cameras. Sure, they have mathematical elements and measurable mechanics and it's possible to compare the number of polygons between this one and that and spin out ten thousand graphs detailing how two specimens compare. But, unlike with the Canon EOS400D, I would have no idea at the end of those 25 pages which game was better or where they would sit on the 'true' scale of quality."

Comments

You can't quantify "Fun"

[Ajaxamander]

And that's why such a comparison would be pointless.

Re:You can't quantify "Fun"

[Last_Available_Usern]

No, you can't quantify fun, but you can qunatify a lot of what about a game makes it fun to most people and allow them to draw conclusions based on that information. Such things include:

1) Interface, controls, information provided in-game and the customization of each.
2) Graphics levels and relative speed comparable to other established games.
3) Complexity and depth of levels/missions.
4) Polish - the presence (or lack of) bugs.
5) Length of play and difficulty compared to other established titles (ie. not as hard as Ninja Gaiden, but close).
6) If it's online, how intuitive the multiplayer functionality works and how it compares to other titles.

The list goes on but you get the idea. You throw out whatever facts you can, compare where appropriate, and let the user make his/her decision from there.

Disparity between Reviewers and Users

[Anonymous Coward]

Just look at the difference between how users score games compared to how reviewers score games and you will see a problem. Grand Theft Auto 4 has a reviewer average of 97.4%/97.3% on the PS3/XBox 360 yet users give it an average score of 79%/83% ... In contrast a game like Mario Kart Wii has a reviewer average of 81.9% and users give it an average score of 96%

I'm not saying that GTA4 is a worse game than Mario Kart Wii, but it is clear that the reviewers are not giving scores which reflect the experience of gamers who own the game ...

(Note: all scores from Gamerankings.com)

Re:Disparity between Reviewers and Users

[faloi]

More importantly, I have to know how a reviewer really rated other games that I like. As the article points out, some metrics are easily measurable. Others are not. If I read a review of a RTS from a person that hates the genre, they're low review might be meaningless. Alternately, if they love the game, it might be an indication that the game reviewed is so good I can't pass it up. Or that the reviewer got a bunch of swag for reviewing the game.

Re:Disparity between Reviewers and Users

[CastrTroy]

I've seen a lot of games get low scores just based on one aspect of the game. Possibly some aspect that the gamer doesn't even care about. I've seen a ratings breakdown like Controls=95% , Replay Value = 95%, Enjoyment=95%, Graphics=85%. And then the final rating ends up being around 85%. In some ways that makes sense. If the control is really bad, it's going to affect everything about the game. If the graphics aren't quite as good as they could be, then most of the time, it doesn't affect the game too badly.

Description

[Enderandrew]

How is a game all that quantifiable?

I'm sick of seeing perfect 10 after perfect 10 review. Not only do I know these are absolutely worthless in regards to objectivity, but very few reviews explain what gameplay is like on a very specific scale.

A good description allows me to decide how much I might enjoy that game. After all, we all enjoy different things.

Re:Description

[TitusC3v5]

Tycho and Gabe have a pretty good idea about why you're seeing so many 10s...

Re:Description

[Phrogman]

There are a few factors at work in a game review - and I speak as someone who wrote dozens of them for a now defunct website

• Most reviewers have only a short bit of time to actually play the game in question, prior to quickly writing up a review for publication and then heading to the next one. I am sure they try to put in all the time they can but its often not possible when your editor wants it *now*.
• Most websites (and undoubtedly magazines) want to be able to continue to review new games. if they diss anything other than an extremely bad game, the publisher will simply not send them the advance copy for review, leaving them out in the cold. Only the top end publishers and websites can probably be all that honest in reviews because of this. I stopped reading game reviews from most places when I realized the scores were essentially bought and paid for in this manner. Luckily for me, my reviews were honest - because I was unpaid :P
• Most reviewers couldn't write their way out of a wet paper bag, and have little or no ability to separate their opinion from an objective loook at the game. A lot of reviews show a heavy bias and one sided reporting. I no longer read most reviews for this reason. In the end they tell me almost nothing about the game in an objective manner

I have given up on game reviews for PC games. There are very few sources for good review information these days, almost everyone is biased in some way or another and few articles focus on providing facts concerning the game and focus on hype or the author's opinion as fact, rather than opinion.

Differences

[JoshJ]

There are certain aspects of a game that are "objective" and some that aren't. Design flaws and low skill ceilings often don't impact the short-term enjoyment of a game, but can make it worthless for long-term play. Mario Kart Wii is a perfect example of this. While skilled players will beat unskilled players nearly every time on Wi-Fi, the item chaos makes high level play worthless on Wi-Fi. (High level play is still quite good on the same console with items set on "strategic" or turned off, of course.)
This is a pretty straightforward analysis, and while it's hard to be 100% objective, it's fairly straightforward to say that the game's design minimizes the small skill gaps and thus replay value is minimized due to the fact that getting better and better at the game has diminishing returns. There's just no point in getting that extra hundredth of a second every lap when you lose 3-5 seconds to a blue shell.

The things that can't be quantified are the problem- is FF7's materia system better than FF8's junction system? That's purely personal preference. You can't go "well, this one requires more skill than the other" or "this one has more replayability" or whatever; because they're fairly similar in those respects and you can't really make a distinction between the two.

But anyone can see that Final Fantasy whatever is a better game than Super Barbie Movie License Cash-In 93 on the Game Boy. The huge, huge difference makes it plain.

I would say that it's probably easier to just lump games into "utter trash", "below average", "average", "above average", and "genre-defining"; and maybe have 2 categories- one for firstplay and one for long-term play.
Gran Turismo 4, for instance, is genre-defining and has excellent long term playability.
Mario Kart Wii is above average when you first play it, but the long term playability is lacking.

Re:Differences

[analog_line]

The problem is that the parameters of your objective analysis are subjectively chosen. At least in the Mario Kart example, the number of people who bought Mario Kart that care about hundredth-of-a-second times in direct online competition is vanishingly small in comparison to the number of people who like to just get online and drive around and have some fun. For that kind of competition there is the world-wide time trial rankings, but "serious" online competition ability just isn't even on the radar screen of people who get online to throw colored shells at other people. It's fun, it isn't serious, no one cares that my Mario Kart ranking is pathetic, and I can crush the hopes and dreams of people with the mighty blue shell because I'm so bad I get it very often. I've never owned any Gran Turismo game, because it would just sit on my shelf forever. Its infinite complexity, customizability, and realism wasted on someone that just doesn't give a crap.

The defining characteristic in the accuracy of a review is point of view intersection between the reviewer and the reader. That's it. That's all that matters. Take the time to know your reviewer and you will never go wrong. I read just about every review I can find for every game I have, and I write lots of reviews (unpublished save for 1 or 2 at gamefaqs) as a writing exercise and because I'm an opinionated SOB (one of these days I'll get around to starting my own review site, or try to get some part time work at an established one). Lots of times I have to wonder whether I am playing the same game as some of these people. I'm sure I am, but it really boils down to the fact that people look for different things in games, and everyone has their red lines, and they're all different to one degree or another.

Re:Differences

[Ren.Tamek]

"Design flaws and low skill ceilings often don't impact the short-term enjoyment of a game, but can make it worthless for long-term play. Mario Kart Wii is a perfect example of this. ... There's just no point in getting that extra hundredth of a second every lap when you lose 3-5 seconds to a blue shell."

Not so! It's very tempting to be dismissive of MK Wii because you have been blue shelled out of first so often. But Mario Kart operates on exactly the same principle that all luck moderated games do - they're very accessible, precisely because a new player always has a chance to beat a pro in a single race. In the long run however, every player has the exact same chance of mishap as every other player, and so as the number of games played gets closer to infinity, the relative skill of each player get easier and easier to measure. This is how tournament players will be picked when the game hopefully gets picked up by the pro circuit.

I know that sounds strange, but some of the worlds best games have operated same principle since games began. Check out how many players are regulars at the World Series of Poker, then sit down and play one hand against any regular player :).

"But anyone can see that Final Fantasy whatever is a better game than Super Barbie Movie License Cash-In 93 on the Game Boy. The huge, huge difference makes it plain."

Ahh, I used to think like you. The difference to you and me seems obvious, but will seem less and less obvious the more gamers you meet, and the less notice you take of reviews. Ask 10 5-year old girls to rate each game after half an hours play, and I guarantee that Barbie Horse Adventures will come up trumps.

Obviously, 5-year old girls often don't know much about games, so you could argue they're doing the review using the wrong principles and values. But actually the game is designed for 5-year old girls to enjoy, so I could just as easily say that you're the one writing the 'wrong' review.