Deconstructing Game Review Structure

Recently there has been a lot of division on the topic of game review structure. Kotaku has an interesting summary of recent commentary, including a piece by GameSetWatch's Simon Parkin and the Taipei Gamer blog. "Except, of course, video games don't work in the same way as toasters or digital cameras. Sure, they have mathematical elements and measurable mechanics and it's possible to compare the number of polygons between this one and that and spin out ten thousand graphs detailing how two specimens compare. But, unlike with the Canon EOS400D, I would have no idea at the end of those 25 pages which game was better or where they would sit on the 'true' scale of quality."

Comments

You can't quantify "Fun"

[Ajaxamander]

And that's why such a comparison would be pointless.

Re:You can't quantify "Fun"

[Last_Available_Usern]

No, you can't quantify fun, but you can qunatify a lot of what about a game makes it fun to most people and allow them to draw conclusions based on that information. Such things include:

1) Interface, controls, information provided in-game and the customization of each.
2) Graphics levels and relative speed comparable to other established games.
3) Complexity and depth of levels/missions.
4) Polish - the presence (or lack of) bugs.
5) Length of play and difficulty compared to other established titles (ie. not as hard as Ninja Gaiden, but close).
6) If it's online, how intuitive the multiplayer functionality works and how it compares to other titles.

The list goes on but you get the idea. You throw out whatever facts you can, compare where appropriate, and let the user make his/her decision from there.

Re:You can't quantify "Fun"

[Omestes]

But these aren't universal metrics, so you can't make a raw score that would actually indicate the objective quality of the game.

For instance, I have a friend that plays WoW on the minimum graphics setting, not because of hardware limitations, but because "he justs like pixels". Sometimes I like games with really simple controls (like Diablo II). Sometimes, as well, I like really simple game play (Tetris).

Polish, I'd agree, might be universal, since bugs are never desirable.

Length of play is subjective too. In several recent RPGs (all of Xenosaga, FFIX, etc...) the game place was TOO long for anyone with a life. And sometimes I like a quick game, really quick, just to waste some time at the dentists office or such.

Online play is a minus for many people too. Online content might be nice, but there still are skads of people out there with dial-up who might be frustrated. I've been getting more and more annoyed with forced online aspects, since I generally play games to ESCAPE from people, especially the average gamer on Xbox Live, or WoW.

Made short, you still can't derive a simple number score from these factors, that will actually say anything about the game. I prefer longish reviews (2-3 pages) talking about the plot, the game play, and just pretty much the view of the games. I also generally prefer the user reviews over the "official" one, even if they often scale higher.

Sure you can.

[Chris Burke]

Remember, "quantifiable" and "subjective" are neither opposites nor mutually exclusive. You can have a quantifiable metric that may vary from person to person, but is still measurable and can thus give a solid indication of how you may react to a game. What quantifiable "fun" metric am I talking about? Well think about what is both fun and measurable and I think you'll see the obvious answer: I'm talking about orgasms.

And no, pleasuring yourself (or having someone else do it -- ha!) while playing doesn't count. Only orgasms resulting from the game itself counts. As you can see, this is a completely quantifiable measure, and while the result is certainly subjective, its measure is not (well, for males, it can be a little iffy with females). How much better is that than a traditional "score" where not only is the reviewer's enjoyment of the game completely subjective, but how they translate that enjoyment into the numerical score is essentially arbitrary.

Now how do games stack up with the Burke Metric? Well, every game I've ever played has a Burke Score of 0, making them all equally bad. Except Rez, which at a score of 1 is objectively the best game I've ever played.

Disparity between Reviewers and Users

[Anonymous Coward]

Just look at the difference between how users score games compared to how reviewers score games and you will see a problem. Grand Theft Auto 4 has a reviewer average of 97.4%/97.3% on the PS3/XBox 360 yet users give it an average score of 79%/83% ... In contrast a game like Mario Kart Wii has a reviewer average of 81.9% and users give it an average score of 96%

I'm not saying that GTA4 is a worse game than Mario Kart Wii, but it is clear that the reviewers are not giving scores which reflect the experience of gamers who own the game ...

(Note: all scores from Gamerankings.com)

Re:Disparity between Reviewers and Users

[faloi]

More importantly, I have to know how a reviewer really rated other games that I like. As the article points out, some metrics are easily measurable. Others are not. If I read a review of a RTS from a person that hates the genre, they're low review might be meaningless. Alternately, if they love the game, it might be an indication that the game reviewed is so good I can't pass it up. Or that the reviewer got a bunch of swag for reviewing the game.

Re:Disparity between Reviewers and Users

[CastrTroy]

I've seen a lot of games get low scores just based on one aspect of the game. Possibly some aspect that the gamer doesn't even care about. I've seen a ratings breakdown like Controls=95% , Replay Value = 95%, Enjoyment=95%, Graphics=85%. And then the final rating ends up being around 85%. In some ways that makes sense. If the control is really bad, it's going to affect everything about the game. If the graphics aren't quite as good as they could be, then most of the time, it doesn't affect the game too badly.

Re:Disparity between Reviewers and Users

[trdrstv]

I've seen a lot of games get low scores just based on one aspect of the game. Possibly some aspect that the gamer doesn't even care about. I've seen a ratings breakdown like Controls=95% , Replay Value = 95%, Enjoyment=95%, Graphics=85%. And then the final rating ends up being around 85%. In some ways that makes sense.

Sure, and the reverse is also true. Some times you might have everything rate an 85% and have the final score being a 90%, because of the synergy involved. "The only exceptional part is how all these things came together to make a better game stronger impact."

Re:Disparity between Reviewers and Users

[oracle128]

In the same vein, reviews are generally going to be relative to other games on the same platform. So if there are 5 games on a console G1-G7, and G1 is bad but G2-G7 are REALLy bad, then G1 is going to get a pretty decent score because there's nothing better than it on the platform.

This is what gets me with some review sites where they have specific staff covering specific platforms 100% of the time - the scores are going to be relative to the other games on their designated platform.

Re:Disparity between Reviewers and Users

[Das Modell]

Neverwinter Nights 2 has a Metacritic score of 82, but most players didn't like it. After the release, there were 2-3 times as many negative comments on the forums than positive ones, and the Metacritic user score is 5.2.

On the other hand, there are lots of incompetent people around, which can distort user scores. For instance, one Metacritic user review for GTA4 with a score of 4 says that "this game is totally over rated all GTA games are the same they are boring with the same missions over and over again it doesn't compare to MGS4." A user review with a score of 0 says: "I will not let my husband play this. There are much better things to do with your time than prostitution, killing, stealing, swearing, drinking, and all forms of immorality!"

... actually... no you can't.

[PC and Sony Fanboy]

Only orgasms resulting from the game itself counts. As you can see, this is a completely quantifiable measure, and while the result is certainly subjective, its measure is not (well, for males, it can be a little iffy with females). This only applies when you're talking about binomial distributions (ie: Either Yes/No, On/Off, Orgasm/No Orgasm).

Your reasoning is flawed because the enjoyment of a game is on several levels, and part of the enjoyment of a game is based on your personal interest/involvement - For example, Final Fantasy Fanboys love Final Fantasy Games. Even when they're crap (FF8), there are still a lot of people who love it, simply because of the FF designation. It has to do with personal commitment and involvement with a game!

Re:... actually... no you can't.

[Chris Burke]

This only applies when you're talking about binomial distributions (ie: Either Yes/No, On/Off, Orgasm/No Orgasm).

Well Orgasm/No Orgasm is what I'm talking about, so yes, I'm glad we agree that my scoring system is quantifiable.

For example, Final Fantasy Fanboys love Final Fantasy Games. Even when they're crap (FF8), there are still a lot of people who love it, simply because of the FF designation.

Did they love it to the point of spontaneous ejaculation? No? Then it scores a Zero on the Burke Scale.

The system works!

Re:... actually... no you can't.

[mooingyak]

I checked out this article just see if anyone complained about the usage of 'deconstruction'. This discussion is somewhat outside of what I had anticipated. Also, I've never played Rez, and now I'm scared to.

Re:... actually... no you can't.

[Chris Burke]

Also, I've never played Rez, and now I'm scared to.

Rez comes with a rather unique accessory, called the "Trance Vibrator". You can figure out the rest. :)

Re:... actually... no you can't.

[trdrstv]

This only applies when you're talking about binomial distributions (ie: Either Yes/No, On/Off, Orgasm/No Orgasm).

EVERYTHING can be broken down to a binomial distribution. Let's take the 1-10 scale for something different... Women. You can throw out the "She's a 7, or she's a 9" argument, because the criteria is personal (not universally agreed upon) and it always boils down to a "Yes/No" question anyway.

Re:... actually... no you can't.

[bigstrat2003]

Even when they're crap (FF8)... Excuse me... not to start a flame war, but I believe you misspelled "FF9". ;)

Re:... actually... no you can't.

[ezzzD55J]

This only applies when you're talking about binomial distributions (ie: Either Yes/No, On/Off, Orgasm/No Orgasm). Huh, do you know what a binomial distribution is? Either you don't or I don't.

All Reviewers Retiring, Reviews to Cease

[Gizzmonic]

The gaming, movie, music, and art press issued a statement today. "We realize now that people have been looking for an objective reviewing standard," read the statement. "We apologize for not noticing earlier, but since they have buying our review magazines and visiting our review websites for decades now, we thought they happy with our reviews. A blog entry on a website "kotaku" has changed all that. We are now retiring en masse. All non-objective reviews will cease immediately. We apologize for wasting your time."

A good review

[elrous0]

A good review is one that agrees with me. Like everything else, it revolves exclusively around my wants, tastes, and desires. Everything that fails to revolve around me, including every other human being who existed before my birth and will exist after my death, is completely irrelevant.

A good review must be score by how much the author represented the right way of thinking about the game (i.e. my way of thinking about the game).

Description

[Enderandrew]

How is a game all that quantifiable?

I'm sick of seeing perfect 10 after perfect 10 review. Not only do I know these are absolutely worthless in regards to objectivity, but very few reviews explain what gameplay is like on a very specific scale.

A good description allows me to decide how much I might enjoy that game. After all, we all enjoy different things.

Re:Description

[trdrstv]

A good description allows me to decide how much I might enjoy that game. After all, we all enjoy different things.

I'm not sure if you noticed, but there are large sections of text either directly above, or directly below the score. This text typically describes the game.

Re:Description

[Enderandrew]

And that text often reads "This game is twelve shades of awesome!" as opposed to adequately describing gameplay mechanics.

Re:Description

[TitusC3v5]

Tycho and Gabe have a pretty good idea about why you're seeing so many 10s...

Re:Description

[Phrogman]

There are a few factors at work in a game review - and I speak as someone who wrote dozens of them for a now defunct website

• Most reviewers have only a short bit of time to actually play the game in question, prior to quickly writing up a review for publication and then heading to the next one. I am sure they try to put in all the time they can but its often not possible when your editor wants it *now*.
• Most websites (and undoubtedly magazines) want to be able to continue to review new games. if they diss anything other than an extremely bad game, the publisher will simply not send them the advance copy for review, leaving them out in the cold. Only the top end publishers and websites can probably be all that honest in reviews because of this. I stopped reading game reviews from most places when I realized the scores were essentially bought and paid for in this manner. Luckily for me, my reviews were honest - because I was unpaid :P
• Most reviewers couldn't write their way out of a wet paper bag, and have little or no ability to separate their opinion from an objective loook at the game. A lot of reviews show a heavy bias and one sided reporting. I no longer read most reviews for this reason. In the end they tell me almost nothing about the game in an objective manner

I have given up on game reviews for PC games. There are very few sources for good review information these days, almost everyone is biased in some way or another and few articles focus on providing facts concerning the game and focus on hype or the author's opinion as fact, rather than opinion.

Re:Description

[trytoguess]

How can one be objective about how enjoyable a game is? A reviewer IS supposed to mix somewhat "objective" values like graphics, and ultimately note how much THEY liked playing the game. How exactly would one write an objective (something that is always true irrelevant to humans views) review anyways?

Re:Description

[Phrogman]

Well of course it all has to come down to the opinion of the author, but they can try to look at things in a dispassionate way.

I can write that "Pirates of the Burning Sea is an MMORPG game that is very oriented towards PvP gameplay, and much less suited to those who don't enjoy player versus player content in their online gaming. While it is possible to play the game without engaging in PvP you will find yourself waiting for a port to leave contention quite frequently, and this will disrupt your gameplay. The alternative is to try to minimize the effect of PvP on your gameplaying experience by avoiding ports in contention, but this is not always possible. That said, the PvP is quite a bit of fun for many players and the game has an excellent combat system at sea".

Or I can write: "This game has terrific PvP gameplay but like EVE Online its not for wusses, and failure it comes at a cost, so enter a battle prepared to risk your ship and its enhancements if you lose. The combat system is complex and requires planning to score a victory in many cases, battles can take a considerable time period between well matched opponents, although ganking teams are pretty prevalent on most servers, so be aware that one lone pirate you attack probably has 5 friends in hiding. Although the game does have crafting and there are roleplayers present on some servers, most carebears need not apply - this is not a game for people who don't have the cojones to participate in Player versus player combat, and of course there are other games that offer that sort of gameplay. If you aren't a PvPer, I highly suggest you avoid Pirates of the Burning Sea and go play CarebearsOnline(tm) instead" or something to that effect.

I have seen comments that were closer to the later than the former in game reviews. The former is me trying to provide a balanced view for all players and would of course elaborate on its points, the later is me attempting to take the view that PvP is the only "real" and "valid" gameplay and that all those who don't like participating in it are somehow limp-wristed failures (an attitude accepted and promulgated by many PvPers on various forums). If I was one of these narrow minded PvPer types I would be most likely to write a review like the later if I wasn't experienced at writing reviews from a more balanced perspective.

Now naturally any review should point out what the reviewer liked about the game, disliked about the game, some examples of their direct experience etc, but those should all be clearly labeled as such and separated from the rest of the review somewhat so they can be seen as less objective.

A hastily written example but hopefully you see my point.

Differences

[JoshJ]

There are certain aspects of a game that are "objective" and some that aren't. Design flaws and low skill ceilings often don't impact the short-term enjoyment of a game, but can make it worthless for long-term play. Mario Kart Wii is a perfect example of this. While skilled players will beat unskilled players nearly every time on Wi-Fi, the item chaos makes high level play worthless on Wi-Fi. (High level play is still quite good on the same console with items set on "strategic" or turned off, of course.)
This is a pretty straightforward analysis, and while it's hard to be 100% objective, it's fairly straightforward to say that the game's design minimizes the small skill gaps and thus replay value is minimized due to the fact that getting better and better at the game has diminishing returns. There's just no point in getting that extra hundredth of a second every lap when you lose 3-5 seconds to a blue shell.

The things that can't be quantified are the problem- is FF7's materia system better than FF8's junction system? That's purely personal preference. You can't go "well, this one requires more skill than the other" or "this one has more replayability" or whatever; because they're fairly similar in those respects and you can't really make a distinction between the two.

But anyone can see that Final Fantasy whatever is a better game than Super Barbie Movie License Cash-In 93 on the Game Boy. The huge, huge difference makes it plain.

I would say that it's probably easier to just lump games into "utter trash", "below average", "average", "above average", and "genre-defining"; and maybe have 2 categories- one for firstplay and one for long-term play.
Gran Turismo 4, for instance, is genre-defining and has excellent long term playability.
Mario Kart Wii is above average when you first play it, but the long term playability is lacking.

Re:Differences

[analog_line]

The problem is that the parameters of your objective analysis are subjectively chosen. At least in the Mario Kart example, the number of people who bought Mario Kart that care about hundredth-of-a-second times in direct online competition is vanishingly small in comparison to the number of people who like to just get online and drive around and have some fun. For that kind of competition there is the world-wide time trial rankings, but "serious" online competition ability just isn't even on the radar screen of people who get online to throw colored shells at other people. It's fun, it isn't serious, no one cares that my Mario Kart ranking is pathetic, and I can crush the hopes and dreams of people with the mighty blue shell because I'm so bad I get it very often. I've never owned any Gran Turismo game, because it would just sit on my shelf forever. Its infinite complexity, customizability, and realism wasted on someone that just doesn't give a crap.

The defining characteristic in the accuracy of a review is point of view intersection between the reviewer and the reader. That's it. That's all that matters. Take the time to know your reviewer and you will never go wrong. I read just about every review I can find for every game I have, and I write lots of reviews (unpublished save for 1 or 2 at gamefaqs) as a writing exercise and because I'm an opinionated SOB (one of these days I'll get around to starting my own review site, or try to get some part time work at an established one). Lots of times I have to wonder whether I am playing the same game as some of these people. I'm sure I am, but it really boils down to the fact that people look for different things in games, and everyone has their red lines, and they're all different to one degree or another.

Re:Differences

[analog_line]

If you try to get *really good* at Mario Kart and put in that sort of effort, you're going to get disappointed because even a second or so of improvement is negligible against the power of a blue shell.

Depends which version of online competition. I agree, for the normal online race mode, there isn't much point in trying to go for the best raw time (especially when it's more effective to run in 2nd place and let your opponent eat the blue shells until the end). That said, if you want serious high level times, the built-in ability to see the top worldwide times for each track seems like a pretty steep long-term challenge to me.

Re:Differences

[Ren.Tamek]

"Design flaws and low skill ceilings often don't impact the short-term enjoyment of a game, but can make it worthless for long-term play. Mario Kart Wii is a perfect example of this. ... There's just no point in getting that extra hundredth of a second every lap when you lose 3-5 seconds to a blue shell."

Not so! It's very tempting to be dismissive of MK Wii because you have been blue shelled out of first so often. But Mario Kart operates on exactly the same principle that all luck moderated games do - they're very accessible, precisely because a new player always has a chance to beat a pro in a single race. In the long run however, every player has the exact same chance of mishap as every other player, and so as the number of games played gets closer to infinity, the relative skill of each player get easier and easier to measure. This is how tournament players will be picked when the game hopefully gets picked up by the pro circuit.

I know that sounds strange, but some of the worlds best games have operated same principle since games began. Check out how many players are regulars at the World Series of Poker, then sit down and play one hand against any regular player :).

"But anyone can see that Final Fantasy whatever is a better game than Super Barbie Movie License Cash-In 93 on the Game Boy. The huge, huge difference makes it plain."

Ahh, I used to think like you. The difference to you and me seems obvious, but will seem less and less obvious the more gamers you meet, and the less notice you take of reviews. Ask 10 5-year old girls to rate each game after half an hours play, and I guarantee that Barbie Horse Adventures will come up trumps.

Obviously, 5-year old girls often don't know much about games, so you could argue they're doing the review using the wrong principles and values. But actually the game is designed for 5-year old girls to enjoy, so I could just as easily say that you're the one writing the 'wrong' review.

Re:Differences

[Ren.Tamek]

That's an interesting opinion, and I can certainly see that you don't find it very fun. But people still play Super Mario Kart and Mario Kart 64 all the time. I own neither, and I played both very recently at different peoples houses. The way I see it, making a game accessible can only make the game more vibrant in the long term, as new players can become competent enough to have fun very quickly, and can compete with a chance against veterans, even if they won't be able to win the majority of races.

The lesson I really want you to take away from this is subjectivity. I have been playing mario kart since I was in single digits - yet now that i'm in my 20's I still love the game to pieces. Obviously it doesn't 'nuke' long term playability for me, or for a number of friends who I still enjoy playing with to this day.

Just the facts, please.

[Lilith's Heart-shape]

I agree. I don't want the author's opinion. I want the facts about the game. I want to know whether or not I should bother to even rent it, let alone buy it. A good review will give me this information (along with the author's opinion).

Only thing I know for sure about the subject

[Pluvius]

Whatever good game review structure might be, this isn't it.

Rob

Re:Only thing I know for sure about the subject

[spir0]

I feel dumber after having read the first few sentences of that.

Re:Only thing I know for sure about the subject

[leoboiko]

Haha disregard that, that one is just perfect. Tim's reviews of Metal Gear Solid 2 and Mother 2 blew my mind. After reading those, I gave up mainstream game media altogether.

In this corner of the net a lot of people have been experimenting with alternative game review forms for a long time. Some fun sites for starters are:

• selectbutton forums
• insertcredit news, of which I'm a humble very occasional contributor
• gamer's quarter magazine, which I reviewed in a vaguely gamer's quarter style here
• actionbutton reviews

Like what you see? Are you into literary criticism? Do you spend more time talking about videogames than playing them? Join us in the pretentious side! We got doujin games! Pongism is the truth!

Re:Only thing I know for sure about the subject

[Pluvius]

The MGS2 review is one of his most tolerable, I'll grant, simply because it's actually about the game (sort of) and doesn't ramble on and on for twenty thousand words. Though he apparently didn't get the memo that splitting your review up into a bunch of one-sentence paragraphs doesn't really lend any dramatic weight to it.

The Mother 2 review, on the other hand, is more of Tim's typical tedious and pretentious shit.

Rob

Re:Only thing I know for sure about the subject

[leoboiko]

The more a review is âoeactually about the gameâ, the worse it is.

From a game reviewer

[LatencyKills]

I've been writing game reviews for about seven years now (shameless plug: game-over.net), and as a writer and a reader I think that game reviews do have merit. If you want someone to tell you if a game is fun or not, you're using reviews the wrong way. What a game review can tell you, especially if you read a lot of them, it what a game is like to play, drawing useful comparisons to other games that you may have played and may or may not have liked. It is also possible to find a reviewer who in general likes the same kind of games you do, and becomes sort of a litmus test for you in the future.

Game reviewing for the non corrupt

[Cathoderoytube]

Game reviewing is pretty straight forward. A lot of people would assume it's based on things like fun factor, aesthetics, controls, etc. This is actually not the case.
There are effectively two factors involved in game reviewing; bribes, and repercussions.

The video card and big gaming companies give a lot of bribes to the game reviewers, so things like graphics and big blockbuster titles are given good reviews regardless. Repercussions are taken into consideration based on on which company produced the game you're reviewing. Say for example if you're reviewing a game from some small time game company. There's no worry about money being pulled from the magazine/website if you ream the game. The complete opposite is true if you're reviewing anything from say Bungee, or Eidos.
The bottom line is, graphics and big titles are the most important things, since that's where all the bribes come from.