Game Reviews are Broken?

Kotaku is running an opinion piece looking at the process of reviewing videogames, and comes to the conclusion that the whole system is entirely broken. Author Mark Wilson takes potshots at the concept of assigning a numerical valuation to a game, and the emphasis on product reviews rather than content reviews. "If there is no such thing as a perfect game, when why the hell are you scoring out of 100? It's not just PC Gamer that thinks this way--most publications, even those who do give out 'perfect' scores, do so begrudgingly. It's as if the developer has somehow cheated and broken their system. The movie reviewers solved this problem a long time ago. That's why most adopted a simpler rating system in which a 4-star movie didn't imply 'perfection' but supreme excellence. In most cases, games are penalized through being divided by a sum that they can never possibly reach."

Comments

Not New

[quanticle]

Game publishers, consumers, and even the reviewers themselves have been going on about the shortcomings of the current system for quite a while now. Yet we never see any alternatives being proposed. I say to the article writer, "Yes, I agree that the current system sucks. But what is your alternative?"

Re:Not New

[Fozzyuw]

If there is no such thing as a perfect game, when why the hell are you scoring out of 100? [...]The movie reviewers solved this problem a long time ago. That's why most adopted a simpler rating system in which a 4-star movie

Am I the only one who finds that comment just odd? While I can agree with you that the system is broken because there's no such thing at a "zero" rated game, but I do not see the difference between 100 points and 4 stars, besides it being simply "divided by 25". Then, of course, that's not good enough, so they start assigning 0.5 stars.

Also, I never considered "100/100" to be perfection, but as "supreme excellence" as noted. After, WTF is the difference between "supreme excellence" and "perfection"? Someone is just trying to argue semantics.

Of course, I don't even like the "four star" or 100 point numerical system. When I ask/tell people about a movie I simply say "Is it worth seeing in the theater?", "Is it worth a theater matinée?", "Is it a rental?" or "not worth your time, period".

In this sense, I saw the "Number 23" in the theater and I recommended that it was worth seeing in the theater. While "28 Weeks Later" was easily worth waiting for a rental (despite being a fan of "28 Days Later" and zombie films in general).

In that regards, I would say games should be rated as "buy it!", "rent it/demo it!", "stay away". (rent for console / Demo for PC games). Guitar Hero games are "buy it" games while something like Zelda:Twilight Princesses might be a "rent it" kind of game (I bought it, I'm a fan of Zelda, but still feel money better spent on a rental. I would not have given it 100/100 as some reviews did). World of Warcraft? "Buy it".

Of course, such a system needs a context. I'm not going to tell a FPS fan to buy an RPG, it's in the context of RPG fans.

Cheers,
Fozzy

Maybe

[Mr_Silver]

Although he has a point, we humans love to compare and if you don't give us any metric by which to do that, then we don't feel like anything has been achieved.

"SuperGame is really good" is meaningless to me. What I want to know is, is it any better than GreatGame? If the reviewer gives a score for both then I can understand which he/she feels is better and by what margin. Since I've played GreatGame (and assuming I trust the reviewer), then I can set some sort of expectation of what SuperGame will be like.

Personally, I use Metacritic which aggregates a number of reviews. Again, it's not perfect, but when it gets a 75 or above score, I can be reasonably certain that I'm not getting a dud game. It might not be my type of game, but if it is, then it shouldn't be disappointing.

The best review

[SuiteSisterMary]

The best review has no score. Simply somebody playing the dang game, and talking about what they like, what they don't like, what they'd improve, what really bothered them, what really excited them.

Find a reviewer with a decent command of the language, and who likes the sorts of games you like, and you're good to go.

Re:The best review

[quanticle]

This is why I like the Zero Punctuation reviews so much. Yahtzee has a decent command of the language, goes through all of the good and bad parts of the games, and gives a quick conclusion stating his opinion of the thing.

Re:The best review

[moderatorrater]

I disagree. For good games, a good review is definitely more than the score, but when I check out a much-hyped game and see a score of 50/100, that's a big deal to me. That means I don't even have to worry about reading the review, and in some cases that's the best review there is. During the era of 3000 RTS games a month, it was nice to be able to sort the good from the bad with a glance. It's also nice to be able to see the high and low points of a series, to be able to see that most people find the mechanics of FFVIII and FFIX lacking so that I know, if i'm going to start the series, start it somewhere else.

As TFA states, the review industry is necessary because of the large amount of games coming out and the large proportion of crap that inhabits it. If something is crap from end to end and at least four different people agree, then there's no need for me to look further.

Exactly...

[rwven]

These very thoughts came to my mind when I was reading all the Halo3 Reviews... When a game has so-so single player and awesome multiplayer...how does that get the game loads of perfect scores? A perfect game wouldn't need to make up for areas of lacking ANYWHERE. That aside, even the multiplay, while fun, is far from perfect. Halo3 was, and is, a great package but it's nowhere near a "perfect" game. I'm not just picking on Halo3 here either. HL2 for instance was a phenomenal game....but to call it "perfect" (like so many reviewers did) is just naive and downright inaccurate.

Re:Exactly...

[Sigma 7]

Over time, the list of 'modifiers' ('this game is a fighting game', 'this is an RPG', 'too short', 'too long', etc.) for a game's score would become standardized, as would the scores each reviewer assigns to each modifier. People reading those reviews could decide what value they wanted to assign to each modifier and so would be able to translate the scores from the reviewer's scale to their own. After being disappointed with games that were rated rather high, this was exactly what I contemplated - I identified specific flaws in games that were already released and see if other games reproduce these flaws. Here's a few examples based on various genres:

• First Person Shooters (Doom-style): Hard to find flaws, but the most common one is slow weapon switching based on interface (e.g. in SiN, if you cycle into the Sniper Rifle by the wheel, you are disabled for ~2 seconds.) Various ways to fix this, best is to abort the current weapon transition as soon as it's obvious the player wasn't choosing that weapon.
• Realtime Tactical Simulations: On completion of an order (e.g. some unit dies), units stop still in their tracks. (Correct behaviour is shown in Tiberian Sun - units keep move to the general location where the target died.)
• Computer Run Adventure Programs (also known as CRPGs):You need massive amounts of grinding just to advance past one obstacle. Correct behaviour requires a smoother flow by not emulating the first edition of Dragon Warrior - although Moria/Angband family of roguelikes seems to be an exception.
• Fighting games: "Unresponsive" controls. Usually this affects PC games where certain keypress patterns don't get recognized by the keyboard (e.g Ctrl+Left+Up registers as Left+Ctrl) but can also mean the game not recognizing actions when they should be valid (e.g. pressing jump one frame early prevents it from registering).
• Puzzle games:No ability to track progress. Giving out passwords is minimally acceptable (especially in the console era), but modern games are expected to show which ones are completed (and how well they're completed as well.

This list doesn't cover the generics, where you don't make Easy this difficult.

The benefit of assuming perfect and stripping away points for known flaw patterns is that you can properly assess how well games stand up to others. It can also allow ratings to be "depreciated" in the same way that other assets do as new flaws get discovered. The disadvantage is that you need to have a lot of experience reviewing and playing games to know and recognize flaw patterns.

Reviews have been fundamentally broken for years

[anss123]

Reviews have been fundamentally broken for years and years. Ten years ago Gamepro gave Bubsy 3D an impossibly high score of 3.5 out of 5 - a score comparable with Screamer 2, tempest, Cruis'n USA and other playable games. Playing Bubsy is about as enjoyable as stabbing your eyes out, it's a turd among turds. Incidentally there was a full page add for, you guessed it, Bubsy 3D in that very issue.

Problem is that these magazines are at the mercy at the games they review. They need to get exclusives, interviews, previews and adds to stay in the game. They are therefore very reluctant to give out bad scores to games from well known publishers.

Once upon a time there was a magazine (Amiga Power?) that did just this, said things as they were, and they found themselves cahoots by devs like Team 17, etc, for simply stating their actual opinions.

How about a survey:

[Bob-taro]

On a scale of 1 to 10, how bad do you think the 10 point rating system is.

Re:Does it matter?

[geminidomino]

There's a stigmata though when actually attaching a number to a review. Umm... ouch.

When game rags claim to put their blood and sweat into the magazine, I never knew they meant it so LITERALLY.

Re:Reviews are bought

[UbuntuDupe]

The solution would be to have a "Consumer Reports" of gaming where people unaffiliated with the publishers, buy the game at a retail outlet like anyone else at launch, play it, write the review, and then do this consistently for all games that are released.

The disadvantages:

-You wouldn't see the review until after launch. (Probably a week for some games.)
-It doesn't seem to have a viable revenue model, unless someone knows a counterexample?