Game Reviews are Broken?

Kotaku is running an opinion piece looking at the process of reviewing videogames, and comes to the conclusion that the whole system is entirely broken. Author Mark Wilson takes potshots at the concept of assigning a numerical valuation to a game, and the emphasis on product reviews rather than content reviews. "If there is no such thing as a perfect game, when why the hell are you scoring out of 100? It's not just PC Gamer that thinks this way--most publications, even those who do give out 'perfect' scores, do so begrudgingly. It's as if the developer has somehow cheated and broken their system. The movie reviewers solved this problem a long time ago. That's why most adopted a simpler rating system in which a 4-star movie didn't imply 'perfection' but supreme excellence. In most cases, games are penalized through being divided by a sum that they can never possibly reach."

Comments

Re:Not New

[Fozzyuw]

If there is no such thing as a perfect game, when why the hell are you scoring out of 100? [...]The movie reviewers solved this problem a long time ago. That's why most adopted a simpler rating system in which a 4-star movie

Am I the only one who finds that comment just odd? While I can agree with you that the system is broken because there's no such thing at a "zero" rated game, but I do not see the difference between 100 points and 4 stars, besides it being simply "divided by 25". Then, of course, that's not good enough, so they start assigning 0.5 stars.

Also, I never considered "100/100" to be perfection, but as "supreme excellence" as noted. After, WTF is the difference between "supreme excellence" and "perfection"? Someone is just trying to argue semantics.

Of course, I don't even like the "four star" or 100 point numerical system. When I ask/tell people about a movie I simply say "Is it worth seeing in the theater?", "Is it worth a theater matinée?", "Is it a rental?" or "not worth your time, period".

In this sense, I saw the "Number 23" in the theater and I recommended that it was worth seeing in the theater. While "28 Weeks Later" was easily worth waiting for a rental (despite being a fan of "28 Days Later" and zombie films in general).

In that regards, I would say games should be rated as "buy it!", "rent it/demo it!", "stay away". (rent for console / Demo for PC games). Guitar Hero games are "buy it" games while something like Zelda:Twilight Princesses might be a "rent it" kind of game (I bought it, I'm a fan of Zelda, but still feel money better spent on a rental. I would not have given it 100/100 as some reviews did). World of Warcraft? "Buy it".

Of course, such a system needs a context. I'm not going to tell a FPS fan to buy an RPG, it's in the context of RPG fans.

Cheers,
Fozzy

Re:Reviews are bought

[UbuntuDupe]

The solution would be to have a "Consumer Reports" of gaming where people unaffiliated with the publishers, buy the game at a retail outlet like anyone else at launch, play it, write the review, and then do this consistently for all games that are released.

The disadvantages:

-You wouldn't see the review until after launch. (Probably a week for some games.)
-It doesn't seem to have a viable revenue model, unless someone knows a counterexample?

Re:Exactly...

[Sigma 7]

Over time, the list of 'modifiers' ('this game is a fighting game', 'this is an RPG', 'too short', 'too long', etc.) for a game's score would become standardized, as would the scores each reviewer assigns to each modifier. People reading those reviews could decide what value they wanted to assign to each modifier and so would be able to translate the scores from the reviewer's scale to their own. After being disappointed with games that were rated rather high, this was exactly what I contemplated - I identified specific flaws in games that were already released and see if other games reproduce these flaws. Here's a few examples based on various genres:

• First Person Shooters (Doom-style): Hard to find flaws, but the most common one is slow weapon switching based on interface (e.g. in SiN, if you cycle into the Sniper Rifle by the wheel, you are disabled for ~2 seconds.) Various ways to fix this, best is to abort the current weapon transition as soon as it's obvious the player wasn't choosing that weapon.
• Realtime Tactical Simulations: On completion of an order (e.g. some unit dies), units stop still in their tracks. (Correct behaviour is shown in Tiberian Sun - units keep move to the general location where the target died.)
• Computer Run Adventure Programs (also known as CRPGs):You need massive amounts of grinding just to advance past one obstacle. Correct behaviour requires a smoother flow by not emulating the first edition of Dragon Warrior - although Moria/Angband family of roguelikes seems to be an exception.
• Fighting games: "Unresponsive" controls. Usually this affects PC games where certain keypress patterns don't get recognized by the keyboard (e.g Ctrl+Left+Up registers as Left+Ctrl) but can also mean the game not recognizing actions when they should be valid (e.g. pressing jump one frame early prevents it from registering).
• Puzzle games:No ability to track progress. Giving out passwords is minimally acceptable (especially in the console era), but modern games are expected to show which ones are completed (and how well they're completed as well.

This list doesn't cover the generics, where you don't make Easy this difficult.

The benefit of assuming perfect and stripping away points for known flaw patterns is that you can properly assess how well games stand up to others. It can also allow ratings to be "depreciated" in the same way that other assets do as new flaws get discovered. The disadvantage is that you need to have a lot of experience reviewing and playing games to know and recognize flaw patterns.