Slashdot Mirror
Are Review Scores Pointless?
donniebaseball23 writes: With Eurogamer being the latest popular video games site to ditch review scores, some are discussing just how valuable assigning a score to a game actually is these days. It really depends on whom you ask. "I've always disliked the notion of scores on something as abstract and subjective as games," says Vlambeer co-founder Rami Ismail. From the press side, though, former GameSpot editor Justin Calvert still believes in scores. "I've been basing my own game-purchasing decision on reviews ever since I picked up the first issue of Zzap! 64 magazine in the UK almost 30 years ago," he says, while admitting that YouTube is certainly changing the landscape today: "There's something very appealing about watching a game being played and knowing that the footage hasn't been edited in a way that might misrepresent the experience."
I've found Metacritic to be a good aggregator of scores, but more importantly, the "users" scores (and reviews) tend to be more reliable in terms of not being overly critical of games that are generally pretty good, but don't meet the expectations of "hard core" gamers.
Betteridge's Law indicates that the answer is "no", when of course, the answer is actually "duh".
This seems like a rather pointless question, since 'reviews' and 'review scores' serve somewhat different purposes.
If you want a comparatively deep examination of a game, strengths, weaknesses, what is it trying to do?, does it succeed?, who is it aimed at?, etc. an answer like "65" or "8" is practically useless. If you want to do a metacritic-style survey(or decide what long-form reviews to read when faced with 2,000 games), though, 3 pages of prose and musing, each, from two dozen sources isn't going to cut it.
Anyone pretending that a hundred-point score is actually that precise is likely fooling themselves; but there's a much stronger argument that you can get at least a 1-10 or so scoring system unless you are a pure, handwaving "It's all, like, intersubjective, man..." type.
Almost no games get below 40, while any game that doesn't get 80 or more is considerd a failure. Then you have people giving games 3 out of 5 stars which translates to a score of 60, which skews things even more. Plus tent pole games like CoD can be executed extremely well but offer nothing new so how do you review that? There are games with low interaction (point and click) or high interaction (RTS). How do you compare one against the other? Good reviews are also often given despite massive bugs, incomplete games being released or week 1 launch disasters (like Diablo III).
It's issues like that which make me understand the no score review trend.
And watching a game on Youtube gives a very misleading impression of gameplay. In fact, I can't fucking stand watching other people play games - especially the kind of games I like, e.g. adventure, where the whole excitement comes from the challenge,
Try Twitch!
Trying to watch that with Flashblock / FlashControl and so on installed can be a real challenge! ;D
Sorry to burst your bubble but this is one situation where Betteridge's law might actually faulter. Game review scores have been broken for sometime and removing them entirely might be a step in the right direction.
The Steam User Score is currently my most trusted metric for how good a game is, something which is considered "overwhelmingly positive" with a couple thousand user reviews is usually a worthy purchase.
For non-steam users, imagine Metacritic except you can only submit your score/review if you own the actual game and it's either thumbs up or down.
The thing that I find increasingly aggravating nowadays is how much is hung on score rather than substantive view of the content of a thing.
For instance (on a sort-of related topic): when a highly-anticipated movie, like "The Avengers" is released for critics and the scores start coming in, and it turns out critics found the movie overwhelmingly positive, the fans get all hopped up when someone dares to give the film a "rotten" instead of "fresh", ruining a "erfect score, as if there was somehow some personal investment in a movie getting 100% of critics to like it (or spoiling of their enjoyment of it if a mere 1% did not).
Except for the fact that not all critics thought the movie was perfect, and the Tomatometer merely indicates that the movie was at least good enough not to be considered bad.
The score is the headline, sure, to draw in people to read the review in the first place. But a lot of people gloss over it and stop engaging their critical faculties, brandishing a metric over true criticism as validation of their personal tastes (like Rotten Tomatoes readers; if you don't believe that people do this, find out what happened to critic Eric D. Snider after he posted a fake negative review of "The Dark Knight Rises" before he'd actually seen it).
I don't have any "infamous" examples of games to point to, though I'm sure examples exist; in fact I wandered into this topic curious about which games were controversial in the same way, since both media have the same kinds of fanatics attached to them.
My thought is to get rid of scores so that people actually consume opinions, not reduce them to a single number, but that's just me.
Some people don't believe in fairies. I don't believe in The Patriarchy.
its a good direction, but someone should come up with an adjustment because:
a) pissed of payers rant more
c) pleased customers usually don't care about feedback
b) too many "happy" reviews that list more negative than positive points
x) positive reviews of sorts "please give them 1-n patches to fix a quadrillion bugs before you vote negative"
etc.
games will be considered Recommended, Essential or Avoid.
Translated to a 1-10 scale, that is 5-8, 9+, and 1-4.
Translated to 5 stars, that is 3+, 4.5+, and 1-2.
Better is to find a specific reviewer that favors the same types of games that you favor and read what they have to say about a particular game. Reviewers themselves should be given scores in different genres to reflect their interest, and scores in different aspects of games that don't necessarily translate between genres and are not necessarily used on every game (perhaps each reviewer chooses 3 most important factors of a set list of say, 10 different areas); then have multiple reviewers on each game.
How should we score an excellent game with severe networking issues? A flawlessly polished game with a hackneyed design? A brilliantly tuned multiplayer experience with dreadful storytelling? If you expect the score to encompass every aspect of a game, the task becomes an exercise in futility. Add an inflated understanding of the scoring scale in many quarters - whereby 7/10 and even sometimes 8/10 are construed as disappointing scores - and you have a recipe for mixed messages.
Excellent game with networking issues:
"Mary the FPS guru" says:
Polish: 9.5/10 "It's pretty!"
Networking: 4/10 "Networking problems ruins everything."
Replayability: 8/10 "Single player scenarios keep me coming back."
"Matt Foley the puzzle champ" says:
Team Balance: 8/10 "Pick your army, its all about skill"
Networking: 6/10 "It's ok because I live in a trailer down by the river!"
Price: 10/10 "Freeware, freeware, freeware."
You get the idea. Sorry for the babbling. No time to reword this.