Are Review Scores Pointless?

donniebaseball23 writes: With Eurogamer being the latest popular video games site to ditch review scores, some are discussing just how valuable assigning a score to a game actually is these days. It really depends on whom you ask. "I've always disliked the notion of scores on something as abstract and subjective as games," says Vlambeer co-founder Rami Ismail. From the press side, though, former GameSpot editor Justin Calvert still believes in scores. "I've been basing my own game-purchasing decision on reviews ever since I picked up the first issue of Zzap! 64 magazine in the UK almost 30 years ago," he says, while admitting that YouTube is certainly changing the landscape today: "There's something very appealing about watching a game being played and knowing that the footage hasn't been edited in a way that might misrepresent the experience."

Comments

Meta scores and user's meta scores

[Deffexor]

I've found Metacritic to be a good aggregator of scores, but more importantly, the "users" scores (and reviews) tend to be more reliable in terms of not being overly critical of games that are generally pretty good, but don't meet the expectations of "hard core" gamers.

Re:Meta scores and user's meta scores

[jclarker6]

Totally agree with this. And taking it a step further one could say any single score on it's own is not that reliable, when taken in aggregate the cream definitely rises to the top

Re:Meta scores and user's meta scores

[houghi]

Scores by themselves are useless in many cases. I once was heavily involved in a customer service survey. It was basically "From 1 to 10, how do you like the service." What we noticed what that Nordic countries gave a complete different number compared to Mediteranian countries.

First they thought it was because the service was much better in some countries compared to others. Looking into it and asking customers we found nothing.

We then started asking a second question: "What service did you expect." and then measure the difference. So if you expect a 6 and you get a 6 it is much better then expecting a 9 and getting a 7. So a 6 can be better then a 7.

The issue was that the first time we did not have a base to start from. In school-tests the base is pretty easy. 100% is perfect without any errors.

Compare it to Americans and Enlish where one would say "Wow, this is AMAZING. It is the best I have ever seen." and the other would say "It's not bad." (I jope you get what I mean.)

When I look at at scres for movies, restaurants, books or whatever, I read the comments to know WHAT they thought about it.

Re: Meta scores and user's meta scores

[Anubis IV]

I'd go even further than that and say that it depends on the type of scale being used as well.

When it comes to user reviews, if the reviews are thumbs up or down, I'll do the same as you and read the thumbs down reviews first, since it's easier to filter out the extreme reviewers and get a sense for the common issues. If it's a 5-point scale, I'll read through the 2s and 4s, since those reviews can give you a quick understanding of the pros and cons for the product, without nearly the level of overstatement that you'll need to filter through in the 1s and 5s. And I don't even bother reading reviews based on 10-point scales, since the way that everyday users grade on a 10-point scale is arbitrary to the point of uselessness (e.g. some people treat it like a 5-point scale with better granularity, while others treat it like an academic scale).

Re:Meta scores and user's meta scores

[Anonymous Coward]

About Nordic countries giving different answers...
Look up into which grading systems are used in schools.

Norway has a scale of 1 to 6, with 6 being best(grade school only).
Finland has a scale of 4 to 10, with 10 being the best (grade school), and 1 to 5, 5 being the highest (higher education).
Sweden uses letter grades, A, B, C etc.
Iceland has a scale of 0 to 10, with 5 - 5.99 being 3rd, 6.0 - 7.24 being 2nd and 7.25 - 8.99 being 1st grade. 9.00 and up is a Fine grade. This is very similar to the grading system used in Greece, and somewhat similar to the one in Spain.
Denmark has a 7-step grade, 12 being highest and -3 being lowest (don't ask, I have no idea either).

It's weird all around Europe. In Germany, grades are 1 to 6 but nr 1 is best! In France, it's 1 to 20.

I never realized it was this bad. I just lost a lot of respect on numerical review scales...