Slashdot Mirror
Are Review Scores Pointless?
donniebaseball23 writes: With Eurogamer being the latest popular video games site to ditch review scores, some are discussing just how valuable assigning a score to a game actually is these days. It really depends on whom you ask. "I've always disliked the notion of scores on something as abstract and subjective as games," says Vlambeer co-founder Rami Ismail. From the press side, though, former GameSpot editor Justin Calvert still believes in scores. "I've been basing my own game-purchasing decision on reviews ever since I picked up the first issue of Zzap! 64 magazine in the UK almost 30 years ago," he says, while admitting that YouTube is certainly changing the landscape today: "There's something very appealing about watching a game being played and knowing that the footage hasn't been edited in a way that might misrepresent the experience."
I've found Metacritic to be a good aggregator of scores, but more importantly, the "users" scores (and reviews) tend to be more reliable in terms of not being overly critical of games that are generally pretty good, but don't meet the expectations of "hard core" gamers.
Betteridge's Law indicates that the answer is "no", when of course, the answer is actually "duh".
I only read Bennett Haselton reviews (& also reviews of him).
Just sometimes? Or all the time? Can it mean free gameplay? Is it inforced? I would be curious to lean more about this as it sounds like a way to avoid bad games and also play them for free. Can you provide linky?
But they should be interpreted correctly: first ask to whom is the scorer trying to appeal. Just as any test designed to reduce something to a point on a spectrum should be very carefully interpreted.
And watching a game on Youtube gives a very misleading impression of gameplay. In fact, I can't fucking stand watching other people play games - especially the kind of games I like, e.g. adventure, where the whole excitement comes from the challenge, not from watching other people having already solved all the problems running through them quickly, or RPG, where again the challenge comes from sitting back and planning, not watching others bash their way through. So, for me, someone's description of how they felt about a game as a whole is way more useful than watching them play it for a few minutes.
If you're rating most games within an interval of [7,10] then yes, you should probably rethink giving a numerical score.
When was the last time you saw a game rated less than a 6/10?
Is the real question
Its not the the scoring system in these mags is bad or worthless, its that the reviewers themselves are increasingly becoming shills for the gaming industry. When you see ads for the game right next to the "review", its a good sign that the magazine might have some vested interests in giving enthusiastic reviews.
This seems like a rather pointless question, since 'reviews' and 'review scores' serve somewhat different purposes.
If you want a comparatively deep examination of a game, strengths, weaknesses, what is it trying to do?, does it succeed?, who is it aimed at?, etc. an answer like "65" or "8" is practically useless. If you want to do a metacritic-style survey(or decide what long-form reviews to read when faced with 2,000 games), though, 3 pages of prose and musing, each, from two dozen sources isn't going to cut it.
Anyone pretending that a hundred-point score is actually that precise is likely fooling themselves; but there's a much stronger argument that you can get at least a 1-10 or so scoring system unless you are a pure, handwaving "It's all, like, intersubjective, man..." type.
As long as the score focuses on "Should you buy this?" and isn't a weight of various factors which doesn't end up in IS IT GOOD / SHOULD I GET IT?!
I guess they are just fine and helpful.
If it was a board-game the amount of miniatures and artwork on cardboard and so on doesn't really help much if the game is shit.
Almost no games get below 40, while any game that doesn't get 80 or more is considerd a failure. Then you have people giving games 3 out of 5 stars which translates to a score of 60, which skews things even more. Plus tent pole games like CoD can be executed extremely well but offer nothing new so how do you review that? There are games with low interaction (point and click) or high interaction (RTS). How do you compare one against the other? Good reviews are also often given despite massive bugs, incomplete games being released or week 1 launch disasters (like Diablo III).
It's issues like that which make me understand the no score review trend.
No.
The scale is all wrong for most review sites.
On sites that use the 10 pt system (10.0 being the best possible score), anything lower than about an 7.0 is typically unplayable to anyone except perhaps die-hard fans of whatever source material the game is based on in the case of movie / book / comic based stuff.
Basically I read 0-10 scores as:
9.5-10.0 - must play. will probably be awesome, and will be disappointed if its not.
8.5-9.5 - must play if in genre I like / typically play. a "maybe" if it's not.
7.5-8.5 - will play if it's a genre I like or looks like an interesting story. will avoid if it's not a genre I like.
7.5 - probably utter crap.
Asking if the number is useful is supposing reviews are honest and unbiased.
Since most reviews are prohibited from coming out before the game, and one assume most of these websites are getting paid for favorable reviews ... the number they put next to their review is probably as meaningless as the entire review in a lot of cases.
Lost at C:>. Found at C.
because those review site review scores are absolutely pointless. So many are fake and paid for.
One person's Schindler's List is another person's Bad Taste. Like reading wine scores, you'll find there are particular reviewers that you agree with and those you don't. When looking at reviews of games you've enjoyed, find the reviewers that agree with you in terms of both score and analysis. Fall back on those gamers for reviews. It's not always possible -- many gaming sites have different people review games, there is staff turnover, etc.
The score will give you a general idea of whether the game is rubbish or not. Scores of 5 or below generally mean it has bugs or serious gameplay issues, regardless of whether it's fun. Scores of 9 and above mean that the reviewer truly believes it's a great game for its genre. Scores between 5 and 9 are subjective and worth reading the analysis.
But you know, it's all subjective. I loved the Dragon Age games -- but Inquisition...it's just too much right now despite the rave reviews. Meh. Shadow of Mordor was the right game for me at the time.
... are paid advertisements and just publishing arms for the game industry. Let's be honest. The entire gaming press is nothing but propaganda of the industry. They are an industry mouthpiece masquerading as journalism.
The Steam User Score is currently my most trusted metric for how good a game is, something which is considered "overwhelmingly positive" with a couple thousand user reviews is usually a worthy purchase.
For non-steam users, imagine Metacritic except you can only submit your score/review if you own the actual game and it's either thumbs up or down.
Yes, review scores are pointless if you use letter grades for your review!
I'll be here all week...
Actually the numbers themselves are almost meaningless but are a nice general gauge, especially when they are user reviews.
The correct way, I think, to use them is to look for a few high scores, and then read the low and medium score reviews. Are the 1 star reviews people who don't even like this type of game? Thats actually a GOOD sign of a decent game.
Are the 1 star reviews complaining about bugs and play control? Watch out.
Overall though, after checking this stuff out, I almost always watch at least the first 5-10 minutes of gameplay walkthrough before deciding to buy. I hate being duped by cutscenes.
"I opened my eyes, and everything went dark again"
Excellent question on an important topic. 7/10
Obliteracy: Words with explosions
As far as I can see the reviewers are little more than lying shills that make up the numbers anyway.
Here is the title of a review column I saw on a popular site
So You Don’t Like TSW? I Don’t Care (if you want to read it google for it, I have no intention of driving traffic there)
Turns out the "Reviewer" was working hand in glove with games community management, they might as well have just taken out the scores and put up a dollar sign with how much they charged for the review.
Simple numeric scores are useful in the same way that having a college degree is useful - it enables people who lack the time or motivation to perform an in-depth examination of a candidate or game to make a snap decision based on a single, over-simplified criterion.
The thing that I find increasingly aggravating nowadays is how much is hung on score rather than substantive view of the content of a thing.
For instance (on a sort-of related topic): when a highly-anticipated movie, like "The Avengers" is released for critics and the scores start coming in, and it turns out critics found the movie overwhelmingly positive, the fans get all hopped up when someone dares to give the film a "rotten" instead of "fresh", ruining a "erfect score, as if there was somehow some personal investment in a movie getting 100% of critics to like it (or spoiling of their enjoyment of it if a mere 1% did not).
Except for the fact that not all critics thought the movie was perfect, and the Tomatometer merely indicates that the movie was at least good enough not to be considered bad.
The score is the headline, sure, to draw in people to read the review in the first place. But a lot of people gloss over it and stop engaging their critical faculties, brandishing a metric over true criticism as validation of their personal tastes (like Rotten Tomatoes readers; if you don't believe that people do this, find out what happened to critic Eric D. Snider after he posted a fake negative review of "The Dark Knight Rises" before he'd actually seen it).
I don't have any "infamous" examples of games to point to, though I'm sure examples exist; in fact I wandered into this topic curious about which games were controversial in the same way, since both media have the same kinds of fanatics attached to them.
My thought is to get rid of scores so that people actually consume opinions, not reduce them to a single number, but that's just me.
Some people don't believe in fairies. I don't believe in The Patriarchy.
The essential / recommended / avoid rating only works if you happen to find a reviewer whose tastes exactly match yours. Which granted given the enormous number of reviews on the Internet is a lot easier now than in the dead tree gaming magazine days. But the key thing to take from that should be that you need to somehow take advantage of the large number of reviews, not simplify the scoring system. The review site should use pattern matching where you list which games you liked, the site searches for other users who liked the same games, and makes recommendations to you based on games those people liked but weren't on your list. Like what Netflix does for its movie recommendations. Essentially, the entire population of game players registered with the site becomes your review database, and the system is set up to weigh more heavily the opinions of gamers who have similar tastes to yours.
Youtube videos of gameplay are good if you already know you might be interested in a game. But just as the Internet has led to the balkanization of reviews so you can now find thousands of reviews, game development is slowly going down the same path (have you ever browsed through all the titles available on Steam?). Consequently, there are a helluva lot more games out there to consider (unless you artificially limit yourself to just the big titles), and some sort of review/rating system is needed to help people sift through them quickly. Watching a youtube video of each one is out of the question. Same goes for free trials - the time investment makes it unwieldy as the number of available games increases.
Similar to Confirmed, Plausible and Busted of the mythbusters a system where the reviewers grade the game into six different levels would be good.
1. This game is really special and well executed [classic high 80s or 90+]
2. This game is Good [classic 80+]
3. For Fans of the Genre and those trying a different experience. Not bad, but nothing you have to see. [classic 70+]
4. Careful, you can have fun and it generally works, but you should expect annoying issues, better read the review [classic 60+]
5. Nothing to see here. You might have fun if you're a fan of the idea.
6. EVADE! Really, this is bad [classic 40-, big RED if it hits 30-]
To see if I should bother even watching the trailer, if I like what I see in the trailer I will watch a review (usually at least two) THEN I go and buy the thing. I also use review aggregators to find out which good games are coming out, there are so many games coming out and I don't keep up with game news so in that regard the current system works very well for me. Although Steam user ratings are getting more useful by the day I still use them only to then check the reviews out.
The review scores are like a broken pencil...
Violence is the last refuge of the incompetent. Polar Scope Align for iOS
yes or no ?
Board game geek attaches a guideline to the number score, so scores are slightly less arbitrary. http://boardgamegeek.com/wiki/page/ratings
Sites like metacritic for games or rottentomatoes for movies tend to provide the most usable results for me. The contrasting reviewer types and ability see at least some negative comments really help sort out the worst choices.
its a good direction, but someone should come up with an adjustment because:
a) pissed of payers rant more
c) pleased customers usually don't care about feedback
b) too many "happy" reviews that list more negative than positive points
x) positive reviews of sorts "please give them 1-n patches to fix a quadrillion bugs before you vote negative"
etc.
While lots of people love to hate Metacritic, and there are some flaws for sure particularly if people use it is a "good-o-meter" I find it to be really useful. If a game has really high scores, I find that it is usually worth my time to check out, even if it isn't the sort of game I'm usually that in to. If all kinds of people loved it, well it probably does something right and I may well enjoy it. Conversely if a game that looks interesting gets really low scores, I may wish to investigate why, as usually it is a sign that something is seriously fucked and I probably don't want it.
More middling scores, well then that's more of an "it depends" situation. Need to read some reviews and see what they actually liked and didn't and so on.
Makes for a useful executive summary though. For many games I want a real quick metric to decide if I should spend further time looking in to it. I'm not willing to spend a couple hours on every game reading/watching multiple reviews. For some games, my interest in them is only passing and an aggregate score can be a good way to tell me if I should look at them harder or just give them a miss.
For example Bioshock Infinite was on my "give it a miss" list since I really didn't enjoy the first one. I wasn't even going to look in to it. However the extremely high scores made me think "Well, you should look at some reviews of it, that is pretty universal praise."
games will be considered Recommended, Essential or Avoid.
Translated to a 1-10 scale, that is 5-8, 9+, and 1-4.
Translated to 5 stars, that is 3+, 4.5+, and 1-2.
Better is to find a specific reviewer that favors the same types of games that you favor and read what they have to say about a particular game. Reviewers themselves should be given scores in different genres to reflect their interest, and scores in different aspects of games that don't necessarily translate between genres and are not necessarily used on every game (perhaps each reviewer chooses 3 most important factors of a set list of say, 10 different areas); then have multiple reviewers on each game.
How should we score an excellent game with severe networking issues? A flawlessly polished game with a hackneyed design? A brilliantly tuned multiplayer experience with dreadful storytelling? If you expect the score to encompass every aspect of a game, the task becomes an exercise in futility. Add an inflated understanding of the scoring scale in many quarters - whereby 7/10 and even sometimes 8/10 are construed as disappointing scores - and you have a recipe for mixed messages.
Excellent game with networking issues:
"Mary the FPS guru" says:
Polish: 9.5/10 "It's pretty!"
Networking: 4/10 "Networking problems ruins everything."
Replayability: 8/10 "Single player scenarios keep me coming back."
"Matt Foley the puzzle champ" says:
Team Balance: 8/10 "Pick your army, its all about skill"
Networking: 6/10 "It's ok because I live in a trailer down by the river!"
Price: 10/10 "Freeware, freeware, freeware."
You get the idea. Sorry for the babbling. No time to reword this.
...on their own, since I typically pay more attention to the descriptive contents of reviews.
However, if I'm looking around for a new piece of hardware (say, a camera), I'll pull up a site's reviews and have a look at the top 10 or 20 or whatever number, and start reading the reviews.
The numbers give a way to sort the reviews and give me a place to start looking.
My two cents...
Game reviews are good at generally identifying the best games, but I wouldn't rely on them for more than a very rough metric. You simply have to play a game to know if you like it.
The best game review I've ever experienced was a live stream of a well-known GTA V speed runner playing the game on next generation hardware. Speed runners know games almost as intimately as the programmers do. The conclusions of the speed runner were dead on. The game has beautiful graphics, but playability had declined in many areas. He immediately noticed very long load times. And what do you know, after a few months people are complaining on Youtube about load times for GTA V on next generation hardware. One thing that I notice on Youtube is that pop in seems to be worse on next generation than previous generation. That's sad.
This 8 hour or so run (I wasn't tuning in continuously) convinced me to not buy a PS 4, a new television, or the next generation version of GTA V. I may take the plunge when the Windows version is released, but I'm going to see how the speed runners like the game. They notice everything right off the bat.
I'm perfectly happy with the PS 3 version of GTA V. It's the only game I play when I'm in the mood for video games.
Depends on if those scores can be trustworthy or not from paid reviewers. Especially if the reviewer is friends with a developer, or is getting back room deals/accessories from the production studio to bump the score higher.
I prefer user scores, sure there will be lots of trolling from people who hate the game for some reason, even if they didn't play it, but a majority of game reviews from users will be trustworthy.
Came looking for Gamergate fodder; left disappointed. :-(
-1, Too Many Layers Of Abstraction
If you find someone that has similar interests as you then their reviews will be good for you specifically. However, general aggregate reviews or population votes are only correct as an average and may have zero accuracy for any specific individual.
Actually I jest, I completely agree that rating scores for pretty much anything are useless. A number expressed as a percentage out of 10 or 5 is simply not enough to tell you if a product is good anymore than a statement of "Do/Don't buy this product." In this case is why most of the five star rating systems are either 1 star (don't buy) or 4 (do buy), nobody gets a 5.
I'm a good cook. I'm a fantastic eater. - Steven Brust
Review scores have always been worthless, with the sole exception that very low scores are almost always indicative of crap. Higher scores don't tell you what you need to know. For instance, I've seen games with terrible gameplay get high score simply because they're visually beautiful. That's probably legitimate for certain types of players, but it is actively misleading for other types (like me) who just want the gameplay to be excellent and couldn't care less how beautiful the art is.
This headline violates Betteridge's Law, and therefore constitutes pro-review score propaganda. I call shenanigans!
A big part of the problem recently is that 'consuming opinions' means that you are getting all the internal bias of the reviewer, warts and all. That means if they get all antsy about not having a playable female character, then the game is toast. If the game has been criticised by Anita Sarkeesian (or will be) then it's awful. If it doesn't cater to alphabet soup people, then it's sucking up to the privileged white males and therefore bad.
So when you have opinions, all you end up with is the game 'media' pushing drivel such as depression quest and gone home. While review scores have plenty of problems, it also allows the consumer to also compare review sites. With a subjective opinion piece, where you have plenty of writers who make a point of not being objective, it just becomes a piece on what's bothering the writer this week.
People like totalbiscuit have proven that you can review games, and still be objective and fair. So while reviews are intrinsically an opinion, some are far more valid than others, and most of these game sites, are best forgotten about.
Also forgot to add that the game sites also tend to gloss over issues which plague a lot of games. Namely debilitating DRM, day one DLC and early access. If they don't stand up for the consumer, then what are they for?
Its kinda obvious that there is an issue when many of these scores are posted by 'authorities' to whom the game companies pay advertising money.
And other (non game magazine) reviewers have a herd mentality to not voice truth if it breaks with the rest of the sychophants.
Even within some of the video reviews you hear " the game is great, and perfect, 5 out of 5, etc..." then they go on to list (serious) things wrong with it - yet has a score of 96% or 9.8 or whatever.
Even some reviewers who voice real reasons why a game is faulty mention the hate mail they get for simply stating the truth.
Grading on a curve? The least stinky pile of shit is 'paradise' ???
As long as the players keep paying for garbage the companies need not create anything better.
They're great for knowing who has publisher dicks in their mouths. So when I know for certain a game is going to be bad and see, oh let's say, completely hypothetically, IGN give it a 9.5/10, I know IGN is sucking cock and is probably going to get a kickback. "God damn fucking IGN get that publisher cock out of your mouth! We all already know better than to buy a $60 game on your say-so already!" is what I would want to say to them when I see them sucking cock like that! Of course this is a completely hypothetical example, and I'm sure that the quality of IGN's integrity, like the quality of their articles, will never actually come into question!
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
Only those scores automatically aggragated by the community of gamers mean anything - see Valve's auto score system. Note how Valve's automatic assignment of a text string to low scored games ("mixed reviews") shows how if you rely on the company that displays the game to score the product, you will not have reliable information.
The audience for any review is 2-fold. .game.
The buying audience, advice whether to bother downloading, signing up, or learning,
and the developer, to provide feedback what worked what didn''t and what's wrong with their
I am not an avid gamer, don't own a gaming PC, nor any consoles, but I do enjoy the occasional strategy game.
Scores on reviews are good for filtering. That's whether I buy a game, for download, or any other kind of product.
For products that are hard to find and few users, I also look at products with low scores. The low scores on specialist products are usually because of user error, or someone who bought the product for a different purpose than it was made for. Yes, racing bike saddles are uncomfortable when riding in jeans. They are supposed to be used with padded shorts, dummies. Yes, biking shorts with padding will give you chafing if you wear underwear. They are supposed to be worn commando, dummies. Etc.
Anyone who is not an imbecile should after narrowing down the selection, consider some more points.
How many persons posted reviews. Anything with just a handful reviews, all positive, is probably posted by the developer and nearest family.
Anything with lots of positive reviews, but with the full text having odd language issues, odd and repeated wording, might be astroturfed.
The read a few of the worst and the best reviews. Using your own knowledge in the area as a reference, make an assessment of whether the revievre is competent and truthful.
Sure, you can miss out on some gems when filtering out only high scored games. However, if a game is specialized enough, you would know about it anyways if it was your cup of tea, since there would be so few to chose between in a search, And for more popular genres,a very low score probably meand the game has poor playability or lots of bugs, and you can save yourself some time and not even bother looking at the full reviews.
I don't read gamers magazines. I imagine if I had paid the $$ fro a magazine subscription I'd at least skim every review of games that fit my genres, regardless what the score was.
-- Another senseless waste of fine bytes.
They're dropping numbers in favor of words. So the review system is completely in place. What was nice was when game magazines had three people review a game, usually one in depth and two smaller reviews. You could see in the mag which authors preferred what kinds of games and then with their reviews you could be like, oh I always like Bobs reviews, his reviews line up well with what I like. The personalization is gone with all the meta reviews.