Do Reviews Still Serve a Purpose?

Via Voodoo Extreme, a post on the Sony-sponsored ThreeSpeech blog asking if game reviews are a thing of the past. Post author 'Azz Hassan' opines that the proliferation of blogs and easy access to game trailers has made the 'biased views' of reviewers a thing of the past. Responding via the Ars Technica Opposable Thumbs blog, Frank Caron offers a rebuttal to the piece. 'The argument presented in the article seems to come with the very slant that it so viciously protests: one of a negative view towards a medium that the writer feels is inadequate. Yes, there is a ton of available media on the net that can help you get a look at a game as it develops, but the problem with videos and pictures is that often the intangible elements are impossible to understand simply from seeing the game in motion--only the written or verbal communication of a person can adequately capture these details.'

Comments

um

[mastershake_phd]

Yes, yes they do. If they didnt people wouldnt read them.

Re:um

[Bios_Hakr]

Well, it depends. I really started noticing crap when I saw an IT something magazine on my boss' desk. I picked it up and looked at it. There was a "review" of some kit we had worked with in the past. The kit was crap but the review was glowing. I stopped reading IT mags after that.

Later, I was reading a RC modeling magazine. I love RC helis. They had a "review" of a a heli that I knew was okay, but not great. They were comparing it to helis that were capable of doing any maneuver that the pilot could throw at it. I stopped reading RC mags after that.

Video game mags are probably the worst. I read PC Gamer for the commentary and previews, but I only read the negative parts of the reviews.

When it comes to specialized gear like an RC heli or a new router, I rely on comments in online forums. I'll jump into IRC and ask people about the bad points of the gear. I'll call the company and speak with engineers or tech support; speaking with sales is a waste. If I have to deal with sales, I ask for written documentation of tests displaying any functionality he claims. If they can't produce a document showing increased throughput, I ignore that point.

When it comes to daily items, I check boards and really read the negative Amazon reviews. I'll google $item + shit or $item + "head to head". I'll check Consumer Reports or check BBB for the company name.

If I don't find a negative opinion about an item, then I can be pretty sure it's untested or the company censors opinions. Either way, it's not worth my money. I read the negative reviews carefully. If the negative is whining, I ignore it. If the negative is a valid complaint, then I call tech support and pretend I have the item and have that same problem. How they answer my questions will determine my purchase.

Finally, just asking questions of my peers can give a lot of insight. I have *very* close contact with peers that work for competitors. It's a fairly small community and we tend to stick together. We usually share knowledge about our mistakes. If someone mentions over a beer that they are thinking about buying Wizbang 2008, then the rest of the group spills every bad thing they have ever heard about it. If the guy comes out of the discussion by answering our points, then we all think about giving it a closer look.

Re:um

[Das Modell]

I know one instance were user reviews turned out to be spot on while professional reviewers were, frankly, full of shit and probably on the publisher's payroll. I am referring, of course, to Neverwinter Nights 2. It has a Metacritic rating of 82, while the user reviews were almost entirely negative. The official forums had two sticky threads, one for negative reviews and one for positive reviews. The negative thread was restarted about four times before the first positive one was even full (the threads were replaced with new ones after a certain amount of pages). The forums were overflowing with complaints. I ended up buying the game anyway, againts my better judgement.

Re:um

[apoc06]

you point out the key issue with reviews. usually they revolve around a single users' experience. if the reviewer that plays the game has a penchant for say... FPS games, if he has to sit down and play a roleplaying game that is generally overlooked in light of its mod community, he is going to score the game lower. a reviewer that prefers tekken will be harder on a title like DoA [IMHO he should be]. whatever that users preferences are come into play.

what i think game magazines/ sites need to do is outline each reviewers tastes and have multiple users review the games. that way readers can find a reviewer whose tastes align more closely with theirs.

Re:um

[Hoi Polloi]

One thing I have noticed is the very nature of reviewing vs daily use. Reviewers are in and out. They won't go indepth into a game or hardware and won't cover long term playing or use of an item. Very few revisit, say, hardware like tomshardware will do sometimes. I've seen hardware with good reviews come back with user reviews a few months later saying "such and such died after a few weeks" or something similar. Games also usually get a review based on graphics but not on deeper gameplay since they don't put the time into it (gotta get the review out fast!).

I'll use the mainstream reviews for screenshots, basic info, specs, etc. If something is horribly bad about it they'll usually admit it. I can wait a month or so for the users to come out with their reviews which are usually closer to the truth. You'll still have the whiners ("It won't run on my ancient box!!") and impatient to sift through though.

There is so much user content for NWN-1 and the reviews for NWN-2 are so lukewarm that I'm passing on NWN-2. What pushed you to buy it against your judgement?

Re:um

[Das Modell]

What pushed you to buy it against your judgement?

Well, when the game was in development it looked really promising. I wasn't into NWN1 because its singleplayer was so lacking, but NWN2 seemed like it was putting more emphasis on the singleplayer campaign. I was also in desperate need of a good RPG. I guess I just wanted to really believe that the game wasn't as bad as the user reviews claimed.

The more hyped a game is, the more disappointed I am in it.

Re:um

[Dutch Gun]

I'm convinced the in-and-out quick process of professional reviewers is why Oblivion scored so high. After my first 10-15 hours of gameplay, my reactions were the same - almost all positive. Unfortunately, after that, the serious game-balance issues caused by the auto-scaling made me lose interest. It's particularly tragic becase, other than that game-breaker, Oblivion is nearly everything I'd want to see in a computer RPG.

I've always thought that the logical way to use reviews is to find a reviewer that tends to match your general tastes, and weight his/her reviews accordingly. I've always wished that review sites grouped reviews by reviewer, so we could choose who we want to listen to and who we want to ignore. The idea of an "unbiased" review is a fallicy. Everyone brings their own preferences and biases, and I wish gaming sites would actually exploit this fact rather than try to surpress it.

Not to me

[Xest]

Because I grew up, got a job and am now gullable enough to buy any crap the games industry throws out. I suffer from obsessive compulsive computer game purchase.

Oh how I miss the days of being dependant on pocket money where every penny had to be spent so wisely.

Re:Not to me

[MichaelSmith]

Oh how I miss the days of being dependant on pocket money where every penny had to be spent so wisely.

You should get married then.

Depends

[El_Muerte_TDS]

On who (the actual person) wrote it, and how well you know the reviewer. Personal preference is always a big factor in game/movie/music reviews. It could very well be that I like a game what a reviewer gave a bad review, but I would only know that if I knew the reviewer's preference.
Ofcourse demo'ing the game is always better than reading a review.
The most useless part of a review is the grade, it says absolutely nothing, except what number the reviewer assigned. They might as well use colors for grading instead of numbers or stars. So... I rate the linked article: purple.

Re:Depends

[grumbel]

### The most useless part of a review is the grade, it says absolutely nothing,

I disagree, ratings give you a simple value to compare different games of similar genres. Sure, it doesn't make sense to compare a The Sims rating with a GTA one, since the games are just vastly different, but comparing GTA vs Crackdown is perfectly doable. Ratings also give you a very direct way to see what the reviewer thought of a game, when the review text just mentioned that the graphics are "good", how good is that "good", is that a RE4 "good" or just an average "good"? A 10/10 in graphics on the other side easily tells me that its among the best to expect on a console.

Beside pages from this, ratings are important for sites like Metacritic which would be rather hard to use without a final rating. When I want to get an impression from a game I search for the reviews that gave it the highest score and those reviews that gave it the lowest, thus I get a good overview of how somebody who likes the game views it vs somebody who doesn't like it. If there would just be text things would get rather hard to find the right reviews.

All that said, rating numbers are of course heavily flawed, many reviewers rate almost every game in a 70-90/100 area and don't make much use of the rest of the scale, another issues is that ratings are often tinted by non-game related issues, like price, if it is a port of an old game and such, which however might not matter at all for having fun with the game, especially since price can lower over time and as long as I don't already know the game it is still 'new' for me. Ratings are also a one dimensional scale, while you really might one a multidimensional, i.e. there are many games out there that are great by concept, but also heavily flawed in implementation, those however just end up in the 70-80% region, which tells little about there flaws or great concept, but just tosses them together with all those games that have an uninteresting concept but flawlessly implemented.

With all the flaws I however still consider ratings far more useful then harmful.

They do

[ShakaUVM]

Reviews let me know if a game totally sucks. Then I avoid it.

But positive reviews are no guarantee of a good game, as the glowing ratings for such moribund stinkers as FFVII and FFX can attest to.

Re:They do

[Tickenest]

I bet your TF mod got poor reviews. :P

Re:They do

[Jim Hall]

Reviews let me know if a game totally sucks. Then I avoid it.

Except in the case of Jaws Unleashed for PS2. In that case, I read the reviews, then immediately ran out to buy a copy. Sometimes, you just have to re-set the metric on "bad". I mean, when someone says "this game sucks" you need to have a metric of how badly the game sucked. Did it suck "Jaws" bad, or "Mark Eko's Getting Up" bad?

And yes, "Jaws" was probably the worst game I've ever played. SPOILER: The best part in the game happens very early on: you are trapped in a tank in a lab, and the only door is controlled by card-key. So you need to get a card-key to open it and make good your escape. But you are the shark so it's not like you have hands. You need to grab a scientist with a key-card, but not eat him, wave his body in front of the card-reader, and the door will open.

Re:They do

[heinousjay]

When's the last time a game got a 0, 1, or 2? I've played games that might have gotten a 6 or 7 that I think deserved a 2 or 3. ...I check some game review sites, like IGN...

I think we've found your problem.

They definitely matter to me

[Don_dumb]

These days I buy only a few games and to try and spend my money on something I will enjoy, what choice do I have? I can either look at all the pictures or I can read what someone says when they have actually played through the game. It isn't just the final score that I look for, there are certain things I dont want in a game (such as loads and loads of unforgiving stealth in an FPS) and I use reviews to try and see if those elements are present, if they are I wont bother wasting my money. Most reviews I have seen will also talk about the hardware requirements. A game that requires massive power (while the official claims are for a mediocre system) is also out.
Yes, just like movie reviews they are someone elses subjective view, but to get your own view you would have to watch every movie made or judge them by the trailer. Both of which are far worse (in my eyes) than seeing the opinion of a reviewer that generally agrees with you and has themselves seen almost every movie ever made.

Having said that, reviews are less useful than demonstration versions, which I wish game makers would use more.

Absolutely they do

[tttonyyy]

...but maybe not as individual reviews.

http://www.metacritic.com/ is a fantastic site which does weighted averaging of scores from many reviews. I use it for games in particular - it's useful to check the reviews that give a high score against the reviews that give a low score to see what is good and what is not about a game before buying. The "averaged" score almost always corresponds with my experience of the games too, so the system seems to work.

So reviews do serve a purpose, but, as with many things in life, to get a balanced opinion you need to sample from a set great than 1.

Reviews serve to limit need to gather info

[Protonk]

Reviewers are just like political parties, insofar as they help distill a vast amount of information in order to allow us to actually make some decisions. The point of political parties isn't to provide perfect information to the voter, the point is to allow the voter to reduce the complexities of the ballot when necessary.

Before you guys get out of hand in the comments, by that I mean that it is functionally impossible for us as voters (in any country) to vet EVERYONE, from the county clerk to the State Senator (Okay, sorry, I don't have a region agnostic example, deal). We may decide on the president based on our input from non-party sources, but the other 18 names on the ballot don't rise to the same threshold. Parties allow us to make an assumption that a representative will align to the basic ideas that we are interested in.

Reviewers serve the same function. I may decide that I 'trust' a particular game or movie reviewer. As a result, I can presume that his/her views on a game are a good proxy for my own. This allows me to narrow the field of games I might be interested in without covering every demo, every press release, and every industry whisper--not to mention playing the game. In this sense, reviewers are even more necessary, because in order for me to make an adequate decision about a game in the absence of press, I would have to play it (or a demo, but even that isn't perfectly enlightening, see Lost Planet). That, of course, would obviate the need for the review.

In this case, just like political parties, we learn to accept bias in our reviewers. In most cases, the biases are benign--we share them. We like that Rogert Ebert doesn't like M. Night Shyamalan, because we don't (obviously only speaking for some of us). We like that the Onion (and pretty much everyone else) hates Uwe Boll, because we hate him. The same thing with the Democratic and Republican (insert Labour/conservative, etc) parties. We accept their biases (when we do) because we share them to some extent.

The case of bias in favor of a game publisher is a little more insidious, and is something that the game press will have to work out, and I suspect that it won't work itself out by eliminating the review. I suspect that certain reviewers (Ars, to name one) will gain greater acceptance as the rest of them keep shilling for bullshit. The same thing happened to the Democrats in the South. The south changed (beyond racism/segreation, which really only explain the first 10-15 years of that seismic shift), becoming more religious, individualistic and pro-business and the Democrats didn't adapt, so the south moved on to the republicans.

excellent

[gEvil (beta)]

Excellent commentary. 8/10

Too lazy to try out game demos and watch videos

[wildzeke]

So reading the conclusion of a review might indicate a game is worth trying out. BTW, can someone read the article for me and give a summary.

Thanks.

useful

[minus_273]

Reviews for games are like movie reviews. If you can find a reviewer(s) who have generally similar tastes, you will be able to judge a game with a fair degree of accuracy before you buy it. In addition without those rewiews, mega hit new games like God of War probably wouldnt be so big. I suspect a significant degree of its popularity came from people going to review sites and seeing the good reviews.

On the other end of the spectrum, how many more Final Fantasy fans would have bought Dirge of Cerebus had the reiwes not told us it was junk?

There's an argument to be made...

[Pluvius]

But it's not the one that's presented. Thinking that reviews can be replaced by trailers is just silly. Trailers are nothing more than ads, and don't really tell you much about what's going on. Neither do demos; even if we assumed that every game has one, oftentimes a demo is much worse or much better than the game that it represents. As for blog opinions, those in themselves are basically reviews, though usually not very good ones unless you have a good idea of the taste of whoever's writing them.

What goes a long way towards making reviews pointless is GameFAQs. No, not the reviews on there; they often suck. I mean the FAQs themselves. A good FAQ will tell you most of what you want to know about the game in great detail in the first few sections, often without spoiling the plot in the process. The only problem is that FAQs require time to be written, time that simple reviews don't need.

Rob

Review == Opinion.

[ThePhilips]

I use review quite simply: I'm looking for reviews of games I know and like - and choose people/sites who have rated games I like highly. And then check what else they have rated highly. That way I have found PC's "Rise of Nation", "Heroes of Annihilated Empires" and "City Life World Edition" - IMHO great games I enjoy and play, but most of high profile review sites have given them crappy/misplaced ratings.

E.g. http://wii.ign.com/ fits me perfectly. But on other side http://ds.ign.com/ - is U-turn in the respect: they gave lots of near-perferct marks to IMHO shit games (e.g. Mario, Partners in time) and underrated lots of games I have liked (e.g. Lost Magic). Reviews on ds.ign.com marked as "UK" are pretty O.K. and mostly fit me.

They have value

[dreemernj]

But they are less or more valuable depending on the person or on what they are about.

I find reviews of RPGs and action games helpful, especially when the reviewer knows their stuff pretty well and starts drawing comparisons with other games, because chances are I'll know at least a few of the other games the reviewer refers to.

But then for something like a fighting game, unless the reviewer is a dedicated fighting game player, I don't find the reviews useful because I know fighting games well and I know specifically what I like about fighting games. The review still has value to the person that just casually plays fighters just like RPG reviews have value for me as a casual RPG player.

Ramblings about Reviewing Enjoyment

[Bobfrankly1]

Reviews are still a useful tool in considering a purchase, but they should not be the end all decision maker. But what one has to remember is that these guys are reviewing what can be compared to art in some ways. Everybody has different taste, gets different levels of enjoyment from different games. Flame wars often occur over a preference of games, and this often demonstrates the difference of tastes (and lack of spelling) that exists in the gaming realm.

Reviews are very useful for determining frustrating issues, such as sloppy controls, lousy camera angles, and bad story translations and speeling. The best reviews are able to draw similarites to other existing games, thus relating the content or style of a game while not confusing the reader with his or her personal taste. I'm rather stingy with my money, but some reviews (mostly at ign.com) have turned me on to games I might otherwise have never known, but throughly enjoyed (Altier Iris series for example).

PC Games you can often easily demo, but not so frequently with many console games. Thus console game reviews are more useful then PC Games reviews.

-Never trust a skinny cook

How it should be done....

[kn0tw0rk]

.. is the way the Retro Gamer magazine articles on "Why you must play ..........."

What they do is gve a detailed break down of the what gameplay elements are used in the game, and how those work to enhance the gaming experience and make it an enjoyable experience. Also they give some history of how they game came to be and the situation of the gaming market it was released into.

As the games mentioned are not being sold in retail stores, there is no $$$ coercion factor that you get with current games.

I've tried several of the games and have discovered some worthwhile divertions of my time.

Cheers

These opinions are all very important, so thanks!

[StarFire2258]

As a future (read: aspiring) videogame reviewer myself, I wanted to thank all current and future comment contributors to this thread. It is the duty of every publication to serve its audience. It's good journalism, and good business sense as well. Every person commenting here who cites a problem with current videogame reviews shows that there is a disconnect between that audience (you) and those publications, and I for one will not let that disconnect go unnoticed. I plan to improve the state of reviews, starting with my own and hopefully inspiring others by example. Therefore, I've saved the current comments to this thread, and will return in one week to do so again, collecting more recent comments. I'll then integrate their ideas into my future reviews, to better serve you all. IMHO, reviews serve an incredibly important purpose, that of informing consumers not as information-savvy as all you /. readers as to which games are worth the money and which should be best avoided. For those who have the time and interest to track new games as they are developed, they are perhaps not as important... but I for one will not purchase a game without reading the review of at least one reviewer I trust.