Do Reviews Still Serve a Purpose?

Via Voodoo Extreme, a post on the Sony-sponsored ThreeSpeech blog asking if game reviews are a thing of the past. Post author 'Azz Hassan' opines that the proliferation of blogs and easy access to game trailers has made the 'biased views' of reviewers a thing of the past. Responding via the Ars Technica Opposable Thumbs blog, Frank Caron offers a rebuttal to the piece. 'The argument presented in the article seems to come with the very slant that it so viciously protests: one of a negative view towards a medium that the writer feels is inadequate. Yes, there is a ton of available media on the net that can help you get a look at a game as it develops, but the problem with videos and pictures is that often the intangible elements are impossible to understand simply from seeing the game in motion--only the written or verbal communication of a person can adequately capture these details.'

Comments

um

[mastershake_phd]

Yes, yes they do. If they didnt people wouldnt read them.

Re:um

[Bios_Hakr]

Well, it depends. I really started noticing crap when I saw an IT something magazine on my boss' desk. I picked it up and looked at it. There was a "review" of some kit we had worked with in the past. The kit was crap but the review was glowing. I stopped reading IT mags after that.

Later, I was reading a RC modeling magazine. I love RC helis. They had a "review" of a a heli that I knew was okay, but not great. They were comparing it to helis that were capable of doing any maneuver that the pilot could throw at it. I stopped reading RC mags after that.

Video game mags are probably the worst. I read PC Gamer for the commentary and previews, but I only read the negative parts of the reviews.

When it comes to specialized gear like an RC heli or a new router, I rely on comments in online forums. I'll jump into IRC and ask people about the bad points of the gear. I'll call the company and speak with engineers or tech support; speaking with sales is a waste. If I have to deal with sales, I ask for written documentation of tests displaying any functionality he claims. If they can't produce a document showing increased throughput, I ignore that point.

When it comes to daily items, I check boards and really read the negative Amazon reviews. I'll google $item + shit or $item + "head to head". I'll check Consumer Reports or check BBB for the company name.

If I don't find a negative opinion about an item, then I can be pretty sure it's untested or the company censors opinions. Either way, it's not worth my money. I read the negative reviews carefully. If the negative is whining, I ignore it. If the negative is a valid complaint, then I call tech support and pretend I have the item and have that same problem. How they answer my questions will determine my purchase.

Finally, just asking questions of my peers can give a lot of insight. I have *very* close contact with peers that work for competitors. It's a fairly small community and we tend to stick together. We usually share knowledge about our mistakes. If someone mentions over a beer that they are thinking about buying Wizbang 2008, then the rest of the group spills every bad thing they have ever heard about it. If the guy comes out of the discussion by answering our points, then we all think about giving it a closer look.

Re:um

[Das Modell]

I know one instance were user reviews turned out to be spot on while professional reviewers were, frankly, full of shit and probably on the publisher's payroll. I am referring, of course, to Neverwinter Nights 2. It has a Metacritic rating of 82, while the user reviews were almost entirely negative. The official forums had two sticky threads, one for negative reviews and one for positive reviews. The negative thread was restarted about four times before the first positive one was even full (the threads were replaced with new ones after a certain amount of pages). The forums were overflowing with complaints. I ended up buying the game anyway, againts my better judgement.

Not to me

[Xest]

Because I grew up, got a job and am now gullable enough to buy any crap the games industry throws out. I suffer from obsessive compulsive computer game purchase.

Oh how I miss the days of being dependant on pocket money where every penny had to be spent so wisely.

Re:Not to me

[MichaelSmith]

Oh how I miss the days of being dependant on pocket money where every penny had to be spent so wisely.

You should get married then.

Depends

[El_Muerte_TDS]

On who (the actual person) wrote it, and how well you know the reviewer. Personal preference is always a big factor in game/movie/music reviews. It could very well be that I like a game what a reviewer gave a bad review, but I would only know that if I knew the reviewer's preference.
Ofcourse demo'ing the game is always better than reading a review.
The most useless part of a review is the grade, it says absolutely nothing, except what number the reviewer assigned. They might as well use colors for grading instead of numbers or stars. So... I rate the linked article: purple.

Re:Depends

[grumbel]

### The most useless part of a review is the grade, it says absolutely nothing,

I disagree, ratings give you a simple value to compare different games of similar genres. Sure, it doesn't make sense to compare a The Sims rating with a GTA one, since the games are just vastly different, but comparing GTA vs Crackdown is perfectly doable. Ratings also give you a very direct way to see what the reviewer thought of a game, when the review text just mentioned that the graphics are "good", how good is that "good", is that a RE4 "good" or just an average "good"? A 10/10 in graphics on the other side easily tells me that its among the best to expect on a console.

Beside pages from this, ratings are important for sites like Metacritic which would be rather hard to use without a final rating. When I want to get an impression from a game I search for the reviews that gave it the highest score and those reviews that gave it the lowest, thus I get a good overview of how somebody who likes the game views it vs somebody who doesn't like it. If there would just be text things would get rather hard to find the right reviews.

All that said, rating numbers are of course heavily flawed, many reviewers rate almost every game in a 70-90/100 area and don't make much use of the rest of the scale, another issues is that ratings are often tinted by non-game related issues, like price, if it is a port of an old game and such, which however might not matter at all for having fun with the game, especially since price can lower over time and as long as I don't already know the game it is still 'new' for me. Ratings are also a one dimensional scale, while you really might one a multidimensional, i.e. there are many games out there that are great by concept, but also heavily flawed in implementation, those however just end up in the 70-80% region, which tells little about there flaws or great concept, but just tosses them together with all those games that have an uninteresting concept but flawlessly implemented.

With all the flaws I however still consider ratings far more useful then harmful.

They do

[ShakaUVM]

Reviews let me know if a game totally sucks. Then I avoid it.

But positive reviews are no guarantee of a good game, as the glowing ratings for such moribund stinkers as FFVII and FFX can attest to.

Absolutely they do

[tttonyyy]

...but maybe not as individual reviews.

http://www.metacritic.com/ is a fantastic site which does weighted averaging of scores from many reviews. I use it for games in particular - it's useful to check the reviews that give a high score against the reviews that give a low score to see what is good and what is not about a game before buying. The "averaged" score almost always corresponds with my experience of the games too, so the system seems to work.

So reviews do serve a purpose, but, as with many things in life, to get a balanced opinion you need to sample from a set great than 1.

Reviews serve to limit need to gather info

[Protonk]

Reviewers are just like political parties, insofar as they help distill a vast amount of information in order to allow us to actually make some decisions. The point of political parties isn't to provide perfect information to the voter, the point is to allow the voter to reduce the complexities of the ballot when necessary.

Before you guys get out of hand in the comments, by that I mean that it is functionally impossible for us as voters (in any country) to vet EVERYONE, from the county clerk to the State Senator (Okay, sorry, I don't have a region agnostic example, deal). We may decide on the president based on our input from non-party sources, but the other 18 names on the ballot don't rise to the same threshold. Parties allow us to make an assumption that a representative will align to the basic ideas that we are interested in.

Reviewers serve the same function. I may decide that I 'trust' a particular game or movie reviewer. As a result, I can presume that his/her views on a game are a good proxy for my own. This allows me to narrow the field of games I might be interested in without covering every demo, every press release, and every industry whisper--not to mention playing the game. In this sense, reviewers are even more necessary, because in order for me to make an adequate decision about a game in the absence of press, I would have to play it (or a demo, but even that isn't perfectly enlightening, see Lost Planet). That, of course, would obviate the need for the review.

In this case, just like political parties, we learn to accept bias in our reviewers. In most cases, the biases are benign--we share them. We like that Rogert Ebert doesn't like M. Night Shyamalan, because we don't (obviously only speaking for some of us). We like that the Onion (and pretty much everyone else) hates Uwe Boll, because we hate him. The same thing with the Democratic and Republican (insert Labour/conservative, etc) parties. We accept their biases (when we do) because we share them to some extent.

The case of bias in favor of a game publisher is a little more insidious, and is something that the game press will have to work out, and I suspect that it won't work itself out by eliminating the review. I suspect that certain reviewers (Ars, to name one) will gain greater acceptance as the rest of them keep shilling for bullshit. The same thing happened to the Democrats in the South. The south changed (beyond racism/segreation, which really only explain the first 10-15 years of that seismic shift), becoming more religious, individualistic and pro-business and the Democrats didn't adapt, so the south moved on to the republicans.

excellent

[gEvil (beta)]

Excellent commentary. 8/10

useful

[minus_273]

Reviews for games are like movie reviews. If you can find a reviewer(s) who have generally similar tastes, you will be able to judge a game with a fair degree of accuracy before you buy it. In addition without those rewiews, mega hit new games like God of War probably wouldnt be so big. I suspect a significant degree of its popularity came from people going to review sites and seeing the good reviews.

On the other end of the spectrum, how many more Final Fantasy fans would have bought Dirge of Cerebus had the reiwes not told us it was junk?