Defying Review Aggregation

Logiksan writes "With the growing number of review aggregation sites like GameRankings and RottenTomatoes, it's becoming increasingly harder for individual game critics to be heard. GameDAILY Biz took a stab at the issue at came up with 5 aggregation-defiant tactics designed to help make reviews relevant again. Among their list of ideas is to destroy the typical review grading curve. The article states, 'If, for instance, a publication could establish a 10 point scale in which reviews were based upon purchase value and average games scored only a 3 or a 4, the higher scores would certainly become far more important. The lower scores would give the publication instant credibility as 'discerning gamers' and would free up the top scores (5-10) to show a more full range of differentiation for the top-tier titles gamers care about most.'"

Comments

Aggregation not the problem

[yotto]

I don't read game reviews any more, but not because of access to aggregation sites. I don't read them any more because they are generally paid ads for the game or fanboys who just go on and on about how great it is that Joanna Dark's boobs are made from so many polygons.

From a former zine game editor....

[MBraynard]

A lot of the smaller guys have trouble being heard - or ever heard of. The aggregator includes your zine's reviews with all the big boys. GameRankings is a big opportunity for free advertising.

If you do a good job, readers may check out your review for a particular game (some people read all reviews) and decide to go directly to you from then on.

Gamerankings is also great because they allow their readers to also rate the reviews themselves. So if you are a good writer, your reviews will stand above the mediocre ones when people look for reviews on a particular game.

Maybe Gamedaily got tired of seeing so many of it's reviews receive one star or less?

If I was really interested in starting a new zine from scratch, I would LOVE to get my reviews into the aggregator, try to accumulate high rankings for my reviews, and the traffic would increase and so would the willingness of devs to send me demo copies and put my quotes on their game boxes and I would also be able to get more advert revenue. It's a win-win-win all around.

Is this a problem?

[BertieBaggio]

Now, I used to read reviews in magazines. I even had a subscription to Edge for a while, and used to like the longer reviews (the pieces on San Andreas from there and other mags come to mind). But now I tend to rely more on peer reviews - and by that I mean people I actually know. Combine that with selectively trying demos and I reckon I have a system that does pretty well (for me anyway).

Of course, if I see a game discussed in depth on the world wide webby and with a great community, I will more than likely give it a shot. Notable games from this category are x/netrek; nethack; and HardWar, to name but 3.

So who is suffering? Well, [formerly?] well-paid magazine or ezine professional game reviewers. And yes, that is a shame to some extent. I do enjoy reading the long reviews -- I appreciate when someone puts an effort in -- but I would rather spend the subscription money elsewhere, say on a new graphics card to take advantage of these great games I should be reading reviews of.

So how do you recapture my (and everyone else's) attention? From TFA:

Ditch The Scores - Makes the review stand on the merit of its content. Good idea - as long as the content is good.

Focus on MegaReviews - Yup. El Goog prvoed that more is less; so write less reviews, but make them longer.

Trumpet Your Own Credibility - Personality can be interesting, but a fine line lies between 'trumpeting credibility' and 'arrogant gits that I won't be rading again'.

Aggregate Reviews on Your Own - Can't beat them, so join them? Might work.

Crunch The Curve - If people are looking for a quick point score, they are looking for something that conforms to their expectations. Giving lower scores will ultimately damage credibility and turn off readers, even if your intentions are noble. Here's a better idea - lose the scores altogether. Give people a well-written indepth review. The ones that are looking for a point score won't read if you give it 4/10 as opposed to 7/10 anyway.

my .02

Finally!

[Tom]

Been about time. When I stopped regularily reading gaming mags 5 or so years ago, 70% (or 7/10 or whatever the scale was) was a pretty mediocre result, and it's only got worse. The same magazines that once celebrated the first ever game to score over 90%, ever - are now giving out 90% every month or so.

It's all marketing pressure. 100% does not mean "perfect" anymore. Problems that would've dumped the score 5, 10 points are shrugged off by many testers as "will certainly be fixed by the next patch" or "but it's still better than elsewhere".

There is one online review page that still writes very criticial and sometimes harsh reviews, where the stuff that rates 94% in your average mag (which only by coincidence has a two-page ad by the same company that month...) - well, that overhyped crap gets its 47% or whatever it's really worth once you remove the "big names" and the photoshopped screenshots.

I just wish for the life of me I could remember the URL. I lost my bookmarks once, and that was the only game review site worth having a link.

I also remember what some of those tester dudes said when flat out confronted with the fact that they only ever seem to review 70%+ games. They said "we don't want to waste precious magazine space with the mediocre games".
Sounds believeable. Except that the rations are still way overblown. I like "The Movies", for example, but it's not a 9/10 game. Civ4 - great game, but 9.7/10? If you can only imagine 3% missing to absolute perfection then you have damn little imagination. And so on, and so forth.

Really, an honest rating should either allow > 100%, or say "90% if you do everything in the best way that I can imagine. Points above that only if you found better ways to do it, and because you had a few years to work on it that's not an unrealistic expectation".