Slashdot Mirror
Ranking of Harshest, Kindest Game Reviewers
GameDailyBiz has an interesting feature up right now discussing which sites and magazines are harshest and kindest with their game reviews. Surprisingly the study shows that, overall, the multi-platform online sites aren't terribly biased, with some being somewhat nicer than the average and some somewhat harsher. Single console gaming magazines end up having the most forgiving coverage, with mainstream news sources being the harshest critics.
...some being somewhat nicer than the average and some somewhat harsher
No way! In a group of sources, some are above average and some are below? Who could have guessed?
Where are the error bars? The graphs are meaningless without them.
Look at GameSpy. Their overall scores are out of five stars, with the possibility of half stars in the score. The overall score that a game can get is severely restricted, and as long as it's decent, it's almost guaranteed to get seven stars or higher - a score that most of us would consider to be "good."
On the other hand, look at the mainstream media. Papers like the USA Today and the Detroit Free Press grade games on a scale of one to four. This is even more restrictive...but not in a good way. A game can only get one of two positive scores - a three or a four - and it's usually got to be bloody amazing to reach the four star level. Most of the games that I see get revieved in papers tend to get scores of two or three stars. That's not terrible, but I know that most of the people I know tend to think of a 3/4 as being much worse than a 7/10. Maybe it just seems harsher. I'm not sure why that is, but observations that I've made over the years seem to back that up.
The scoring systems vary from site to site and from one kind of media to another, and that could be enough to make up the difference.
Goo goo g'joob.
i'm shocked they're so much harsher than everyone else. maybe they're not in the game companies' pockets afterall.
Single console gaming magazines end up having the most forgiving coverage
According to TFA, multiplatform magazines were the most forgiving. Single platform magazines were the second harshest, right behind mainstream media.
That said, I would like to see some more information about how Game Daily came up with these absolutely magical numbers. In particular, I'd like to see how all four of the categories of publications could score below average. Is there some secret "fansite" category that isn't shown in any of the graphs but is clandestinely throwing all of the numbers all out of whack?
Personally, I rely on word-of-mouth review way more than game site reviews or newspaper reviews. (Those are the inky paper thingums, right?) I would be far more likely to buy or not buy a game because of some off-hand chatter in an IRC chat than because of any review site.
And then there's the tried and true way: Rent it or play it at a friends house, decide for yourself if you like it, then decide if you want to buy it, or borrow it, or wait til' it's cheap and buy it, and not buy it and all. Or whatever.
Of course, with Halo 2, that could go any number of ways.
It's academic, really, since I don't have a modern console anymore.
*is run over by rotten tomatoes*
The meta-review is pedestrian, at best. 2 out of 5 stars.
I have to say that for me, the problem isn't whether or not one review site is more generous than another overall. It's more a problem of game reviewers--and players--being biased towards certain genres, tricks, or characteristics for reasons that have little to do with anything but fad, popularity, or familiarity.
As I see this latter bias being more prevalent, it's more difficult to quantify. It has to do more with praise given without justification, or to a greater extent than is justified. Inconsistencies in reviews or across reviews are also annoying--e.g., something that is lambasted in one game will be condoned in another game, or some flaw will be briefly mentioned but then seemingly forgotten later on.
Recent reviews of Halo 2 and Half-Life 2, in my mind, are examples of such things. I understand that both are good games. But the effusive praise given to both seem unjustified--they are good, even great games, but not genre-busting or wildly innovative. It is especially upsetting to me that reviewers will state these things at the beginning of a review as caveats, then quickly dismiss them (see, e.g., the recent Slashdot review of HL2) in a way they would not for other games. Another characteristic of these reviews is that recent, or even old, developments are quickly glossed over to make a game seem more innovative than it is. Witness, for example, praise of physics engines in HL2, without comparisons to Farcry; I'm not saying Farcry was a better game, or even equal to it, only that HL2 does not radically innovate in that arena.
I supsect that in the case of Halo 2 and HL2, some of this is accounted for by the current popular taste for FPS. It's not new though--I remember, for example, when reviewers revised their reviews of Deus Ex because they initially didn't "get" the cross-genre characteristics of the game, because it wasn't familiar to them. Similar things, though, have occured in other genres, such as in turn-based versus real time in strategy games.
All in all, it was an interesting article, but I feel that more subtle--but more powerful--forms of bias exist.
I find that the vast majority of reviews are pretty kind. The assumption seems to be that 50% is only reserved for bad games rather than games of only average quality. So on a 10 point rating scale we should expect most games to bell-curve out towards the middle. Most publications, however, tend to rank poor games that most people would not consider purchasing around 50-60% which in my mind constitutes being above average. These types of ratings end up going down to single percentage points to compare the many games clustered in the upper reaches.
I believe this may have come about due to the American educational system's common method of ranking student work wherein 70% is average and anything below 50% is ignored as being poor.
Corporate websites (Nintendo Power) are on that list, and mine isn't? Fuck that noise. Completely useless data.
schild
editor, f13.net