Ranking of Harshest, Kindest Game Reviewers
GameDailyBiz has an interesting feature up right now discussing which sites and magazines are harshest and kindest with their game reviews. Surprisingly the study shows that, overall, the multi-platform online sites aren't terribly biased, with some being somewhat nicer than the average and some somewhat harsher. Single console gaming magazines end up having the most forgiving coverage, with mainstream news sources being the harshest critics.
...some being somewhat nicer than the average and some somewhat harsher
No way! In a group of sources, some are above average and some are below? Who could have guessed?
I think in many ways it's good to limit it in this manner. There is a significant amount of error in all these ratings, and they are trying to give a quantitative score from qualititive opinions. If one site gives Halo 2 a 9.5, and Half-Life 2 a 9.6, does that mean Half-Life 2 is than Halo 2? I think the scale gamespy has is very reasonable, and it gave both games 5 stars. There is no way numbers could tell you which of these games you'll like more, but they do reflect that they are both excellent games. If you want to know which one is better and for what reasons, you are just going to have to go read the review.
I find that the vast majority of reviews are pretty kind. The assumption seems to be that 50% is only reserved for bad games rather than games of only average quality. So on a 10 point rating scale we should expect most games to bell-curve out towards the middle. Most publications, however, tend to rank poor games that most people would not consider purchasing around 50-60% which in my mind constitutes being above average. These types of ratings end up going down to single percentage points to compare the many games clustered in the upper reaches.
I believe this may have come about due to the American educational system's common method of ranking student work wherein 70% is average and anything below 50% is ignored as being poor.