Slashdot Mirror
Analyzing Game Journalism
SSDNINJA writes "Joseph Jackmovich of gamrFeed analyzes 161 articles from Kotaku, Joystiq, and Destructoid to discover how well they report gaming news. He looks to find out if the stereotypes of game journalism being poorly sourced and sexist are anecdotal or based in fact."
And this is news? No, really. When you even have guys from those review sites occasionally joking things like "we wanted to move to a zero to five star rating system, but EA demanded 95% or more rating for their latest game, so we moved to a 95% to 100% system and gave them 95%", or when you occasionally see a review totally hating everything about a game (e.g., see the old Black And White review on Firing Squad which even went the extra mile to say that you might like it if it's your first game and can't compare it to a good game, but otherwise stay off) and then give it a 87% score... tell me anyone actually is gullible to base their buying decisions on that.
Even the relatively 2000's trend of some site to pick on some 20 year old freeware game to trash and valiantly give a 5% rating, or make a list of "top 10 worst games ever" that nobody ever heard of, isn't really enough to make anyone with half a brain notice that you still don't see them giving less than 90% to anything new from a major publisher, or that they fail to mention major problems for major publishers.
Well, I suppose it's good to have it officially. Maybe it'll sink in this time. Nah, who am I kidding.
A polar bear is a cartesian bear after a coordinate transform.
TFA strikes me as a bit of a missed opportunity. It makes some extremely valid points about the dependancy of gaming news sits on the good-will of developers and the impact that this has upon review scores (hello Kane & Lynch). Unfortunately, it drowns this in a morass of po-faced moralising about what "proper" journalism could be. Having a pop at Kotaku for mixing in silly or immature stories among the rest of the headlines is kind of missing the point; these are ultimately entertainment sites, talking about an entertainment medium while seeking to entertain their readers at the same time. A bit of sillyness is going to be par for the course.
For the most part, gaming news is not real news. That's not to say there isn't a place for it. But it isn't in the same category as the kind of news coverage we expect of politics, wars or crime and it doesn't need to be held to the same standards. Occasionally, the gaming world produces a genuine news story (anybody remember Infinium Labs?), but I've generally found that the gaming press isn't too bad at covering these when they do occur. For the rest of the time, think of it as belonging to the same category as sports or showbiz news.
The area where there is cause for concern is that surrounding developer/publisher pressure over review scores. Reviews, unlike gaming "news", fall more into the category of consumer advice than journalism and I think it's reasonable to expect appropriate standards. It really is quite obvious these days that games which come with a big name attached often seem to get review scores they don't deserve. Final Fantasy XIII is a dreadful game. You spend the first twenty five or so hours running down a straight path, fighting endless waves of identical enemies with an almost-uninteractive combat system. No matter how much the game may improve after that point (and the improvement is only mild), there's no way that 25 hours of boring on-rails gameplay shouldn't have a profound negative impact on review scores; and yet the game carried off a slew of 8/10 and 9/10 scores. Mario Galaxy 2 is not a bad game (it has some clever level design), but it has a good number of flaws, including a lack of innovation compared to its predecessor, an imprecise control system (in a game that requires a high degree of precision on many occasions) and outdated game-mechanics such as a lives-system. And yet it had ecstatic reviews, including a frankly incomprehensible 10/10 from IGN.
The thing is that it's by no means clear that publisher pressure was responsible for the scores in the two cases listed above. Square and Nintendo both tend to have a pool of rabid fans (albeit a shrinking pool in both cases) and it's quite possible that the games were just handed to fanboys for review, who were never going to hold the game to objective standards. But the fact is that there are enough incidences of genuine publisher pressure (yes, Kane and Lynch, I'm still talking about you) that gamers' suspicions are inevitably going to be aroused. I think review sites need to do more to enhance their credibility.
Some obvious steps might be:
- Removing advertising on the actual review pages (advertising elsewhere on the site - ok - but it sends the wrong message when you plaster advertising over the review itself and people will see corruption even where it doesn't necessarily exist).
- When a game review comes back with a score of 9/10 or higher (or whatever the equivalent in the site's scoring system), get a second opinion in there as part of the review.
- Greater acknowledgement of bugs and stability issues in reviews and scores. If you look at the reviews of Medal of Honour and Fallout: New Vegas, two recent games that launched in a highly buggy state (on consoles as well as PC), it's clear that some sites acknoweldge the bugs in their review while others don't, but that it's very rare to see bugs actually taken into account in scoring.
As someone who writes reviews (of games, hardware, apps etc) I have certainly come across the issues cited here although it does seem to be more of a problem with American writers/publishers Even back in the 80s/early 90's fellow writers moaned about US publications where the editors and writers were pressured by the advertising department to up ratings as said product's developer wanted to place ads. Other countries were less prone to this and in the UK where I have most experience, the editors used to delight in telling the ad department to 'go away' if they tried that stunt. Sure, some magazines did fall for that sort of pressure and most writers knew who they were and stayed well clear for reputational reasons.
The bottom line is that 80-90% of anything you get sent to review is a 6 or 8 out of 10. Really crap stuff just doesn't get to market unless something's gone horribly wrong. In the main, stuff works well enough to fullfill its requirement in a reasonably well implemented way. Every now and then something truly bad would come along and that was wonderful, a chance to give a lower rating and hopefully some inciteful reasons as to why the product sucked. I've got a book here on the 'to be reviewed' pile right now that's going to get marked down because frankly, the title is a total lie. The content is OK but it's not what the title says it is. There is also the occassional item that is truly exception and will earn a 9 or very rarely a 10 but these are once or twice a year things.
The web doesn't seem to have changed the overall dynamic much with writers producing copy that will attract clicks rather than do the job. Many publishers have dropped the per-word basis for paying writers and moved to a per-click basis. If your article gets lots of clicks, you earn more.
I want a list of atrocities done in your name - Recoil
One of the main(if not the main) problem with games journalism is the fact that now everyone is rushing to be first. How many times have you seen reporters rush to break some "big story" only to have them sheepishly, if at all, admit a couple of days later that the story wasn't real or at the very least was greatly exaggerated? Games journalists are no different, everyone wants to be the first one to break a story, review a game etc to the point that they barely spend any time with the product that they are reviewing. How long does it take to complete a single player game nowadays? Usually 40 hours at least, and then if you consider online and how long it takes to really evaluate a game(balance issues, matching algorithms, stability etc) then you are looking at least a week, probably more for anyone just to slog through the thing, let alone arrange their thoughts into a coherent review.
So either a) the reviews are rushed to get something out the door as soon as possible after release or b) the journalists are given review copies but basically told that they better not give the game a bad review or else the gravy train is going to be derailed.
It's obviously a very difficult place most game journalists are in and the internet isn't making it any easier. Maybe someone should start a "slow cooked game review" site where they spend at least a week reviewing each game. Even if it means that you won't get the reviews until a week after launch.....
Monstar L
Disclaimer: I'm a reader of Kotaku, Destructoid and, occasionally, Joystiq. This said, I think the study should have been performed on proper gaming sites like IGN, Gamestop and GamesRadar, not those sellout ex-blogs whose main concern is now selling ads, forwarding marketing and PR emails and "going big".
This article attempts to make a assessment of the quality of game journalism, without mentioning Edge, which is one of the most well received papers by the games industry. Indeed, Kotaku (which is banned from even mainstream forums such as Neogaf due to its poor quality) and Destructoid are not aiming for quality but general coverage. To my knowledge, Edge is the only gaming publication that attempts to write reviews and games theory articles on par with what movie and music critics do.
Sadly the only time I see "this game sucks!" is when it is either A)- A game by an unknown publisher, or B)- A game by a publisher that hasn't got the money to bribe them anymore with swag. The last really sucky review I remember seeing was for Blacksite: Area 51 which I actually own (hey it was $2, and was worth maybe $3) which was put out by Midway right before they went tits up. But if it is a triple AAA title you may as well give it up, as they will ALL trip over themselves to tell you how wonderful it is. I don''t think I saw a single reviewer even point out how they dumbed down things like hacking in Bioshock II to make it more of a console game.
But I don't know how much of this we can blame on game reviewers, when the whole idea of "journalism" seems to be but a distant memory anymore. Hell just look at how many news groups (I'm looking at you CNN) basically were ALL FOR the government hiding as much as possible and were all calling for Assange's head? Pretty much ALL media anymore is "Here take this press release and tell us how wonderful we are bitch" and that is what you get. At least the game reviewers are honest whores and don't pretend (for the most part) to be anything more than entertainment. I don't see the MSM bragging in articles "Hey look at how much cool swag we got for sucking up!!!" which I have seen quite a few times on game sites.
ACs don't waste your time replying, your posts are never seen by me.
It says something that the closest thing to an "honest" review is butting right up against being a new application of Poe's Law.
I also think there's room for concern when you see things like the metacritic page for Farcry2 where the "critic" rating is overwhelmingly positive with 26 positive reviews, 8 mixed, and none negative... while the user reviews are 275 positive to 271 negative with 105 mixed.
It's the most blatantly lopsided set of reviews I've seen so far.
A bullet may have your name on it but splash damage is addressed "To whom it may concern."
Many online news sites are driven by marketing considerations. This is not news. Marketers need to be dragged into the street and shot.
Ultimately I had to stop reading the article because even though the points he makes are valid overall, his methods were shit. You don't do into something like this with a hypothesis of game journalists in bed with companies and use metrics like "is this headline misleading?" You take your hypothesis, keep it to yourself, and give the metrics to a large group of people, and aggregate their answers. And you don't compare three different sites on the same graph with wildly different totals, as he did with the graph of how many sources each site used. Putting 9/46/8 for Kotaku next to 4/44/3 for Joystiq next to 11/36 for Destructoid only tells me the author can't count, and doesn't understand graphs.
..is that they never do proper critique of products that are shit. instead they hype up something that's safe to hype up.
This boils down to one, major problem: most game reviewers review games like they're critics, rather than gamers. That's one big reason why I love Ben Kuchera's reviews on Ars Technica. He's had some public confrontations before in the comments and on his twitter, but his reviews almost always read like they were written by a gamer, not a journalist.
Living With a Nerd