Slashdot Mirror
Analyzing Game Journalism
SSDNINJA writes "Joseph Jackmovich of gamrFeed analyzes 161 articles from Kotaku, Joystiq, and Destructoid to discover how well they report gaming news. He looks to find out if the stereotypes of game journalism being poorly sourced and sexist are anecdotal or based in fact."
And this is news? No, really. When you even have guys from those review sites occasionally joking things like "we wanted to move to a zero to five star rating system, but EA demanded 95% or more rating for their latest game, so we moved to a 95% to 100% system and gave them 95%", or when you occasionally see a review totally hating everything about a game (e.g., see the old Black And White review on Firing Squad which even went the extra mile to say that you might like it if it's your first game and can't compare it to a good game, but otherwise stay off) and then give it a 87% score... tell me anyone actually is gullible to base their buying decisions on that.
Even the relatively 2000's trend of some site to pick on some 20 year old freeware game to trash and valiantly give a 5% rating, or make a list of "top 10 worst games ever" that nobody ever heard of, isn't really enough to make anyone with half a brain notice that you still don't see them giving less than 90% to anything new from a major publisher, or that they fail to mention major problems for major publishers.
Well, I suppose it's good to have it officially. Maybe it'll sink in this time. Nah, who am I kidding.
A polar bear is a cartesian bear after a coordinate transform.
..is that they never do proper critique of products that are shit. instead they hype up something that's safe to hype up.
the best game review I saw for a game was published around two decades ago(+couple of years), it was in finnish and said "p*sk* commando klooni", which translates to shit commando clone. the game was a shitty commando clone so that was just the right amount of information about it.
for example, mass effect 1&2 were praised a lot on these journos - but none of them critiqued the blatant flaws in the game, so I'm pretty sure the next iteration will just expand more on those flaws(it's just a tunnel shooter disquised as a kotor clone).
world was created 5 seconds before this post as it is.
How could you analyze that which does not exist?..
TFA strikes me as a bit of a missed opportunity. It makes some extremely valid points about the dependancy of gaming news sits on the good-will of developers and the impact that this has upon review scores (hello Kane & Lynch). Unfortunately, it drowns this in a morass of po-faced moralising about what "proper" journalism could be. Having a pop at Kotaku for mixing in silly or immature stories among the rest of the headlines is kind of missing the point; these are ultimately entertainment sites, talking about an entertainment medium while seeking to entertain their readers at the same time. A bit of sillyness is going to be par for the course.
For the most part, gaming news is not real news. That's not to say there isn't a place for it. But it isn't in the same category as the kind of news coverage we expect of politics, wars or crime and it doesn't need to be held to the same standards. Occasionally, the gaming world produces a genuine news story (anybody remember Infinium Labs?), but I've generally found that the gaming press isn't too bad at covering these when they do occur. For the rest of the time, think of it as belonging to the same category as sports or showbiz news.
The area where there is cause for concern is that surrounding developer/publisher pressure over review scores. Reviews, unlike gaming "news", fall more into the category of consumer advice than journalism and I think it's reasonable to expect appropriate standards. It really is quite obvious these days that games which come with a big name attached often seem to get review scores they don't deserve. Final Fantasy XIII is a dreadful game. You spend the first twenty five or so hours running down a straight path, fighting endless waves of identical enemies with an almost-uninteractive combat system. No matter how much the game may improve after that point (and the improvement is only mild), there's no way that 25 hours of boring on-rails gameplay shouldn't have a profound negative impact on review scores; and yet the game carried off a slew of 8/10 and 9/10 scores. Mario Galaxy 2 is not a bad game (it has some clever level design), but it has a good number of flaws, including a lack of innovation compared to its predecessor, an imprecise control system (in a game that requires a high degree of precision on many occasions) and outdated game-mechanics such as a lives-system. And yet it had ecstatic reviews, including a frankly incomprehensible 10/10 from IGN.
The thing is that it's by no means clear that publisher pressure was responsible for the scores in the two cases listed above. Square and Nintendo both tend to have a pool of rabid fans (albeit a shrinking pool in both cases) and it's quite possible that the games were just handed to fanboys for review, who were never going to hold the game to objective standards. But the fact is that there are enough incidences of genuine publisher pressure (yes, Kane and Lynch, I'm still talking about you) that gamers' suspicions are inevitably going to be aroused. I think review sites need to do more to enhance their credibility.
Some obvious steps might be:
- Removing advertising on the actual review pages (advertising elsewhere on the site - ok - but it sends the wrong message when you plaster advertising over the review itself and people will see corruption even where it doesn't necessarily exist).
- When a game review comes back with a score of 9/10 or higher (or whatever the equivalent in the site's scoring system), get a second opinion in there as part of the review.
- Greater acknowledgement of bugs and stability issues in reviews and scores. If you look at the reviews of Medal of Honour and Fallout: New Vegas, two recent games that launched in a highly buggy state (on consoles as well as PC), it's clear that some sites acknoweldge the bugs in their review while others don't, but that it's very rare to see bugs actually taken into account in scoring.
The inputs taken from each source are unven, and yet he's comparing them by percentages of the whole. If I were to make a study of 4 men and 2 women taking a test, and then complain that 66% of errors made by all of them were by men, then that would be misleading.
I haven't had an interest in video games since the great console war between the Sega Megadrive and Super Nintendo. And Sega Megazone is the one magazine I have ever truly loved. They weren't afraid to get stuck into anything. I remember* once seeing a full-page ad for the ultra-hyped "Rise of the Robots" that was just pages away from a review that gave the game less than 30% and concluded with a statement along the lines of "Once you finish this game (during an ad break on the Simpsons) you will almost certainly never play it again". That sort of thing was par for the course. Sounds like the modern-day gaming media could do with a little of that.**
*It was 15 years ago. Details may vary.
**Megazone was a horribly, horribly sexist publication. Which is something the modern gaming media could definitely do without.
press "gift pack", well written magazine/website filling with stats/data and print friendly colorful images.
http://www.qj.net/qjnet/xbox-360/dean-takahashi-halo-3-press-kit-nothing-less-than-a-bribe.html
Follow the press kits, gifts.
Watch the "transcribe often, anything positive goes, remove prior art comments" cult at work to ensure the material wealth keeps falling from the postal van.
Domestic spying is now "Benign Information Gathering"
As someone who writes reviews (of games, hardware, apps etc) I have certainly come across the issues cited here although it does seem to be more of a problem with American writers/publishers Even back in the 80s/early 90's fellow writers moaned about US publications where the editors and writers were pressured by the advertising department to up ratings as said product's developer wanted to place ads. Other countries were less prone to this and in the UK where I have most experience, the editors used to delight in telling the ad department to 'go away' if they tried that stunt. Sure, some magazines did fall for that sort of pressure and most writers knew who they were and stayed well clear for reputational reasons.
The bottom line is that 80-90% of anything you get sent to review is a 6 or 8 out of 10. Really crap stuff just doesn't get to market unless something's gone horribly wrong. In the main, stuff works well enough to fullfill its requirement in a reasonably well implemented way. Every now and then something truly bad would come along and that was wonderful, a chance to give a lower rating and hopefully some inciteful reasons as to why the product sucked. I've got a book here on the 'to be reviewed' pile right now that's going to get marked down because frankly, the title is a total lie. The content is OK but it's not what the title says it is. There is also the occassional item that is truly exception and will earn a 9 or very rarely a 10 but these are once or twice a year things.
The web doesn't seem to have changed the overall dynamic much with writers producing copy that will attract clicks rather than do the job. Many publishers have dropped the per-word basis for paying writers and moved to a per-click basis. If your article gets lots of clicks, you earn more.
I want a list of atrocities done in your name - Recoil
Since when do blogs count as journalism?
The source material is linked as a matter of course because it's being covered on a blog. That's how Slashdot does it, and that's that the entire Internet does it...except for real news sites, at least.
I visit these sites to get an entertaining take on the game industry, nothing more.
One of the main(if not the main) problem with games journalism is the fact that now everyone is rushing to be first. How many times have you seen reporters rush to break some "big story" only to have them sheepishly, if at all, admit a couple of days later that the story wasn't real or at the very least was greatly exaggerated? Games journalists are no different, everyone wants to be the first one to break a story, review a game etc to the point that they barely spend any time with the product that they are reviewing. How long does it take to complete a single player game nowadays? Usually 40 hours at least, and then if you consider online and how long it takes to really evaluate a game(balance issues, matching algorithms, stability etc) then you are looking at least a week, probably more for anyone just to slog through the thing, let alone arrange their thoughts into a coherent review.
So either a) the reviews are rushed to get something out the door as soon as possible after release or b) the journalists are given review copies but basically told that they better not give the game a bad review or else the gravy train is going to be derailed.
It's obviously a very difficult place most game journalists are in and the internet isn't making it any easier. Maybe someone should start a "slow cooked game review" site where they spend at least a week reviewing each game. Even if it means that you won't get the reviews until a week after launch.....
Monstar L
But if anyone has read any of the responses to anything Leigh Alexander has written recently, it's patently obvious sexism is (unfortunately) alive and well. She gets attacked for expressing her opinion. Now if it was Alex Leighander, then this would likely not be the case.
There is no -1 Disagree.
Disclaimer: I'm a reader of Kotaku, Destructoid and, occasionally, Joystiq. This said, I think the study should have been performed on proper gaming sites like IGN, Gamestop and GamesRadar, not those sellout ex-blogs whose main concern is now selling ads, forwarding marketing and PR emails and "going big".
This article attempts to make a assessment of the quality of game journalism, without mentioning Edge, which is one of the most well received papers by the games industry. Indeed, Kotaku (which is banned from even mainstream forums such as Neogaf due to its poor quality) and Destructoid are not aiming for quality but general coverage. To my knowledge, Edge is the only gaming publication that attempts to write reviews and games theory articles on par with what movie and music critics do.
According to the author this article on Joystiq by Justin McElroy is sexist because it contains "implied or condescending remarks about women": www.joystiq.com/2010/10/05/heres-where-cammie-dunaway-literally-went
I am not informed what passes for sexism in contemporary US higher-education circles but at least my own radar seems to be off.
Unless the intent is not to review, so much as to bitch about it. Zero punctuation does this well and anyone who hasn't experienced the wonderful bitch-moan-complain-fest that is the Zero Punctuation video review then I would highly recommend it.
It is now a cornerstone of my pre-game purchase evaluation plan:
1. Hear about game.
2. Watch Zero Punctuation.
3. If ZP gave it a bad review, but mentions that 1 small part of the game is enjoyable, GO OUT AND BUY IT BECAUSE IT WILL BE SO MUCH FUN.
Many online news sites are driven by marketing considerations. This is not news. Marketers need to be dragged into the street and shot.
Ultimately I had to stop reading the article because even though the points he makes are valid overall, his methods were shit. You don't do into something like this with a hypothesis of game journalists in bed with companies and use metrics like "is this headline misleading?" You take your hypothesis, keep it to yourself, and give the metrics to a large group of people, and aggregate their answers. And you don't compare three different sites on the same graph with wildly different totals, as he did with the graph of how many sources each site used. Putting 9/46/8 for Kotaku next to 4/44/3 for Joystiq next to 11/36 for Destructoid only tells me the author can't count, and doesn't understand graphs.
Kotaku clearly is not a site that limits itself to gaming. So their graph of off-topic articles is a bit odd if you include Kotaku in the equation.
... is a joke. The only reviews to trust are the cream of the crop user reviews.
The whole "gaming news industry" is filled with morons and kickbacks. And the pretentiousness of the game industry that it is no par with movies is ridiculous. Games and movies are fundamentally different mediums even if they do have some amount of cross over. We've seen how crappy many games have turned out when they put too much emphasis on the movie part in their games and not enough game. Final fantasy 13 I'm looking at you!
Warning: require_once(/data/r1/ioi/non_web/gf_config.php) [function.require-once]: failed to open stream: No such file or directory in /data/r1/ioi/gamrfeed.vgchartz.com/setup.php on line 2
Fatal error: require_once() [function.require]: Failed opening required '/data/r1/ioi/non_web/gf_config.php' (include_path='.:/usr/local/CustApache22T/lib/php') in /data/r1/ioi/gamrfeed.vgchartz.com/setup.php on line 2
The main issue seems to be in the ratings systems. Game reviews rate games on an absolute scale, not a relative scale. If reviewers honestly rated EA's crap games relative to EA's gems, then there would be a whole lot of sub-5 ratings. However, even a relatively crap EA game, assuming you had never played a game before in your life, is still an impressive piece of work. Consumers would have to start demanding relative scales on ratings before they would help out in selecting a game for anyone but someone who had perhaps never played a game before, in which case any major release correctly would be really impressive to that person... until they play one that really is a 10/10 on the relative scale.
stuff |
So I'm supposed to believe that a website replacing S's with Z's in their name is more of a "respectable gaming publication" than the ones they are criticizing?
These people aren't journalists FFS.
There hasn't been any really great game journalism and reviews since Scorpia's column in CGW. Since then its been adequate at best.