Do Reviews Still Serve a Purpose?
Via Voodoo Extreme, a post on the Sony-sponsored ThreeSpeech blog asking if game reviews are a thing of the past. Post author 'Azz Hassan' opines that the proliferation of blogs and easy access to game trailers has made the 'biased views' of reviewers a thing of the past. Responding via the Ars Technica Opposable Thumbs blog, Frank Caron offers a rebuttal to the piece. 'The argument presented in the article seems to come with the very slant that it so viciously protests: one of a negative view towards a medium that the writer feels is inadequate. Yes, there is a ton of available media on the net that can help you get a look at a game as it develops, but the problem with videos and pictures is that often the intangible elements are impossible to understand simply from seeing the game in motion--only the written or verbal communication of a person can adequately capture these details.'
Well, it depends. I really started noticing crap when I saw an IT something magazine on my boss' desk. I picked it up and looked at it. There was a "review" of some kit we had worked with in the past. The kit was crap but the review was glowing. I stopped reading IT mags after that.
Later, I was reading a RC modeling magazine. I love RC helis. They had a "review" of a a heli that I knew was okay, but not great. They were comparing it to helis that were capable of doing any maneuver that the pilot could throw at it. I stopped reading RC mags after that.
Video game mags are probably the worst. I read PC Gamer for the commentary and previews, but I only read the negative parts of the reviews.
When it comes to specialized gear like an RC heli or a new router, I rely on comments in online forums. I'll jump into IRC and ask people about the bad points of the gear. I'll call the company and speak with engineers or tech support; speaking with sales is a waste. If I have to deal with sales, I ask for written documentation of tests displaying any functionality he claims. If they can't produce a document showing increased throughput, I ignore that point.
When it comes to daily items, I check boards and really read the negative Amazon reviews. I'll google $item + shit or $item + "head to head". I'll check Consumer Reports or check BBB for the company name.
If I don't find a negative opinion about an item, then I can be pretty sure it's untested or the company censors opinions. Either way, it's not worth my money. I read the negative reviews carefully. If the negative is whining, I ignore it. If the negative is a valid complaint, then I call tech support and pretend I have the item and have that same problem. How they answer my questions will determine my purchase.
Finally, just asking questions of my peers can give a lot of insight. I have *very* close contact with peers that work for competitors. It's a fairly small community and we tend to stick together. We usually share knowledge about our mistakes. If someone mentions over a beer that they are thinking about buying Wizbang 2008, then the rest of the group spills every bad thing they have ever heard about it. If the guy comes out of the discussion by answering our points, then we all think about giving it a closer look.
I'd rather you do it wrong, than for me to have to do it at all.