On the Integrity of Hardware Review Sites

leathered writes "Charlie Demerjian of The Inquirer has posted an interesting article on the integrity of hardware review sites. Apparently the benefits of running such a site go far beyond advertising revenue with a fair amount of 'sweeteners' from the hardware manufacturers to say the least. All is not lost as Charlie informs us that there are a small number are flying the flag for trustworthy reviews, but the question of which sites we can trust remains." I like Daniel Rutter's (of Dan's Data) policy best.

Comments

Re:Integrity?

[kannibal_klown]

Hardware benchmarks aren't what bother me, it's stability that matters to me.

The first PC I built from scratch was in college (my previous computers were either my parents or a small-shop one I bought). I hit the hardware sites and many claimed that a particular ASUS board was great and rock solid. I was naive and took the 3 or 4 sites word at it and bought it. I started having MAJOR problems and later found out it was the Via chipset on the board. Forums were FLOODED with the exact same complaints. I eventually had to replace the board.

Since then I don't buy hardware until I've searched forums for personal experiences with said product. I'll still look at some review sites but I now take what they say with a grain of salt.

Im not sure if he understands multithreading

[0kComputer]

On multitasking and multithreaded apps, they will shine like the sun, but how many of these are there? How many times do you encode a movie while typing a document, zipping your C drive, doing some heavy CFD work all while listening to a few MP3s?

Correct me if im wrong but isn't multithreading/multitasking pretty damn important considering all the background tasks/services that are needed just to keep an OS running?

This is not journalism

[maynard]

These are accusations of bribery and conflict(s) of interest against unspecified hardware review sites. Without naming the organization(s) and specific instances of bribery, with resulting proof, this is just irresponsible hand waving. Journalists are supposed to print facts. It's important to realize the distinction between whether bribery is taking place (possibly so), and whether this article in question backs up the assertion of bribery with documentation and quotable sources on the record. It does not. IMO: This article does NOT deserve this level of publicity, nor did it deserve publication. --M

Crux of the problem

[onyxruby]

Crux of the problem is fairly simple. If you want to get the newest shiniest hardware to review and get your review out right away you can only do so with pre-release hardware. The only way to get that is to play ball with the hardware co's. Why do you think so many sites were mum about the dual athlon mp boards until one day you had a couple dozen reviews all of a sudden? That's the day the NDA's all expired.

Without this hardware you have to buy your hardware yourself. Not only is this expensive, but by the time your review is out the ad value of your review of bleeding edge hardware is kaput. Unfortunately, these are the ones that do the most honest and best reviews. Pre-release hardware is often picked out from a large selection to make sure that the review site gets a good "sample". These reviews are also the least profitable for the review sites to do. It's a nasty catch 22.

"I take goods you send me for free & review"?

[sulli]

Consumers Union has the right policy:

We accept no advertising, and buy any products we test on the open market. We are not beholden to any commercial interest.

I'm sorry, some guy who writes reviews, even ostensibly fair ones, in exchange for free product can't stand up to this.

Article is Flamebait

[Phatboy]

I'm sorry, but he states he was angry while writing this and you can tell. It's the epitome of bad journalism - he doesn't have any proof, nor make specific allegations, and those he does make seem insubstantial. For example, he flames a site that did "a review in full, or at least in half", which is quite probably Anandtech, the article of which makes it quite clear that it is only the first part.

His only criticisms of the review are that it was an exclusive (which the article makes clear) and that it doesn't cover gaming (although it is only the first part of the review). He himself admits that gaming is not the point of these chips, so why does he feel that Anandtech should have to focus on gaming is the first part of their article? Indeed, in their second part they do cover gaming and conclude that you should buy an Athlon 64 if you mainly play single-threaded games, a fact that would be obvious to anyone who regularly reads any hardware site.

I can't claim that the hardware review sites are all without bias, but compared to mainstream news, hardware reviews are some of the hardest to bias given the ease of doing standardised, repeatable benchmarks.