On the Integrity of Hardware Review Sites

leathered writes "Charlie Demerjian of The Inquirer has posted an interesting article on the integrity of hardware review sites. Apparently the benefits of running such a site go far beyond advertising revenue with a fair amount of 'sweeteners' from the hardware manufacturers to say the least. All is not lost as Charlie informs us that there are a small number are flying the flag for trustworthy reviews, but the question of which sites we can trust remains." I like Daniel Rutter's (of Dan's Data) policy best.

Comments

That's the truth

[Anonymous Coward]

We (voodoo networks) are in the process of starting up a review site for backend systems for server rooms, closets, etc. The reasons that drove us to do this are simple - it's almost impossible to find objective reviews of non-consumer gear, and when we do find them they are pretty much copies of manufacturers flyers.

The odd thing so far has been the reaction (or lack thereof) from manufacturers. I've lost track of the number of PR reps I've spoken with who asked my how much a listing or review cost. Most where surprised when I told them that serverroomstuff.com won't be charging for reviews - they are objective.

We are planning on accepting advertising, but the plan for that is simple - your ad dollars won't buy you a good review if your product doesn't stand up to it's claims, period.

epinions

[unk1911]

epinions.com remains a good across-the-board review site, not just for hardware. good detailed specs appear next to user writeups. plus you can make money by writing reviews, which i have done.

--
http://unk1911.blogspot.com

Re:Two Words

[Anonymous Coward]

Two more. Bull shit. As someone who used to work as a technical contact within a hardware marketing team, I can personally guarantee that Tom's Hardware demanded sweeteners, more than any other review site we worked with. And they were arrogant jackasses to boot.

Re:Im not sure if he understands multithreading

[krf]

It depends upon what you are doing. Most games probably aren't written to take advantage of multiple threads, but things like renderers or image processors most definately are. The difference is that people (legitimately) running high-end software are more likely to have higher-end machines, so the programmers can aim their system requirements higher to get the job done.

Also, some kinds of tasks are just more amenable to parallelization than others. For example, if you are trying to ray trace an animation, you can usually process two frames independantly from each other. Running convolution filters on large images can also be split into convenient chunks.

In a game, you could do things like run your AI code in a separate thread, and it would run faster on a multiple cpu system. But it's harder to write. If dual-core cpus become more cheaper and more popular, you'll probably see more games written to take advantage of them. Game developers just haven't bothered yet, because the extra complexity involved with writing everything to be thread-safe isn't worth the speed improvement for just a small number of users.

viperlair did a good writeup

[enrico_suave]

viperlair did a good write up on what i like to call hardware review site payola a while back.

e.

Bribed, greedy and/or imcopentent

[9mm Censor]

Having written a few reviews for www.overclockers.com myself, I have looked into this and found a few things, while reading other reviews.

Reviews tend to fall into a few catagories.

Bribed: The supplier has give something to the review as incentive for a good review. This is the supplier doing the dirty deed, the reviewer just goes along with it because they are profiting.

Greedy: The reviewer hands out praise and awards like candy, to keep the goods flowing. The reviewer is just keeps pumping out favourable reviews, to keep getting products for the review to play with or sell. This is the reviewer doing the dirty deed and the supplier just goes along with it, because its good PR.

Incompetance: Some reviewers just suck. They simply dont know how to get the numbers right, or their testing done properly. I just call them idiots.

Lastly and most rare, is the competant, unbiased reviewer. They know what they are doing and dont pull any BS.

I find www.overclockers.com and www.procooling.com to be good, www.xtremeresources.com and www.anandtech.com are also pretty good.

HardOCP is ok.

www.madshrimps.de and Toms Hardware both suck.

Re:sponsorship

[cheinonen]

First, I recall him getting a BMW convertable back then, not a Porsche. Second, that article definately takes a stab at Anand when it talks about how one site came up with a way to benchmark when people are running multiple apps at once (which Anand did), but then attacks them for not benching games. Had the person read Anand's blog on the site to keep up with what he was working on, he could have seen that Anand lacked the time to get all the benchmarks done over the weekend, did the ones he thought most important, and the gaming results went up yesterday or today.

I think it was well assumed that games would show no advantage on standard benchmarks since they don't spawn multiple threads, and every site that ran conventional benchmarks showed that on Monday. Today, however, Anand has those results, and then also tests games with thinks like Norton Antivirus and MS Antispyware running in the background, which many of us do have running when we game. On those, the dual core showed better performance.

Anand is the one site I always trust, it's too bad the article didn't have the guts to come out and name who they thought was being bought.

The good, the bad, and the ugly

[Sivar]

Slashdotters are far more likely than the average joe to recognize a good review from a bad one.

Please post your good/bad/ugly (badly written but probably not corrupt) website opinions here.

I have been following hardware reviews since about 1995, have written a few myself, and am fairly knowledgeable on the inner-workings of hardware and of English, so I have become fairly critical of reviews. Here's my brief list:

The good (Quality, non-biased, well-written reviews with reproduceable results and non-marketfluff technical explanations):

- Ace's Hardware (on the odd occasion they review something)
- StorageReview.com (I am a volunteer moderator on the forums there, so I may be considered biased, but I doubt anyone can point out a website with better hard drive reviews)
- xbitlabs.com
- Anandtech (usually, Anand usually writes well but some of the staff writes rather awkwardly, and some of their reviews, like that of the Addtronics W8500 case (and most of their other case reviews) are staggeringly awful)
- Ars Technica, though I wouldn't classify them as a hardware review site.
- Maybe HardOCP. I don't read them much, but the odd occasion I have read them didn't set off any BS detectors. Perhaps a bit unprofessional.
- Maybe DansData.com. I rarely read them, but I have always been impressed with what I read.

The bad:
- tomshardware.com (These guys were my favorite review site years and years ago, but their quality and either integrity or competence (or both) dropped staggeringly years ago. Their hard drive reviews in particular are awful beyond belief, though that may be because I am more familiar with that type of review than others).
-

The ugly:
- Just about any review site whose URL contains the word "Game." These websites tend to be full of reviewers that seem to be random average highschool kids with no real understanding of what they are reviewing, how to write a review, or how to test or rate the product they are reviewing. The word "arbitrary" comes to mind. Note that I do not put Gamespot.com in this category (even considering their owner).
- CNet. I can't tell whether the reviewers are incompetent or bribed, but the reviews (usually, but not always) are quite lame.

Any opinions on other sites?