Slashdot Mirror
On the Integrity of Hardware Review Sites
leathered writes "Charlie Demerjian of The Inquirer has posted an interesting article on the integrity of hardware review sites. Apparently the benefits of running such a site go far beyond advertising revenue with a fair amount of 'sweeteners' from the hardware manufacturers to say the least. All is not lost as Charlie informs us that there are a small number are flying the flag for trustworthy reviews, but the question of which sites we can trust remains." I like Daniel Rutter's (of Dan's Data) policy best.
Tom's Hardware
We (voodoo networks) are in the process of starting up a review site for backend systems for server rooms, closets, etc. The reasons that drove us to do this are simple - it's almost impossible to find objective reviews of non-consumer gear, and when we do find them they are pretty much copies of manufacturers flyers.
The odd thing so far has been the reaction (or lack thereof) from manufacturers. I've lost track of the number of PR reps I've spoken with who asked my how much a listing or review cost. Most where surprised when I told them that serverroomstuff.com won't be charging for reviews - they are objective.
We are planning on accepting advertising, but the plan for that is simple - your ad dollars won't buy you a good review if your product doesn't stand up to it's claims, period.
epinions.com remains a good across-the-board review site, not just for hardware. good detailed specs appear next to user writeups. plus you can make money by writing reviews, which i have done.
--
http://unk1911.blogspot.com
Anandtech is hardly the site I'd be holding up as an example of payola. The man has reasonable benchmarks, asks for user input on tests, and isn't afraid to say "hey, your product has no place in the market". They only gave a silver to the SN25P, even though it more or less shined in every conceivable way. Nobody buys 2nd place.
Anand's probably rolling in dough because not only is his hardware site one of the most popular, but he also has one of the most popular forums on the Internet in it. If you factor in that the ads on there are actually pretty well-targetted, I wouldn't be surprised to hear he has an excellent click-through rate.
It's probably difficult to remain objective, but that's what professionals do. Anand and his crew are obviously professionals.
-Erwos
Plausible conjecture should not be misrepresented as proof positive.
It depends upon what you are doing. Most games probably aren't written to take advantage of multiple threads, but things like renderers or image processors most definately are. The difference is that people (legitimately) running high-end software are more likely to have higher-end machines, so the programmers can aim their system requirements higher to get the job done.
Also, some kinds of tasks are just more amenable to parallelization than others. For example, if you are trying to ray trace an animation, you can usually process two frames independantly from each other. Running convolution filters on large images can also be split into convenient chunks.
In a game, you could do things like run your AI code in a separate thread, and it would run faster on a multiple cpu system. But it's harder to write. If dual-core cpus become more cheaper and more popular, you'll probably see more games written to take advantage of them. Game developers just haven't bothered yet, because the extra complexity involved with writing everything to be thread-safe isn't worth the speed improvement for just a small number of users.
my comic
viperlair did a good write up on what i like to call hardware review site payola a while back.
e.
Build Your Own PVR/HTPC news, reviews, &
"...a lot of the 'reviews' are just a "thank you for free hardware" pieces..."
;)
Hmm...that's funny. I used to work for a review site and we always had to return the hardware after we reviewed it. Now software, that's a whole other story
"A truly wise man realizes he knows nothing."
Having written a few reviews for www.overclockers.com myself, I have looked into this and found a few things, while reading other reviews.
Reviews tend to fall into a few catagories.
Bribed: The supplier has give something to the review as incentive for a good review. This is the supplier doing the dirty deed, the reviewer just goes along with it because they are profiting.
Greedy: The reviewer hands out praise and awards like candy, to keep the goods flowing. The reviewer is just keeps pumping out favourable reviews, to keep getting products for the review to play with or sell. This is the reviewer doing the dirty deed and the supplier just goes along with it, because its good PR.
Incompetance: Some reviewers just suck. They simply dont know how to get the numbers right, or their testing done properly. I just call them idiots.
Lastly and most rare, is the competant, unbiased reviewer. They know what they are doing and dont pull any BS.
I find www.overclockers.com and www.procooling.com to be good, www.xtremeresources.com and www.anandtech.com are also pretty good.
HardOCP is ok.
www.madshrimps.de and Toms Hardware both suck.
Reminds me of the time, many years ago, when I read a "shootout" style review of several network cards. Throughout the review, two things became clear: they really liked the Linksys offering (please no fanboy flames here; I'm not giving an opinion of the cards, just relating what was written) and didn't care too much for the card from 3Com.
Imagine my surprise when the 3Com card was declared the winner of the shootout in the conclusion. I reread the review, and yep, it still sounded the same; didn't like the 3Com, liked the Linksys. I pondered this for a few seconds, but all became clear when I noticed the 3Com banner ad at the top of the page.
The fact is, people are stupid and greedy. They always have been, they always will be. Keep it in mind whenever you read anything written for public consumption and you will find you get disappointed (and misled) far less often.
I think it was well assumed that games would show no advantage on standard benchmarks since they don't spawn multiple threads, and every site that ran conventional benchmarks showed that on Monday. Today, however, Anand has those results, and then also tests games with thinks like Norton Antivirus and MS Antispyware running in the background, which many of us do have running when we game. On those, the dual core showed better performance.
Anand is the one site I always trust, it's too bad the article didn't have the guts to come out and name who they thought was being bought.
I can't give you any examples, as I haven't gone there in a couple of years, but I found his site to be one of the worse ones. Toms too.
If you want accuracy in your information, best is to check out a support site for the product that isn't run by the manufacturer. amdmb.com for example I've found to be great for finding out about unstable hardware. I've seen critical posts disappear in manufacturer run forums before, so I wouldn't assume that a lack of people with problems there means there aren't any.
-1 Uncomfortable Truth
They weight reliability and depreciation quite high in their rankings, and the reality is that the guy who buys a Toyota/Honda will, on average, have far fewer problems and will recoup a lot of his initial expense than the guy with a Ford for instance.
In any case their top ranked small car for a while has been the Ford Focus. Is it possible that you were a little jaded that they didn't pick your pet car/minivan as the best?
Why would they sell harware they use daily?
They don't do comprehensive motherboard or processor reviews like other sites. You're not going to find a table with 15 different combinations of hardware compared. Usually it's just one product.
Please post your good/bad/ugly (badly written but probably not corrupt) website opinions here.
I have been following hardware reviews since about 1995, have written a few myself, and am fairly knowledgeable on the inner-workings of hardware and of English, so I have become fairly critical of reviews. Here's my brief list:
The good (Quality, non-biased, well-written reviews with reproduceable results and non-marketfluff technical explanations):
Computer Science is no more about computers than astronomy is about telescopes. --E. W. Dijkstra