On the Integrity of Hardware Review Sites

leathered writes "Charlie Demerjian of The Inquirer has posted an interesting article on the integrity of hardware review sites. Apparently the benefits of running such a site go far beyond advertising revenue with a fair amount of 'sweeteners' from the hardware manufacturers to say the least. All is not lost as Charlie informs us that there are a small number are flying the flag for trustworthy reviews, but the question of which sites we can trust remains." I like Daniel Rutter's (of Dan's Data) policy best.

Comments

Trust? On the net?

[winkydink]

I don't trust anybody. I'll read multiple reviews and, if available, end-user experiences as well before making a serious buy decision.

But, then again, how do you know I'm not just making this up? :)

Integrity?

[schild]

I'm not so worried about the integrity of hardware sites so much as software. But they fall under the same categories. Those that shill and whore themselves out the most get the most goodies and are loved by companies. Those that are hard on product and serious with reviews tend to be ignored. As in, just not taken seriously. The entire press outlet/developer relationship is as corrupt as 1930's Manhattan. Integrity is not a word that should be used anywhere near it. At least hardware sites SEEM to be giving real benchmarks.

Re:Integrity?

[kannibal_klown]

Hardware benchmarks aren't what bother me, it's stability that matters to me.

The first PC I built from scratch was in college (my previous computers were either my parents or a small-shop one I bought). I hit the hardware sites and many claimed that a particular ASUS board was great and rock solid. I was naive and took the 3 or 4 sites word at it and bought it. I started having MAJOR problems and later found out it was the Via chipset on the board. Forums were FLOODED with the exact same complaints. I eventually had to replace the board.

Since then I don't buy hardware until I've searched forums for personal experiences with said product. I'll still look at some review sites but I now take what they say with a grain of salt.

Re:Integrity?

[schild]

This is exactly the sort of thing I'm talking about. You never hear about the terribly awful in reviews when they are the things that really need to be highlighted. I'm not trying to pimp myself here - so if someone has a better example of focusing on the bad, do tell - but here's my website's review of the new Splinter Cell: Chaos Theory for the PC: Linkie. Everyone else is giving it high marks, we refused to play it because it has computer crippling, invasive techniques in it's use of Star Force copy protection. Good luck finding that on a corporate site. And this is why I hate the industry I've forced myself into.

Re:Integrity?

[ArbitraryConstant]

Take that a step further...

I won't buy a major part until I've specifically found some problems with it. Nothing is perfect, and until I feel I've got a reasonable sample of the sorts of problems a part has, I'm not comfortable buying it.

Besides, the stuff I do is generally improved by throwing gobs of RAM at the problem. That's generally easy to do no matter which parts you're buying.

sponsorship

[theMerovingian]

It seems like I remember Anand buying himself a Porsche for his 16th or 18th birthday, using the payola from his hardware review site.

While his business acumen is to be commended, I can imagine it would be difficult to remain 100% objective under such circumstances.

http://www.anandtech.com

Re:sponsorship

[Erwos]

Anandtech is hardly the site I'd be holding up as an example of payola. The man has reasonable benchmarks, asks for user input on tests, and isn't afraid to say "hey, your product has no place in the market". They only gave a silver to the SN25P, even though it more or less shined in every conceivable way. Nobody buys 2nd place.

Anand's probably rolling in dough because not only is his hardware site one of the most popular, but he also has one of the most popular forums on the Internet in it. If you factor in that the ads on there are actually pretty well-targetted, I wouldn't be surprised to hear he has an excellent click-through rate.

It's probably difficult to remain objective, but that's what professionals do. Anand and his crew are obviously professionals.

-Erwos

Re:sponsorship

[cheinonen]

First, I recall him getting a BMW convertable back then, not a Porsche. Second, that article definately takes a stab at Anand when it talks about how one site came up with a way to benchmark when people are running multiple apps at once (which Anand did), but then attacks them for not benching games. Had the person read Anand's blog on the site to keep up with what he was working on, he could have seen that Anand lacked the time to get all the benchmarks done over the weekend, did the ones he thought most important, and the gaming results went up yesterday or today.

I think it was well assumed that games would show no advantage on standard benchmarks since they don't spawn multiple threads, and every site that ran conventional benchmarks showed that on Monday. Today, however, Anand has those results, and then also tests games with thinks like Norton Antivirus and MS Antispyware running in the background, which many of us do have running when we game. On those, the dual core showed better performance.

Anand is the one site I always trust, it's too bad the article didn't have the guts to come out and name who they thought was being bought.

Re:sponsorship

[ShieldW0lf]

I can't give you any examples, as I haven't gone there in a couple of years, but I found his site to be one of the worse ones. Toms too.

If you want accuracy in your information, best is to check out a support site for the product that isn't run by the manufacturer. amdmb.com for example I've found to be great for finding out about unstable hardware. I've seen critical posts disappear in manufacturer run forums before, so I wouldn't assume that a lack of people with problems there means there aren't any.

Identification of sites he's accusing?

[philgross]

The [H]ardOCP review made clear that dual core chips were weak for gaming. Any idea which sites he's fingering?

Re:Identification of sites he's accusing?

[PaisteUser]

The [H]ardOCP review made clear that dual core chips were weak for gaming. Any idea which sites he's fingering?

Since the article he wrote was released today, I took a look at both Tom's Hardware and AnandTech. Both had "previews" of the Dual Core Intel chips...both major sites. That's just an educated guess, but seems correct.

Re:Identification of sites he's accusing?

[hattig]

The only sites I trust at the moment are The Tech Report and Aces Hardware. These sites rarely get exclusives of course! HardOCP I think is also kosher.

Now sites like Toms Hardware Guide and Anandtech I am not sure about.

At least you know you are getting a biased view at a site like AMDZone, heh.

The Inquirer paid /. to run this article.

[mcguyver]

Well, not really but wouldn't it be ironic if they did?

Re:The Inquirer paid /. to run this article.

[sonoluminescence]

Maybe I'm paranoid (sitting here in my tin foil hat) but this was just what crossed my mind.

I remember reading a comment of slashdot some time ago that the internet was "the last source of uncensored, unbiased information" and a +5 insightful reply saying "you're half right"

every site on the web is pushing some sort of agenda, whether it be for political motive, religious belief or big handfuls of cash.

Lets face it ./ its self is not exactly unbiased.

Huh?

[Quixote]

In this world, if we can't trust TrustedReviews.com , then who can we trust??

// didn't RTFA :-)
//// no interest in trustedreviews.com either, just thought the name was funny
////// sosumi

Re:Huh?

[sulli]

Trusted Computing?

My pick

[cy_a253]

I consider Scott Wasson's Tech Report to be one of the best "independent" review sites around.

Im not sure if he understands multithreading

[0kComputer]

On multitasking and multithreaded apps, they will shine like the sun, but how many of these are there? How many times do you encode a movie while typing a document, zipping your C drive, doing some heavy CFD work all while listening to a few MP3s?

Correct me if im wrong but isn't multithreading/multitasking pretty damn important considering all the background tasks/services that are needed just to keep an OS running?

Re:Im not sure if he understands multithreading

[krf]

It depends upon what you are doing. Most games probably aren't written to take advantage of multiple threads, but things like renderers or image processors most definately are. The difference is that people (legitimately) running high-end software are more likely to have higher-end machines, so the programmers can aim their system requirements higher to get the job done.

Also, some kinds of tasks are just more amenable to parallelization than others. For example, if you are trying to ray trace an animation, you can usually process two frames independantly from each other. Running convolution filters on large images can also be split into convenient chunks.

In a game, you could do things like run your AI code in a separate thread, and it would run faster on a multiple cpu system. But it's harder to write. If dual-core cpus become more cheaper and more popular, you'll probably see more games written to take advantage of them. Game developers just haven't bothered yet, because the extra complexity involved with writing everything to be thread-safe isn't worth the speed improvement for just a small number of users.

Makes Sense

[Philosinfinity]

After reading the article, I must admit that it does make sense. However, there is a self-preservation outlook that can explain the same phenomena. Hardware review sites want the latest and greatest toys as quickly as possible. If my hardware review site publishes an article that doe not look faovrably upon Company X's latest high end product, how likely is Company X to send me their next greatest product as quickly? Granted, it shouldn't happen this way, but Company X is in business to make a profit. They don't want bad PR and they want as many people to buy their most profitable items as possible. My hardware review site wants to stay in business. It wants to make a profit, and to do so, I have to act in a certain way. I have to ensure that I have a product to sell. It isn't fun and it isn't nice, but that's business.

Let's list them then folks...

[HerculesMO]

The one site I like, though the reviews are few and far between, is Ars.Technica. Only reason, is because they BUY THEIR OWN HARDWARE :)

Anybody have any sites that they feel are bad or good (with respect to this article)? Please list a few reasons too, few examples if you can -- it makes it nice to see if these points are driven home over time by reading the reviews on different sites :)

Re:Let's list them then folks...

[SuperBanana]

Only reason, is because they BUY THEIR OWN HARDWARE :)

Given that Ars Technica probably then would like to sell what they've bought, there's still conflict of interest. If you buy a $2k system and say "ugh, it's junk", you're not going to have much luck selling it, are you?

If they sell the stuff before the review is published, fine...

Re:Let's list them then folks...

[hawkear]

Why would they sell harware they use daily?

They don't do comprehensive motherboard or processor reviews like other sites. You're not going to find a table with 15 different combinations of hardware compared. Usually it's just one product.

Re:Trust? On the net?

[ZephyrXero]

Perhaps there should be a system for all of these sites to verify each other's data. For example, if more than one site benchmarks the same hardware they could combine their scores for a more accurate average? As far as the written portion, it's all opinion so you can never have a truely "valid" review there...

This is not journalism

[maynard]

These are accusations of bribery and conflict(s) of interest against unspecified hardware review sites. Without naming the organization(s) and specific instances of bribery, with resulting proof, this is just irresponsible hand waving. Journalists are supposed to print facts. It's important to realize the distinction between whether bribery is taking place (possibly so), and whether this article in question backs up the assertion of bribery with documentation and quotable sources on the record. It does not. IMO: This article does NOT deserve this level of publicity, nor did it deserve publication. --M

Re:This is not journalism

[Jurph]

I think he was pretty clear about calling it a rant. If you're going to write journalism about the web -- real investigative journalism -- you can't be embroiled in the conflict yourself, as that hardware reviewer clearly is. And even though he didn't name any names, he pretty clearly identified Anandtech and a few other sites.

Re:This is not journalism

[Lord_Dweomer]

I respectfully disagree. If they are not naming review sites, then nobody is being singled out. All the article does is serve to increase public awareness of the fact that on the internet, people may have a motive to not tell you the whole truth, or to sugar coat it.

This will encourage people to check with multiple sources, talk to people, and make product decisions for themselves, which is possibly one of the most important skills one can learn for surfing the internet.

That's the truth

[Anonymous Coward]

We (voodoo networks) are in the process of starting up a review site for backend systems for server rooms, closets, etc. The reasons that drove us to do this are simple - it's almost impossible to find objective reviews of non-consumer gear, and when we do find them they are pretty much copies of manufacturers flyers.

The odd thing so far has been the reaction (or lack thereof) from manufacturers. I've lost track of the number of PR reps I've spoken with who asked my how much a listing or review cost. Most where surprised when I told them that serverroomstuff.com won't be charging for reviews - they are objective.

We are planning on accepting advertising, but the plan for that is simple - your ad dollars won't buy you a good review if your product doesn't stand up to it's claims, period.

Re:That's the truth

[jackbird]

No, you need to follow the example of Consumer's Union (publisher of Consumer Reports) if you want to be unimpeachably objective:

Buy your own hardware at retail, using obfuscated identifying information.

Carry NO 3rd-party advertising of any kind.

Disallow manufacturers from using your name, excerpts from reviews, or any other acknowledgment that you exist in THEIR advertising.

Of course, that means your revenue model is going to have to change, either to a subscription site; privately funded, capped, and transparent donations; or through selling stuff (books?) of some kind.

Crux of the problem

[onyxruby]

Crux of the problem is fairly simple. If you want to get the newest shiniest hardware to review and get your review out right away you can only do so with pre-release hardware. The only way to get that is to play ball with the hardware co's. Why do you think so many sites were mum about the dual athlon mp boards until one day you had a couple dozen reviews all of a sudden? That's the day the NDA's all expired.

Without this hardware you have to buy your hardware yourself. Not only is this expensive, but by the time your review is out the ad value of your review of bleeding edge hardware is kaput. Unfortunately, these are the ones that do the most honest and best reviews. Pre-release hardware is often picked out from a large selection to make sure that the review site gets a good "sample". These reviews are also the least profitable for the review sites to do. It's a nasty catch 22.

epinions

[unk1911]

epinions.com remains a good across-the-board review site, not just for hardware. good detailed specs appear next to user writeups. plus you can make money by writing reviews, which i have done.

--
http://unk1911.blogspot.com

Re:epinions

[ergo98]

epinions.com remains a good across-the-board review site

You really think so? Personally I think it's absolutely terrible - apart from the "trying to make a living out of it" so-called-reviewers trying to pad up reviews with a bunch of copy/paste tripe, for serious hardware -- stereos, cameras, cars, whatever -- the reviews are overwhelmingly positive.

I attribute this to two probable causes - one is that when these reviewers actually own the product there is a natural tendency to defend what you bought (even if it turned out to be a dud). If someone saved up for months to buy their Kenwood Shittastic 5000, they're going to be damn sure in their head that it's the world's greatest stereo.

The second problem is that a lot of reviewers seem to correlate their own goals (making money) to epinions goals (making money). Epinion makes most of their money, I presume, from people getting SOLD on a product. Thus it is naturally in every reviewers best interest to shill every POS to unsuspecting visitors.

I've tried to use epinions a few times to help make purchasing decisions, however each time I found it absolutely terrible.

"I take goods you send me for free & review"?

[sulli]

Consumers Union has the right policy:

We accept no advertising, and buy any products we test on the open market. We are not beholden to any commercial interest.

I'm sorry, some guy who writes reviews, even ostensibly fair ones, in exchange for free product can't stand up to this.

Re:Trust? On the net?

[ShaniaTwain]

But, then again, how do you know I'm not just making this up?

..because you were moded up?

you have the support of your peers, I trust you. what do you want me to buy?

Not Anandtech

[tyates]

"Single threaded gaming performance is, as we mentioned in the first article, no different than the single core Pentium 4 of the same clock speed. And as we know from all of our previous comparisons, the Athlon 64 is the clear choice for single threaded gaming performance." http://www.anandtech.com/cpuchipsets/showdoc.aspx? i=2389&p=6

what if everyone's in on it?

[SuperBanana]

I'll read multiple reviews and, if available, end-user experiences as well before making a serious buy decision.

What good is reading multiple reviews if they're all crap? What good are end-user experiences posted on the net, if companies are posting fake reviews, which they are?

News flash- even if they're not getting "payola" (let's call it what it is- bribe money/gear), they're controlled quite effectively by hardware companies because everyone wants to be the first site with a review of Hot Product X to drive hits to their site to earn advertising revenue. Write something bad about a product, and that company will drop you to the bottom of the list.

Let's not forget that most of these guys litter their sites with advertisements for the very product they are reviewing, too. Bob's Extreme Hardware isn't going to be very happy if young Johnny says the PC case Bob just stocked is crap- and he's going to tell young Johhny that.

Why is any of this a surprise to any reasonably intelligent individual?

Re:what if everyone's in on it?

[badmammajamma]

This stuff is easy to detect. The most important thing to remember is that whenever a site is doing a "preview" of an unreleased product, it will ALWAYS be very optimistic and glowing. Ignore previews at all costs. (Sites that are super-honest NEVER get previews unless the product company knows without question they have a slam-dunk.) This is often how their arrangements work. They offer glowing previews, keep their advertising revenues going, and then get honest when the real review comes down the pike. The nice part is that the reviewing site gets to keep their rep intact since they can always point to a real review that is mostly honest.

In cases where reviews are bad, it usually comes a fair bit after release to give the public a chance to waste their money on it before they are notified that the thing in question is a piece of shit. It's a win-win for the product companies and the reviewers.

Re:Trust? On the net?

[badasscat]

I don't trust anybody.

I hope that includes Charlie Demerjian. This jumped out at me:

"here is the truth, if you are going to multitask and do and do anything that tasks both of the CPUs, one of those is going to be a game."

Bullshit. This drives me crazy on hardware sites, this supposition that the only reason anyone could ever want high performance in their PC is to play games.

At home, I use Photoshop, Illustrator and Flash, I run Windows Media Center for SDTV and DVD viewing, I do video encoding using various tools (Windows Media Encoder, Dr. Divx, and others). I often do all these things at the same time on the same PC, with a hacked version of Media Center that lets me log in remotely at the same time another account has the TV going.

I studiously avoid playing games on this system, because I'm asking it to do quite enough already - I've got another system that I play games on. But I would love a dual-core CPU for this thing, as it would help me out a lot.

Graphics professionals, photographers, multimedia content producers and other high-end users are, surprise surprise, a real market, and they spend even more money than gamers do. I don't see why that's so hard to grasp. I read the specific preview of Intel's dual-core CPU's that Charlie's talking about in his comment up there and I actually found it a refreshing change to find some real-world benchmarks that were not strictly based on playing Doom 3.

That said, I'm sure there is payola going on in the industry. But I worry more about the small sites that seem to give positive reviews to every single component they get sent for free than I do about sites that realize non-gamers are a legitimate group of users that require their own set of benchmarks.

Re:Trust? On the net?

[gl4ss]

well.. the scores don't mean anything. seriously, they don't.

the real problem with these hardware sites is that the writers are often clueless about the actual hardware and what it does - and make sometimes claims that are not even physically possible. trusting such guys to review something that supposedly does something is no good when they lack the knoweledge to make the decision if the product even works as advertised or not. a lot of the 'reviews' are just a "thank you for free hardware" pieces that are basically referates of the products description followed by a thumbs up icon(or whatever the particular site uses for 'editors choice' that every product they review happens to get).

Re:Trust? On the net?

[Shambhu]

For example, if more than one site benchmarks the same hardware they could combine their scores for a more accurate average?

More accurate than what? The mean? No, wait... doesn't work. I know! More accurate than the ones that aren't very accurate!

As far as the written portion, it's all opinion so you can never have a truely "valid" review there...

Words bad! Numbers good! Except some of the numbers! Thog take average, find good numbers.

Sorry, couldn't resist. ;)

Re:Two Words

[sonoluminescence]

Reading the perview of the pentium D on tom's hardware today it includes plenty of game benches and also some fair justification of why intels lastest offerings are likely to suck until software catches up.

So if you intended to say the article was talking about toms then I'd say you're wrong.

If you were saying toms is a shining light in a sea of intels bitches, I'd say not far off.

Re:Two Words

[Anonymous Coward]

Two more. Bull shit. As someone who used to work as a technical contact within a hardware marketing team, I can personally guarantee that Tom's Hardware demanded sweeteners, more than any other review site we worked with. And they were arrogant jackasses to boot.

might be true, but...

[potatoBBQ]

when was the last time you actually bought something because of great reviews... and it totally sucked? most products at their respective price points these days are fairly competitive with each other and if you do even a little bit of research, you should be fairly satisfied with your purchases. i think although many of the reviews are biased in some way or another, crappy products and good produtcs tend to seperate themselves on their own fairly quickly. if a product just straight out sucks, no review can save it.

Article is Flamebait

[Phatboy]

I'm sorry, but he states he was angry while writing this and you can tell. It's the epitome of bad journalism - he doesn't have any proof, nor make specific allegations, and those he does make seem insubstantial. For example, he flames a site that did "a review in full, or at least in half", which is quite probably Anandtech, the article of which makes it quite clear that it is only the first part.

His only criticisms of the review are that it was an exclusive (which the article makes clear) and that it doesn't cover gaming (although it is only the first part of the review). He himself admits that gaming is not the point of these chips, so why does he feel that Anandtech should have to focus on gaming is the first part of their article? Indeed, in their second part they do cover gaming and conclude that you should buy an Athlon 64 if you mainly play single-threaded games, a fact that would be obvious to anyone who regularly reads any hardware site.

I can't claim that the hardware review sites are all without bias, but compared to mainstream news, hardware reviews are some of the hardest to bias given the ease of doing standardised, repeatable benchmarks.

Reviews are mostly BS

[Anita Coney]

Most software and hardware reviews go out of their way to find SOMETHING good about a product. For example, a reviewer might complain about poor drivers but then point out that "new ones are on the way."

If I'm in doubt about something, I'll read reviews from actual people, e.g., at newegg. When someone gets screwed over a product, they aren't going to gloss over the problems. They are going to tell us as bluntly as possible.

viperlair did a good writeup

[enrico_suave]

viperlair did a good write up on what i like to call hardware review site payola a while back.

e.

Re:Trust? On the net?

[ZephyrXero]

"...a lot of the 'reviews' are just a "thank you for free hardware" pieces..."

Hmm...that's funny. I used to work for a review site and we always had to return the hardware after we reviewed it. Now software, that's a whole other story ;)

Re:Two Words

[Axe]

Could you point to a particular story? Was the doodad you wanted to troll of a particular interest to TH target audience?

Among several review sites I look through, TH did not seem particularly bad, maybe above average on the integrity of review front. And they usually pick fairly representative sample - do not remember obvious omissions or mismatches in comparisons.

Though it is not my favorite site, indeed.

Sweeteners, sure, everybody needs to eat. It is what you do with them. I would not mind demanding something to place a story, especially about some fringe product nobody cares about - I would only mind outright lie in a story, or some really poor review from a technical point of view.

Re:Two Words

[nogginthenog]

Or one word - 'Usenet'. Always good to check what problems people have been having with a particular pice of hardware.

Re:Trust? On the net?

[Desert Raven]

this supposition that the only reason anyone could ever want high performance in their PC is to play games.

Agreed.

I don't play computer games, I don't even *own* any. None of my systems have ever had a game installed on them (yes, this includes solitare). All of my systems are used for work, and non-work related research. Yet, over half of my systems are dual-cpu. I multi-task, a lot. I often have graphics filters chewing away, or CDs burning, while I'm doing something else. On my servers, I want to know that if I need to compile something, archive something or do some other processor-intensive task, that there's enough processor power left to continue with the server's normal tasks.

I read reviews, but they account for less than 20% of the weight I give to my purchasing decisions. The only reviews I will pay close attention to are the *bad* reviews. They're so rare that one has to believe the product must have been truly hideous.

Re:Trust? On the net?

[lgw]

I just think it's funny that a guy who calls attention to his game collection in his sig is whining about people assuming he only uses his computer for gaming.

Graphics professionals are indeed a market, but professionals of any stripe generally use the hardware their company procures for them for that profession (I know I do). The gaming machine is the one you build yourself from parts. The gaming bias of the review sites makes a lot of sense in that respect.

Bribed, greedy and/or imcopentent

[9mm Censor]

Having written a few reviews for www.overclockers.com myself, I have looked into this and found a few things, while reading other reviews.

Reviews tend to fall into a few catagories.

Bribed: The supplier has give something to the review as incentive for a good review. This is the supplier doing the dirty deed, the reviewer just goes along with it because they are profiting.

Greedy: The reviewer hands out praise and awards like candy, to keep the goods flowing. The reviewer is just keeps pumping out favourable reviews, to keep getting products for the review to play with or sell. This is the reviewer doing the dirty deed and the supplier just goes along with it, because its good PR.

Incompetance: Some reviewers just suck. They simply dont know how to get the numbers right, or their testing done properly. I just call them idiots.

Lastly and most rare, is the competant, unbiased reviewer. They know what they are doing and dont pull any BS.

I find www.overclockers.com and www.procooling.com to be good, www.xtremeresources.com and www.anandtech.com are also pretty good.

HardOCP is ok.

www.madshrimps.de and Toms Hardware both suck.

What I do...

[sgant]

I read a lot of review sites...which usually has a forum attached to it. I try to read as much as I can on a given product plus the forum replies plus other places on the net.

Through all the data that I read, my brain forms an opinion and I weed out the bullshit and the hyperbole to find the heart of the matter. It usually works most of the time too.

Your bullshit detector has to be in good shape and you have to know how to weed out the crap before you get down to the nitty-gritty.

But if you're naive about the net and you go online maybe once a month...then you're a raw piece of meat in a pool full of sharks.

Re:What I do...

[lee7guy]

But if you're naive about the net and you go online maybe once a month...then you're a raw piece of meat in a pool full of sharks.

I am one of those naive that really don't have a clue what hardware review sites to trust. My comfort is that I am probably far from alone, in this matter.

To assist me and other naives(sp?), please join this silly poll and review the following sites (regarding credibility) with a scale ranging from 1 to 10, where 1 is "No credibility at all" and 10 is "Perfect credibility, these guys wouldn't post a biased review for world domination":

About PC Hardware Reviews
Ace's Hardware
Anandtech
Ars Technica
Beyond 3D
Cnet Reviews
Dan's Data
Dev Hardware
Extremetech
Firingsquad
[H]ard|OCP
Hardware Analysis
Hardwarecentral
Hardwarezone
IT Reviews
OcPrices
Overclockers.com
ProCooling.com
The Tech Report
The Tech Zone
Tom's Hardware
TrustedReviews
Viperlair
Xtreme Resources

If you know only a few of them, give your opinion on those.

Maybe someone with the right facilities could set up an independent poll?

It's the meta-information that's important.

[Matey-O]

Looking for Home theatre speakers? (I was) 160 people voting 4.5 out of 5 is better than 3 people giving a perfect 5. Looking for a cellphone? (I was) 0 noka6820sucks.com type websites is better than a half-dozen MotoV600Bites.com Buying a Camera? The user opinions that sound like they've used it should count more than the ones that said 'I bought 'competing product' because this camera didn't have blue LEDs A particular revew site's integrity will stand out against the rest of the other reviews. Beware when Gamersite X is giving a game a 98% when everybody else is giving it a 45%.

Another way of finding good hardware...

[Xugumad]

...is to find a components and systems supplier you like, and see what they build their systems from. Admittadely, you're more likely to get stable components than blisteringly fast/overclockable, but that's fine for me. So far I've had very good experiences with pretty much duplicating the systems I've seen offered for sale (generally changing only a few components to versions I prefer).

Say it ain't so!!!

[halepark]

You mean the media can be bought?!? I feel like I'm taking CRAZY PILLS!!

My experience with both links

[DigitalDragon]

When I went to Dan's homepage and clicked on one of the reviews, my antivirus program has informed me that there was a malware Javascript virus that tried to be executed. That certainly means that I am not going to his website ever again.

As for the main article, I am surprised how badly it is written. Poor language use and overall structure.

Re:Trust? On the net?

[Anonymous Coward]

what do you want me to buy?

A new jumpsuit. That fish patterned thing is horrible.

You get what you pay for.

[fox9397]

Free reviews. No such thing. Someone has got to pay. If you aren't paying for a subscription to Consumer reports or other "independent, you paid for it service", than advertising, free gadgets, and first scoop Rights, will gladly fund your opinion for you.

Re:"I take goods you send me for free & review

[ergo98]

They weight reliability and depreciation quite high in their rankings, and the reality is that the guy who buys a Toyota/Honda will, on average, have far fewer problems and will recoup a lot of his initial expense than the guy with a Ford for instance.

In any case their top ranked small car for a while has been the Ford Focus. Is it possible that you were a little jaded that they didn't pick your pet car/minivan as the best?

Not any more Re:Two Words

[D3]

Sorry, Tom's started out as a hobby/enthusiast many years (7+) ago but no longer holds much integrity. Look at all the ads and their reviews are guilty of some of the shenanigans listed in the article like including obscure games to give better numbers, etc. I don't trust any of the sites anymore.

Re:Two Words

[CyberKnet]

The personal guarantee of an anonymous coward.

Priceless =)

It's more than just that...

[Omega]

It's more than just avoiding the appearance of impropriety.

If you actually BUY the product you are reviewing, you're subject to the same likelihood of manufacturing defects, poor workmanship or shipping/transporting accidents as anyone else who buys it -- so you can work that into your review to give an overall impression of what a REAL consumer will face if they buy the product.

If you're testing a product that was specially given to you by the manufacturer, they want as favorable a review as possible, so your product may have been specially checked for defects, or have extra durability built in, and it will likely arrive insured by special courier or some other express guaranteed delivery -- not representative of the average customer experience at all.

IMHO, Consumers Union (publisher of Consumer Reports) is THE authority when it comes to honest, reliable and unbiased reviews.

On the integrity of the Inquirer

[d_jedi]

It has none.

This is the same site that called a Microsoft representitive the "spokesvole".

The article complains that Anandtech (it's obvious for anyone who read the review in question) doesn't use any gaming benchmarks.

Well, take a closer look.. the article is called Intel Dual Core Performance Preview Part I: First Encounter
very conveniently, (and completely expected, as well..) Anand posted the second part today, which included the gaming benchmarks we all expect.

Way to troll Inquirer writer, way to troll..

Re:Two Words

[SavannahLion]

Tom's Hardware was the primary reason I stopped reading his site... er....

Let me clarify, A long time ago, during the BH6 heydays, I used to read Tom's Hardware almost religiously. I scarfed down every article with near fever, trying to stay on top of the latest hardware reviews. Then I got stupid and started dating, so I ignored Ol' Tom for a few years. When my other half and I broke up, I started digging into the hardware scene again, so I went back to Tom's to get up to speed.

A few articles into it, I realized the, "feel," of the articles changed. There didn't seem to be as much useful, practical, information in them anymore. They didn't have any interesting opinions on the hardware being reviewed. In fact, I don't think I found an article that was harshly negative in any way. Nor did I find any articles with useful grit-in-your-teeth information. Initially, I thought the, "neutralism," being presented in the articles were because of the close tolerances of the hardware being reviewed. I mean overclocking a 333MHz CPU to 450MHz had greater net results than overclocking the more modern CPU's now. But I put my faith in Tom, because it's supposed to be the best. After all, his site does say:

Tom's Hardware Guide readers have come to rely on the site for unbiased and authoritative articles on the products....

So who was I to argue with that?

I just can't put my finger on what, exactly, Tom is missing. As near as I can figure, Tom's Hardware reviews read too much like brochures. It's just enough to try and get a person interested. But whatever it is, it's enough to prevent me from utilizing his site for anything more than keeping track of the latest hardware. Nowadays, if I want real hard and honest opinions, I just hop on over to my favorite forum/BBS/IRC/whatever and sort through the flamefests to get a feel for a particular piece of hardwares viability.

Best reviews ever: IRC

[EvilStein]

booradley> I'd like to perform a one act play I call, "Creative screwed me like a bitch"
booradley> Buy me! I'm ever so sexy
booradley> ok. come home with me and we'll play among the stars
booradley> tee hee! I love you, boo!
booradley> I love you too, audigy
booradley> :: later ::
booradley> there, you're all installed. how do you feel?
neshura> down in front!
booradley> audigy> LET JESUS FUCK YOU! VRAAAGH!
* audience gasps.
booradley> * audigy is putting noise across your PCI channels
booradley> hard drive> Mein leben!
booradley> * hard drive has died
booradley> audigy> Blaaah! blaaaugh! your mother sucks cocks in hell! graaagh!
booradley> modem> aaieee
booradley> *modem has died
booradley> and the new modem I got connects at 32k tops
Shendal> By far, that's the best one-act IRC play I've read this season. Do I smell a Tony award?
----

That's from bash.org. It's pretty much how it goes. I ask friends that have more disposable income and they tell me about products they've blown money on. heh. It's another reason that I chose Sirius over XM for my satellite radio - finding an objective review online was a waste of time because everyone was just spewing the same marketing drivel or Google results were flooded with e-stores. I just spent some time and talked to people I knew about both products and went from there.

The Internet is becoming increasingly *useless* for information that doesn't come straight out of some marketing droid.

The good, the bad, and the ugly

[Sivar]

Slashdotters are far more likely than the average joe to recognize a good review from a bad one.

Please post your good/bad/ugly (badly written but probably not corrupt) website opinions here.

I have been following hardware reviews since about 1995, have written a few myself, and am fairly knowledgeable on the inner-workings of hardware and of English, so I have become fairly critical of reviews. Here's my brief list:

The good (Quality, non-biased, well-written reviews with reproduceable results and non-marketfluff technical explanations):

- Ace's Hardware (on the odd occasion they review something)
- StorageReview.com (I am a volunteer moderator on the forums there, so I may be considered biased, but I doubt anyone can point out a website with better hard drive reviews)
- xbitlabs.com
- Anandtech (usually, Anand usually writes well but some of the staff writes rather awkwardly, and some of their reviews, like that of the Addtronics W8500 case (and most of their other case reviews) are staggeringly awful)
- Ars Technica, though I wouldn't classify them as a hardware review site.
- Maybe HardOCP. I don't read them much, but the odd occasion I have read them didn't set off any BS detectors. Perhaps a bit unprofessional.
- Maybe DansData.com. I rarely read them, but I have always been impressed with what I read.

The bad:
- tomshardware.com (These guys were my favorite review site years and years ago, but their quality and either integrity or competence (or both) dropped staggeringly years ago. Their hard drive reviews in particular are awful beyond belief, though that may be because I am more familiar with that type of review than others).
-

The ugly:
- Just about any review site whose URL contains the word "Game." These websites tend to be full of reviewers that seem to be random average highschool kids with no real understanding of what they are reviewing, how to write a review, or how to test or rate the product they are reviewing. The word "arbitrary" comes to mind. Note that I do not put Gamespot.com in this category (even considering their owner).
- CNet. I can't tell whether the reviewers are incompetent or bribed, but the reviews (usually, but not always) are quite lame.

Any opinions on other sites?

Two words: PR

[Eminence]

Right now IT journalists live in a symbiotic relationship with various PR types, who companies hire to project desired image of their products and themselves to the public using the media. This relationship rarely takes form of direct bribes or buying reviews, especially respectable media don't do that. It's rather that PRs meet with journalists all the time, build relationships, distribute pre-release hardware for reviews (and it's up to them who will get it first), do them some little favors etc. So if a given piece of hardware is really bad no journalist would cover that up, but some minor quirks won't be exposed or treated too harshly in the review.

PR is a neat invention. Just the idea of making it someone's responsibility and livelihood to ensure that products and company get presented in the media in a desired way and with desired frequency is brilliant. Those people are really good at it, the result is that currently technical journalism without corporate PR is hard to imagine.

Re:Trust? On the net?

[Ohreally_factor]

I used to work for a company that did training in digital video technology. I was a trainer, sent around the country to train people on Final Cut Pro. I was also a producer on several tutorial CD Roms, and did product reviews on the company website.

We also did marketing CDs for several equipment manufacturers, although our name never appeared in any credits. That part of the business was hush hush. We weren't to disclose those relationships.

We got lots of free equipment from these manufacturers, and others that wanted to be reviewed. We were specifically told to push certain products at the seminars and classes, and to give good reviews to "our" manufacturers products. We were also supposed to push product on the web site's forums, when people asked for advice. We never returned anything. In fact, the boss would give product to us employees as bonuses. I just checked the website, and they're still doing the same thing.

I openly questioned the ethics of this, and this and other factors led the boss to conclude that I wasn't a "team player", so I was let go. Honestly, I was glad, because there was a lot of other bullshit at that company as well. For a while, though, I was able to rationalize this because the products we were pushing were good products (for example, Canon DV cameras or Miller tripods). Still, when something is wrong, you know it's wrong, no matter how you try to justify it. If it wasn't wrong, why did we have to hide our business relationship with Canon?

Since this is pretty much par for the course, I think that if a magazine or web site really wanted to present unbiased, ethical reviews, they should state somewhere that they return the equipment, that their editorial is separate from advertising, and also disclose if the product manufacturer is an advertiser.

Of course, there still could be abuses, but when someone puts a policy in writing, it's a little harder to wiggle around basic ethics.

Reviewers are all liars, whores, and crooks!!

[geekwithsoul]

I tested and wrote for a trade publication that did hardware and software reviews for three years. During that time, I learned the following:

• Companies will do anything they can to 1) get products reviewed and 2) get products reviewed favorably
• It is usually not worth it for companies to ask for most products back, so the publication/reviewers usually end up keeping them. I'm not just talking USB keys either, think brand new notebooks, PCs, servers, and monitors
• While there was much talk about the separation between sales and editorial, whenever it came time to hand out awards, it was always amazing to see staff recommendations changed to the benefit of major advertisers
• Almost every review ever written is entirely made up of sections almost straight from a press release or the back of the box. I've spotted many where they didn't even bother rewording them.
• Freelance reviewers (which I sadly wasn't) can make excellent money. One that worked for my publication pulled down $250,000 (US) in a single year for a number of half-ass reviews and buyer's guides.
• Until there is a publication that, like Consumer Reports, does not accept advertising AND has the technological expertise and resources to truly test equipment, never believe a single review you read in a print or online publication.

Anand's Rebuttal

[Anonymous Coward]

Hahah.. Anand posted this rebuttal in his latest blog:

$26

My dad came to this country with something like $26 in his pocket; $26 and a scholarship to UMASS. I didn't know about that until I was around 14, our family went to another family's house for dinner and it came up in after dinner conversation. I was honestly floored - all of the sudden everything in life made sense to me, I was given the drive that I needed to go anywhere and do anything in life. My dad took essentially nothing and raised a family out of it; we all helped, but one man's ability to do that is what I truly define as successful. We weren't rich, but he (along with the help of my mom) made sure that we could live in comfort as a family. My dad is what sparked my drive; he's the reason that if I get a product to review on Friday, and the review has to go up on Monday, I bust my ass all weekend to make sure it gets done. He instilled in me a true understanding of what hard work is really about, and that's a major cornerstone of who I am today.

My mom gave me an understanding of how to do something with myself and an understanding of ethics. She showed me what truly caring about something really meant, about what being selfless meant and gave me the foundation that allowed me to develop my own perspectives on the world. She didn't teach me right from wrong, she taught me how to figure out what's right and what's wrong. And I'll never forget that which she's taught me to this day. It wasn't until college that I really understood what she had done for me; she dropped out of college to take care of me, to raise me. She gave up her dreams of being a doctor, to live her new dream of having a son. She cared for me more than anyone ever could, and seeing and understanding that also helped shaped who I am today.

I started AnandTech almost exactly 8 years ago: April 26, 1997. I was a freshman in high school, 14 at the time, and completely into this stuff. I started AnandTech not as a business, but as something that I thought would be cool to do. I started it humble, and to this day I will never forget my beginnings. There's no room for big egos in writing, I hate reading it and I'm sure you all do too.

I started the website with nothing, it was a free site on Geocities and I had no hardware other than the scraps of my system. But I worked hard these past 8 years, AnandTech grew from nothing to where it is today - with over 6 million monthly unique readers. I've had one basic principle when it comes to how to deal with those readers, and it goes something like this:

Regardless of how many people come to the site, I look at it as each person coming to me with their money in hand, wanting to know what to purchase. Let's say the average hardware upgrade costs $150, that's 6 million people x $150. I don't have to let you know that that's an absolutely ridiculous amount of money. To trade the trust that you all are placing in me and my staff for any amount of anything, is just unfathomable. While I'm sure there are folks that do it, I am not one of them.

At the same time, if we didn't value your trust so highly, we'd be gone in an instant. AnandTech readers make their buying decisions based, in part, on our articles. If we gave some bad advice that resulted in a poor purchase, do you think we're going to keep those readers for long? Nope, common sense right?

Next let's talk about this myth of articles and exclusivity. To a journalist, an exclusive on an article is a huge deal, because it means that you'll get all the attention about this one topic. Yet another reason why I hate journalists, they are far too short sighted. One thing I learned very early on (and you'll notice this in the work I do) is that being first to break a story gives you a large influx of short term traffic, but does nothing for you long term. You can have all the exclusives in the world, but if your content is crap then they mean nothing. At the same time, you can be 3 weeks late to review something, but if it's the most thorou

Re:Two Words

[Bill Privatus]

If true, that is one of the most damning links I've ever viewed.

I've had limited-skills techs working for me in the past - usually it's the "software testers" - who refer to disk as "memory", but to see it at a hardware review site is just incredible .

There's simply no other explanation than corruption. The site's not being phished. It might be a DNS poisoning attack, but I doubt it.

It may be a case where an "editorial board" saw a check and passed, unedited, unexamined even, a piece of dreck so obviously unworthy of publishing that the child who authored it - or the marketroid - would have received failing grade in Computer Science 101.

And, as if you could not guess, there is no byline. No name. No title, no email address.

This is the nuclear bomb for Tom's Hardware. Nothing less. It makes me sad - they have been around forever, and now they're dead to me.

I was a Tom's fan for years - I bought my BP6 (dual celerons), then my VP6 (dual P-III), based on Tom's say-so (along with a few others I trusted) - but no more :-(

Don't mod me up, mod the parent up - between 500,000 and 1,000,000 visitors to that horrible web page will cause Tom's to pull it, and for those multitudes to never again darken "Tom's Door".

Any opinions are mine alone, not those of the honest, ethical software/hardware/services multinational company for which I labor so assiduously.