Slashdot Mirror
Are Review Units Better Than Store Versions?
Anonymous Howard writes "Every now and then you hear about hardware manufacturers optimizing their hardware for certain tests or games to make their hardware look superior. I was surprised to hear of a new controversy brewing over reviewer units sent to hardware reviewers. This article claims that Samsung is sending LCD monitors with a contrast ratio of 700:1 when the consumer version of the same monitor has a contrast ratio of 450:1. Various sites list different specs for the same model, so it's somewhat confusing to know for sure which is correct. I don't doubt this happens, but I'm surprised that it would be this blatant. Has anyone heard of other stories of manufacturers being deceptive so that they could get better reviews?"
Han shoots first. I think it's different in the retail version.
But only by the fact that Samsung have never sent me any such thing.
Dammit, I got into this business for the corruption. But do I get over-spec high-dollar hardware, automobiles or prostitutes? No, I do not. It's a bloody swindle, I tell you.
Look, Samsung. 20 inch diagonal, 1600 by 1200, 700:1 contrast ratio, 16ms response time. Is that too much to ask?
Delivery address provided on application. Favourable review guaranteed.
This is a crummy thing for the companies to do but it also makes you wonder about the reliability of reviewing companies. Like how he stated that designtechnica prided itself on reviewing retail products, but then never explained why they were using a review unit, and after noticing the discrepency did a lot of talking but still did not bother to pick up a shelf unit and test it, to see if it was true. Most of the hardware reviewers seem really flakey to me, more fan boys than reliable testing labs.
Unfair tweeking is part of the reason why Consumer Reports never accepts review units from companies, but rather buys them from retail stores, just like anyone else would. The other reason is that receiving free stuff creates a potential conflict of interest which is why they also do not have any advertizing in their magazine or their website. This means that you won't have reviews out before products are released, and operating this way is more expensive, relying on subscribers to run, but it is worth it. I don't always agree with CR's subjective descriptions of products (cars especially), but the hard numbers they provide are the most usefull I have found, and have saved me plenty of money.
I really wish that there was some site equally trustworthy in the computing world. For providing informative analysies there are usefull sites (I have always been impressed with anandtech). But for reviewing components, I have yet to find one I trust.
has the same model number as other retailers, but a lower price. If you look at the Bestbuy HW vs the other retailers, the best Buy HW actually is missing some 'components/functionality'.
Take a look real hard at that stereo reciever before you buy it....
-Shadow
I was reading a review of a Subrau WRX on caranddriver.com and it mentioned that their original review unit was able to post a 0-60 time of about 5.4 seconds. The requested another car from Subaru for a "long-term" test drive, which for them is about 60,000 miles over the span of about two years. This long-term car was equipped very similarly to the previous model, yet, it was never able do below a 5.9 0-60 time.
Now, ever car enthusiast knows that 0-60 times and such the like are subject to various conditions, but that's a pretty large inequity in the difference between the two cars. They said they must have just gotten a lucky hot car, but I believe that perhaps they got a cherry that didn't have to last as long as the car on the long-term test. If they were only going to have the car for a few weeks, then it didn't matter if it was as reliable as a longterm car, so they upped a few things and gave it to them for review. Same thing with the monitors, I guess. Since its just for the flat panel review, they might as well spice it up. These companies base a ton of business on "independent" reviews, so I suppose its worth it to fix the results.
Just because they don't accept advertising, and they buy units off the shelf, doesn't mean the CR review is a de-facto "better review," or is definitely less biased than a trade magazine review. Based on my experience, they represent the worst possible example of "sound bite product reviews," and they rarely give me truly useful data. (For examples, read one of their loudspeaker, audio receiver, or sports car reviews. These are generally valuable only for the pictures.)
CR tries to distill down all sorts of subtle performance parameters into a box score that ranks easily against competitive products, and in the meantime, miss the value of those parameters. Quite honestly, I'd be surprised if CR could accurately determine if they had a cherry LCD display or not, given the "rounding error" of their review/comparison methodology.
Tim
I review computer hardware/software for a daily newspaper, and I can tell you that, in many cases, the quality of the hardware I get is less than what you'd see on store shelves. This is because reviewers often get pre-production units, which are essentially lacking some of the bells/whistles and fit/finish of production-line products. Whenever possible, I try to insist on production models, but that's not always possible. It's even tougher for dead-tree magazines, which work on lead times of months, to get full production units.
That said, there's no way for reviewers to know with pre-production units whether they are getting what will eventually be on the shelf - and it may not be a case of the manufacturer trying to get away with something. A processor in a pre-production unit may be faster, or an LCD screen have a greater contrast ratio, than what ends up at retail, but the reason often is that design changes are made at the last minute related to cost or part availability. In fact, sometimes the product may be less powerful in pre-production than what is finally delivered to buyers. This was particularly true in the days of falling RAM prices - I'd get review PCs with 128 MB of RAM, and when they shipped they'd have 256 MB.
C'mon, baby, kiss The King.