Slashdot Mirror
Are Review Units Better Than Store Versions?
Anonymous Howard writes "Every now and then you hear about hardware manufacturers optimizing their hardware for certain tests or games to make their hardware look superior. I was surprised to hear of a new controversy brewing over reviewer units sent to hardware reviewers. This article claims that Samsung is sending LCD monitors with a contrast ratio of 700:1 when the consumer version of the same monitor has a contrast ratio of 450:1. Various sites list different specs for the same model, so it's somewhat confusing to know for sure which is correct. I don't doubt this happens, but I'm surprised that it would be this blatant. Has anyone heard of other stories of manufacturers being deceptive so that they could get better reviews?"
Review units are free and Store units are...well...not free. That would sway my opinion.
Well, you know how some radeon graphics cards can be 'unlocked' and some can't? I'll give you one guess at which I bet radeon sent to all the reviewers.
When anger rises, think of the consequences.
Confucius (551 BC - 479 BC)
Han shoots first. I think it's different in the retail version.
buy their stuff off the shelf to use in reviews. Otherwise companies will send the cherries to reviewers.
I worked for a couple of electronic manufacturers that had a standard operating policy to do this very thing.
Take the cheese to sickbay, the doctor should see it as soon as possible - B'Elanna Torres, "Learning Curve"
But only by the fact that Samsung have never sent me any such thing.
Dammit, I got into this business for the corruption. But do I get over-spec high-dollar hardware, automobiles or prostitutes? No, I do not. It's a bloody swindle, I tell you.
Look, Samsung. 20 inch diagonal, 1600 by 1200, 700:1 contrast ratio, 16ms response time. Is that too much to ask?
Delivery address provided on application. Favourable review guaranteed.
This is a crummy thing for the companies to do but it also makes you wonder about the reliability of reviewing companies. Like how he stated that designtechnica prided itself on reviewing retail products, but then never explained why they were using a review unit, and after noticing the discrepency did a lot of talking but still did not bother to pick up a shelf unit and test it, to see if it was true. Most of the hardware reviewers seem really flakey to me, more fan boys than reliable testing labs.
Unfair tweeking is part of the reason why Consumer Reports never accepts review units from companies, but rather buys them from retail stores, just like anyone else would. The other reason is that receiving free stuff creates a potential conflict of interest which is why they also do not have any advertizing in their magazine or their website. This means that you won't have reviews out before products are released, and operating this way is more expensive, relying on subscribers to run, but it is worth it. I don't always agree with CR's subjective descriptions of products (cars especially), but the hard numbers they provide are the most usefull I have found, and have saved me plenty of money.
I really wish that there was some site equally trustworthy in the computing world. For providing informative analysies there are usefull sites (I have always been impressed with anandtech). But for reviewing components, I have yet to find one I trust.
Bah. They had the same model number on two different models. That's a lie.
Is the difference between an outright lie and a deception really that important here anyway? They were expected to send the same product real consumers get. They didn't. That's enough to condemn them in my book, whether there's an outright lie there or not.
has the same model number as other retailers, but a lower price. If you look at the Bestbuy HW vs the other retailers, the best Buy HW actually is missing some 'components/functionality'.
Take a look real hard at that stereo reciever before you buy it....
-Shadow
Like in "here's a free expensive item for review that you get to keep. We'll be watching the review to see if you get anything else to review? Oh, it's still happening, but sending the reviewer a item that isn't the same as the crap they intend to sell you and me is just a little added insurance.
You can pretty much see this in a lot of reviews that are written too. The only reviews that merit much trust are the independent ones where the reviewer actually went out and got an off-the-shelf item to review; but this is an almost dead pratice. No only does the reviewer not get neat fre stuff then, but his review may be months after the reviews by the company shills come out, and he ends up with the same crap you and I get rather than the free good working versions.
I'm an American. I love this country and the freedoms that we used to have.
Yes, companies will send their best stuff to reviewers, but there is a push from the opposite direction: they want reviewers to get their stuff early. In the computer world, this means that reviewers often get essentially prototypes. I've found that "first test" reviews of CPU's get processors that are worse than what the consumer will buy once production ramps up, because by then, many bugs get ironed out. AMD chips overclock much better later into the production process compared to the "for review only" samples. That's just one example.
I was reading a review of a Subrau WRX on caranddriver.com and it mentioned that their original review unit was able to post a 0-60 time of about 5.4 seconds. The requested another car from Subaru for a "long-term" test drive, which for them is about 60,000 miles over the span of about two years. This long-term car was equipped very similarly to the previous model, yet, it was never able do below a 5.9 0-60 time.
Now, ever car enthusiast knows that 0-60 times and such the like are subject to various conditions, but that's a pretty large inequity in the difference between the two cars. They said they must have just gotten a lucky hot car, but I believe that perhaps they got a cherry that didn't have to last as long as the car on the long-term test. If they were only going to have the car for a few weeks, then it didn't matter if it was as reliable as a longterm car, so they upped a few things and gave it to them for review. Same thing with the monitors, I guess. Since its just for the flat panel review, they might as well spice it up. These companies base a ton of business on "independent" reviews, so I suppose its worth it to fix the results.
No, they do not. Cars that must be stock for racing are stripped down and rebuilt with individual parts off the line that most closely match spec.
This is far more certain than testing every vehicle to find the "good" ones, which will never, ever, be quite as good as one assembled specifically to be good.
Even well heeled amatuers with access to a dealer's or distributer's parts bins do this. Hence classes like Star Mazda and Legends where the motor can only be touched by an official builder and has a seal affixed to it to prevent tampering.
This practice was first started in the 60's by the official Austrian Formula Vee team ( a class where every engine part must be absolutely box stock). Jochen Rindt simply ran away from the international field with a perfectly legal engine whose parts had all been individually cherry picked.
But the engine was completely stock.
KFG
I review computer hardware/software for a daily newspaper, and I can tell you that, in many cases, the quality of the hardware I get is less than what you'd see on store shelves. This is because reviewers often get pre-production units, which are essentially lacking some of the bells/whistles and fit/finish of production-line products. Whenever possible, I try to insist on production models, but that's not always possible. It's even tougher for dead-tree magazines, which work on lead times of months, to get full production units.
That said, there's no way for reviewers to know with pre-production units whether they are getting what will eventually be on the shelf - and it may not be a case of the manufacturer trying to get away with something. A processor in a pre-production unit may be faster, or an LCD screen have a greater contrast ratio, than what ends up at retail, but the reason often is that design changes are made at the last minute related to cost or part availability. In fact, sometimes the product may be less powerful in pre-production than what is finally delivered to buyers. This was particularly true in the days of falling RAM prices - I'd get review PCs with 128 MB of RAM, and when they shipped they'd have 256 MB.
C'mon, baby, kiss The King.
You've never bought a digital SLR camera from New York -based website, have you?
Yes, great price, 15% to 30% below everyone else. Then once you order it, they call and mention that the camera doesn't have a warranty, that's extra. You know, the plastic lens mount is of lesser quality, you'd be much better of with model with the metal mounting ring. Of course the battery charger and battery is not included in that model.
Whoops, the price is now 25% above everyone else, and you have the standard package that everyone sells. Those other models you've upgraded from don't exist.
Of course, if you stick to your guns and insist on the advertised price, it's mysteriously never in stock or gets lost in the shipping system.
BTW, there are 3 or 4 genuine web shops in NY, check DPReview forums, etc to find them.