Slashdot Mirror
Are Review Units Better Than Store Versions?
Anonymous Howard writes "Every now and then you hear about hardware manufacturers optimizing their hardware for certain tests or games to make their hardware look superior. I was surprised to hear of a new controversy brewing over reviewer units sent to hardware reviewers. This article claims that Samsung is sending LCD monitors with a contrast ratio of 700:1 when the consumer version of the same monitor has a contrast ratio of 450:1. Various sites list different specs for the same model, so it's somewhat confusing to know for sure which is correct. I don't doubt this happens, but I'm surprised that it would be this blatant. Has anyone heard of other stories of manufacturers being deceptive so that they could get better reviews?"
Review units are free and Store units are...well...not free. That would sway my opinion.
Well, you know how some radeon graphics cards can be 'unlocked' and some can't? I'll give you one guess at which I bet radeon sent to all the reviewers.
When anger rises, think of the consequences.
Confucius (551 BC - 479 BC)
Han shoots first. I think it's different in the retail version.
I heard Mandrake only sent copies of 9.2 to reviewers who didn't have LG CDROMS.
buy their stuff off the shelf to use in reviews. Otherwise companies will send the cherries to reviewers.
I worked for a couple of electronic manufacturers that had a standard operating policy to do this very thing.
Take the cheese to sickbay, the doctor should see it as soon as possible - B'Elanna Torres, "Learning Curve"
But only by the fact that Samsung have never sent me any such thing.
Dammit, I got into this business for the corruption. But do I get over-spec high-dollar hardware, automobiles or prostitutes? No, I do not. It's a bloody swindle, I tell you.
Look, Samsung. 20 inch diagonal, 1600 by 1200, 700:1 contrast ratio, 16ms response time. Is that too much to ask?
Delivery address provided on application. Favourable review guaranteed.
This is a crummy thing for the companies to do but it also makes you wonder about the reliability of reviewing companies. Like how he stated that designtechnica prided itself on reviewing retail products, but then never explained why they were using a review unit, and after noticing the discrepency did a lot of talking but still did not bother to pick up a shelf unit and test it, to see if it was true. Most of the hardware reviewers seem really flakey to me, more fan boys than reliable testing labs.
Unfair tweeking is part of the reason why Consumer Reports never accepts review units from companies, but rather buys them from retail stores, just like anyone else would. The other reason is that receiving free stuff creates a potential conflict of interest which is why they also do not have any advertizing in their magazine or their website. This means that you won't have reviews out before products are released, and operating this way is more expensive, relying on subscribers to run, but it is worth it. I don't always agree with CR's subjective descriptions of products (cars especially), but the hard numbers they provide are the most usefull I have found, and have saved me plenty of money.
I really wish that there was some site equally trustworthy in the computing world. For providing informative analysies there are usefull sites (I have always been impressed with anandtech). But for reviewing components, I have yet to find one I trust.
Bah. They had the same model number on two different models. That's a lie.
Is the difference between an outright lie and a deception really that important here anyway? They were expected to send the same product real consumers get. They didn't. That's enough to condemn them in my book, whether there's an outright lie there or not.
has the same model number as other retailers, but a lower price. If you look at the Bestbuy HW vs the other retailers, the best Buy HW actually is missing some 'components/functionality'.
Take a look real hard at that stereo reciever before you buy it....
-Shadow
A while back, I bought a 17" samsung monitor which had 102kHz vertical refresh listed in the online "review" of the reseller, but upon closer examination, I discovered that it was, in fact, only 96kHz, so I informed them of this.
What they told me was quite strange at the time, they said their review unit had a different refresh rate and that they checked with Samsung, but that there was no definite answer as to how this could have happened. All in all, they gave me a 19" for free for the trouble (which they apparently had no part of.)
This happened in Toronto, Canada in 1998.
It is good to know SlashDot picks up on such small things.
Like in "here's a free expensive item for review that you get to keep. We'll be watching the review to see if you get anything else to review? Oh, it's still happening, but sending the reviewer a item that isn't the same as the crap they intend to sell you and me is just a little added insurance.
You can pretty much see this in a lot of reviews that are written too. The only reviews that merit much trust are the independent ones where the reviewer actually went out and got an off-the-shelf item to review; but this is an almost dead pratice. No only does the reviewer not get neat fre stuff then, but his review may be months after the reviews by the company shills come out, and he ends up with the same crap you and I get rather than the free good working versions.
I'm an American. I love this country and the freedoms that we used to have.
Gezunteit.
#define DRM chmod 000
Yes, companies will send their best stuff to reviewers, but there is a push from the opposite direction: they want reviewers to get their stuff early. In the computer world, this means that reviewers often get essentially prototypes. I've found that "first test" reviews of CPU's get processors that are worse than what the consumer will buy once production ramps up, because by then, many bugs get ironed out. AMD chips overclock much better later into the production process compared to the "for review only" samples. That's just one example.
I was reading a review of a Subrau WRX on caranddriver.com and it mentioned that their original review unit was able to post a 0-60 time of about 5.4 seconds. The requested another car from Subaru for a "long-term" test drive, which for them is about 60,000 miles over the span of about two years. This long-term car was equipped very similarly to the previous model, yet, it was never able do below a 5.9 0-60 time.
Now, ever car enthusiast knows that 0-60 times and such the like are subject to various conditions, but that's a pretty large inequity in the difference between the two cars. They said they must have just gotten a lucky hot car, but I believe that perhaps they got a cherry that didn't have to last as long as the car on the long-term test. If they were only going to have the car for a few weeks, then it didn't matter if it was as reliable as a longterm car, so they upped a few things and gave it to them for review. Same thing with the monitors, I guess. Since its just for the flat panel review, they might as well spice it up. These companies base a ton of business on "independent" reviews, so I suppose its worth it to fix the results.
And there's even an advertisment campaign for Gainward's line of graphics cards that specifically pokes at this concept, and doing so for quite some time ... here's just an example, and a Google search turns up many more results of this advertising campaign and the resulting products from it ...
...
Perhaps I'm just overly cynical, but I tend to trust reviews where the reviewer went out and purchased an off-the-shelf retail copy of X rather than those where the company sent something. Of course, this is hard to do in print publications, because of the time-lag that magazines run through (ie, two months after it's released on the shelves, they have a review of it), but I see no reason (aside from money, which is a big reason) that online reviewers can't do things such as this. I also tend to look towards user-reviews and give those a pretty good weigh-in when I'm making a purchase decision. This is the first instance that I can recall where products are blatently better when given to reviewers than those that are store-bought, but I get the feeling that it's been done in the past.
The above paragraph reflects what I do for my personal buying choices and should in no way construe that that's the optimal/correct/whatever way for large corporations/organizations/whatever to buy-in-bulk
topreacher@signature.slashdot.org 1% rm -rf sig
Every once in a while i read the Auto comparisons on Consumer Reports to get myself all worked up.
Consumer Reports will not accept donations of vehicles or products from manufacturers or vendors just for this reason. They will discreetly send someone out 'under cover' to go acquire the products in an "off the lot" or "off the shelf" state.
This is good, and commendable.
However, i see a lot of times they will end up mis-matching the cars and trucks they compare. Usually it is simply a matter of trim levels on similar classed models. This *will* have an impact on the final outcome. Obviously it's difficult to do things *exactly*.
Less often, but still wrongly, they will compare vehicles from incompatible classes- things like Buick Century vs. E-class Mercedes vs. Toyota Camry. Or the classic truck comparisons with the 3/4 ton, V8 powered Dodge and Chevy fullsize trucks, against a V6 F150, against V6 Toyota Tundra and Nissan.
Consumer Reports might do this to other product reviews too, but i only pay attention to their auto ads for `entertainment'.
I guess that no matter what, *any* test can be flawed.
do() || do_not();
No, they do not. Cars that must be stock for racing are stripped down and rebuilt with individual parts off the line that most closely match spec.
This is far more certain than testing every vehicle to find the "good" ones, which will never, ever, be quite as good as one assembled specifically to be good.
Even well heeled amatuers with access to a dealer's or distributer's parts bins do this. Hence classes like Star Mazda and Legends where the motor can only be touched by an official builder and has a seal affixed to it to prevent tampering.
This practice was first started in the 60's by the official Austrian Formula Vee team ( a class where every engine part must be absolutely box stock). Jochen Rindt simply ran away from the international field with a perfectly legal engine whose parts had all been individually cherry picked.
But the engine was completely stock.
KFG
I review computer hardware/software for a daily newspaper, and I can tell you that, in many cases, the quality of the hardware I get is less than what you'd see on store shelves. This is because reviewers often get pre-production units, which are essentially lacking some of the bells/whistles and fit/finish of production-line products. Whenever possible, I try to insist on production models, but that's not always possible. It's even tougher for dead-tree magazines, which work on lead times of months, to get full production units.
That said, there's no way for reviewers to know with pre-production units whether they are getting what will eventually be on the shelf - and it may not be a case of the manufacturer trying to get away with something. A processor in a pre-production unit may be faster, or an LCD screen have a greater contrast ratio, than what ends up at retail, but the reason often is that design changes are made at the last minute related to cost or part availability. In fact, sometimes the product may be less powerful in pre-production than what is finally delivered to buyers. This was particularly true in the days of falling RAM prices - I'd get review PCs with 128 MB of RAM, and when they shipped they'd have 256 MB.
C'mon, baby, kiss The King.
The early Samsung 955DF was a perfect 19" flat-black screen CRT with .20mm dotpitch. The control panel was a rectangle in the center that when pushed, slowly slid diagonally down to reveal the control buttons. Very slick.
Early Samsung 955DF
Now the "Samsung 955DF" has controls on the front, the screens are much more reflective and oily-looking, and black appears grey even when the brightness is all the way down. More recent Samsung 955DF
You've never bought a digital SLR camera from New York -based website, have you?
Yes, great price, 15% to 30% below everyone else. Then once you order it, they call and mention that the camera doesn't have a warranty, that's extra. You know, the plastic lens mount is of lesser quality, you'd be much better of with model with the metal mounting ring. Of course the battery charger and battery is not included in that model.
Whoops, the price is now 25% above everyone else, and you have the standard package that everyone sells. Those other models you've upgraded from don't exist.
Of course, if you stick to your guns and insist on the advertised price, it's mysteriously never in stock or gets lost in the shipping system.
BTW, there are 3 or 4 genuine web shops in NY, check DPReview forums, etc to find them.
Fuzzbuster gets busted by Escort and Car And Driver Mag for stuffing Escort guts inside of the Fuzzbuster supplied for radar detector review (1979):
:)
http://www.valentine1.com/lab/MikesLabRpt5.asp
While you're there, please check out a Valentine One. Mike Valentine makes by far the best detector on the planet, and he's a heck of a nice guy!
--
Out of order? Fuck! Even in the future nothing works! - Dark Helmet (Rick Moranis) "Spaceballs"