Slashdot Mirror
Memory Manufacturers Could be Cheating
Mark Brown writes "Tom's Hardware is live-testing DDR2 memory products in order to determine whether memory manufacturers submit cherry-picked products for reviews. 'GeIL DDR2-667 that was claimed to be purchased performed worse than the review samples they got: 471 MHz for the review samples vs. 421 MHz for the retail memory.'"
Oh, my, what ever will we do? Maybe the memory manufacturers should divorce and marry other companies.
In other words, set their affairs in proper order...
I never trust specs on any product I buy. For example, if I buy a hard drive the first thing I do is open it up, shake all the bits out it and count them. If they don't add up to exactly what is listed in the spec, I return it.
Oh dear lord, a company wants to make sure their product gets the best review possible and tests it before they send it.
I'm shocked!
... Job seekers have been putting ONLY their best accomplishments on their resumes
... Advertisers are STAGING their product photo shoots
... etc
No way, there can't be anyone making dishonest or cheap mem... PAGE_FAULT_IN_NONPAGED_AREA
There's a reason why Consumer Reports buys all of their products for testing through normal retail outlets.
Obviously companies will test any unit they send out to be reviewed to make sure it works as well as it can. The question is, how many other units did they test? If they only went out and bought one other unit, and the discrepancy was that large, it could be that the unit they bought was defective. They would need to buy several units from several retailers, preferrably in geographically dispersed areas, to get a real feel for how well these things will perform on average.
In order to evaluate this claim we need to know about the reliability of the test. What is the variance if the test is repeated many times on the same RAM? Without this piece of information we don't know if 50 MHz is a small or large difference, or if even if it is a real one.
Memory is rated to perform within certain specifications. If it doesn't perform well within this range, that's a legitimate complaint.
Tom's is complaining about something totally different. They are seeing how well the memory will overclock. But the manufacturer makes no claims about how well it will overclock. They explicitly tell you that they cannot guarantee what will happen. This is a reasonable position on their part.
But what Tom's is asking is for all memory from a given manufacturer to overclock the same. This is crazy. The manufacturer has every right to switch production methods and to make other changes which could affect overclocking performance. The only question should be: does the memory perfom as specified.
If you overclock your memory and it works well, good for you. But you have no right to complain if overclocking doesn't work as well as you want!
1) The article says that they bump the clock rate until the systems crash. ... just to carry on running with bad data.
... one for the review sample, and one for the retail purchased.
I'd be a little happier with running a memory test and running at progressively faster speeds until it detects an error. Some memory errors might not cause the system to crash
2) They have two "identical" systems
How do they know that all the components in the identical systems really have exactly the same specs? It would be more fair use just one system, or after the tests complete to swap the ram and re-run.
Taking a sample size of 1, not really. Their test leaves something to be desired. They really ought to be testing both memories in both systems, several times before jumping to conclusions. Slight variations in PCBs and silicon can build up to cause appreciable differences. Ultimately overclocking is taking entire designs well outside their specified operating limits. To do this reliably you need to test thoroughly on many samples.
The part of it that convinced me that they're right anyhow is the memory supply voltages. "Normal" on the cherry picked Gigabyte board was ~2.2V, normal on the storebought was ~1.83V (FVI 1.8V is the DDR2 spec supply voltage). You'll have to take my word for it, but THAT variation is huge. People who build computers do not tolerate voltage discrepancies like that, it's out of spec for the devices which usually allow 5% variation (1.71V-1.89V). You can verify this by going to Hynix/Micron/Infineon and pulling down a DDR2 component datasheet.
The headline is beyond wrong though, it's probably actually criminal. GeIL does not control the memory supply voltage (they make the DIMM), Gigabyte does (they make the mobo). GIGABYTE is cheating.
It's very easy to figure out if memory makers are cheating: take the heatsink off, look at the device part numbers and look them up. There's not a whole lot to tweak that doesn't involve a complete redesign of the DIMM. If they cheat it's almost always because they used a DDR2-400 device but branded their DIMM as DDR2-something_higher.
The ram was rated as DDR2 667 even the retail at 421 MHZ. That comes out to DDR2-842 doesn't it?
The ram met and far exceeded it's rated clock speed. Sure the give good stuff to reviewers. If the review sites want to do valid tests of which brand of ram is the best for over clocking they would have to purchase multiple samples of each brand from the retail channel.
When overclocking the truth is your results may very. If you are pushing past specs then some will work and some will not. Heck even different production batches will give different averages.
See my blog http://ilovecookes.blogspot.com/ for light hearted technical information.
Horsepower in cars rarely meets up with the numbers. Fuel efficiency, either. Carb content in food is labeled, but most people don't read the serving size, so that is advertising funk, too.
Why should this be different? When a company ships a product to be reviewed and tested, they'll ship the best. When they test their own, they'll test the best. You should NEVER accept that specs are factual, and you should spend some time confirming what you bought.
This is the great thing about specs -- if they're lies, just return the product. If a company lies enough, the customers will go elsewhere.
It is really all common sense.
http://www.hardocp.com/reviews.html?cat=MjUsRGVza3 RvcCBDb21wdXRlcnMsaGNvbnN1bWVyLCws
What they are doing is having other people buying systems and then reviewing those systems. They will only review systems where they have an agreement with the manufacturer that the computer can be returned at the end of the review. The key is that the manufacturer never knows who is getting a system which may be subject to review.
It actually works well for both parties. Some manufacturers are proactive in the forums and even acted on complaints received, strengthing their processes.
* Winners compare their achievements to their goals, losers compare theirs to that of others.
Hmmm... If I read this right, it looks like the motherboard that came w/ the memory had its voltage increased to induce higher speeds. This would skew any test - not just overclocking, unless you knew to reset it.
Well, the article says the RAM is DDR2-667 which (I'm pretty sure) implies a clock rate of 333MHz (somebody correct me if I wrong). So gamers are still getting more than they paid for...
What I want to know is where do Tom's Hardware get off thinking this is statiscally significant? Basically their saying "We took one part from the suppliers, and one part from retail sources. The retail parts performed worse. OH MY GOD, that must mean they're cheating!!!" Compare numerous examples from each source and then I'll be more easily swayed to their argument...
To me, the increased voltage on Gigabyte's motherboard is far more interesting...
TWW
"Encyclopedia" is to "Wikipedia" what "Library" is to "Some people at a bus stop"
Correct me if I'm wrong, but isn't DDR2-667 only certified to run at 333MHz? Either way, 471 MHz and 421 MHz are both well above that... It's not as if they're claiming it runs at 471 and it actually runs at 421... they're only guaranteeing it to run at 333... right?
From TFA:
"Its DDR2-667 memory......"
"maximum clock speed of 471 MHz, which corresponds to DDR2-942"
vs
"a memory clock of 421 MHz (DDR2-842)"
So its more than 20% faster than what it is rated at... Whats the big deal? Everyone knows there are certain processors/memory modules from the same exact part# that outperform others. This has been the case since before the Celeron 300a even. If the memory performed below its rating, then there would be a problem
If all of the manufacturers cheat, then none of them are cheating.
Recall the hubub from as recently as a half-decade ago, when video card manufacturers were rigging their drivers (or the cards themselves) to recognize when they were being asked to draw the same patterns over and over again (like, say, 10,000 colored boxes, or circles... like benchmark programs do) and would silently decide to perform only a fraction of them to jack the benchmark numbers up?
Never, ever trust the results from an item that the company sent you when they knew you were a reviewer. You should just go out and buy one off the shelf in a store. If you can't afford to do that, buy one from a store and ask the company for a review sample, return the sample to the store and test the, now free, one that you got "in the wild", as it were.
Hardly.
A DDR-667 chip (or more specifically, a PC2-5300 stick) is supposed to run at at 333 MHz. So one runs at 421 MHz and the other runs at 471 MHz. To me, it looks like both of those sticks are performing way faster than the specification requires.
Isn't this just the price the user pays for being too stingy to pay for a memory stick which is actually rated to run at 400 MHz in the first place?
Karma: It's all a bunch of tree-huggin' hippy crap!
Corporations are ripping off its customers with rigged tests... I'm truly shocked.
They aren't necessarily rigging anything -- chip production runs always produce a range of qualities, and they're submitting the best they have. To not do so, especially when everyone else does, would be to sabotage your own reviews. There are no "unbiased" samples.
The only practical way to fix this is to establish a standard for what companies should send in -- preferably something like five to ten random chips that have passed basic testing.
It's tragic. Laugh.
Let's see - the GeIL memory is rated at DDR2-533. The module from the vendor ran at DDR2-942. The module from the store ran at DDR2-842. Now, Tom makes this out to be some big controversy, but it seems to me that a module running 36% faster than specified is no small thing, particularly at that high of a data rate.
I'm an engineer who designs memory modules. In most cases, our modules are overclockable, at least to some degree - some go faster than others. At the sort of speed that Tom's Hardware is running, I'm not really surprised that there's more than a 2 or 3% variation in performance, espeically if the chips on those modules came from different manufacturing lots. At the outer limits of memory speed performance, the tiniest changes in parasitic capacitance can be death to performance - and those values change from lot to lot, even from wafer to wafer.
When manufacturers specify that 2% to 3% tolerance, they're referring to the module's performance at its rated speed, and that makes sense. Plug two modules into a system and they will run in virtual lockstep - at their rated speed. There are a million analogies that I could use, but the bottom line is that there are assumptions and statements in Tom's article that just aren't right.
Maybe the module was cherry-picked and maybe it wasn't, but, if nothing else, a sample of two doesn't make for much of a study. After all, if the retail module had been DOA, a pedantic person could say that GeIL cherry-picked the evaluation samples and sends all the defective modules to retail.
-h-
Get the writer loaded and laid.
Seriously. Many years ago, I worked as a technician for a (now defunct) major audio equipment manufacturer. When a writer from "Stereo Review" or "Audio" magazine came to visit, we'd play with the equipment a little, my Engineering boss would hand him some specs, and they'd go out on the town (leaving me to work the rest of the day {grumble, grumble}). A few months later, we'd see those exact specs printed in the magazine, along with some well-placed ads. I never believe a review I read in a trade publication.
Consumer Reports lacks technical expertise in many areas, but at least their approach has some level of integrity.
Research shows that 67% of those who use the term "research shows", are just making shit up.
Actually, it's innate human nature to think of things that way. Put a one pound weight in one hand, and a two pound weight in the other - virtually everybody will be able to tell the difference between the two. Now put a forty pound weight in one hand, and a forty-one pound weight in the other - very few people will be able to tell the difference, despite the fact that it's a difference of one pound in both cases.
The reason we perceive the two cases differently is that, in the first case, "B" is twice as heavy as "A", whereas in the second case, "B" is only 2.5% heavier than "A". Or if you don't have heavy objects handy, get a three-way lightbulb and a lamp to match. Notice how the jump from 50 to 100 watts seems like a bigger jump in brightness than the jump from 100 watts to 150 watts. That's because, in percentage terms, it is a bigger jump. It's how we're wired to see the world, in terms of percentage differences.
ABSURDITY, n.: A statement or belief manifestly inconsistent with one's own opinion.
So if you want the best stuff, convince them you're a review site and just wait for them to ship you the cream de la creme.
"It's the height of ridiculousness to say for those 9 lines you get hundreds of millions."
i work at hp, and i'm sure this is standard practice across every industry... all review units go through a series of stringent screening process to determine the absolutely best units.
seriously think about it.... if you had a hot date, would you show up in a yellow wife-beater, messy hair and bad breathe, and ask her to pay the cab that's been waiting for them for the last 30 minutes?
You're having problems with the megaHURTS?
She: Hey, are you a traitor? Me: No, I'm atheist.
Tom's Hardware and other reviewers may not be able to buy their tested items from retailers, but I can think of a great way to get retail items without any cost to them. When they receive a "cherry picked" piece of hardware they can post it on their website and ask for users to register to purchase a matching retail item to trade. The "winning" user can then get a retail part, ship it to the reviewer, and receive the primo hardware in return. This way the reviewer gets to test both parts, and the user has a good chance of getting a hand-picked piece of hardware. Win-win. Just an idea.
What about the reviewers? To be accepting free samples (aka bribes) seems asking to get tricked like this. I doubt the review companies/reviewers care, as they kind of like their perks. A little like lobbyists and politicians.
The only magazine I know of that buys their test samples retail is Consumer Reports, and they do it for this reason (as well as to avoid any conflict of interest).
Let me explain. If a major computer manufacturer, major RAM stick manufacturer, etc. needs parts, they get a contract---usually months or years in advance---with one or more of the vendors that provide that part. In that contract, the chip manufacturer (in this case) agrees to provide a quantity of parts at a particular speed, with guarantees that the RAM stick manufacturer (in this case) will be able to get that many at the speed in question, and at the price specified.
What usually happens in these cases is that the manufacturer of the part can't call it a faster part because that could be in violation of the contract terms for other manufacturers which may specify that they can't sell the faster parts to their competitors at a lower price. Thus, they are required to remark the chips at a lower speed than they were actually built to support in order to comply with their contractual obligations. This sort of thing can happen regularly, particularly if the manufacturer operates fairly close to their maximum yield; it doesn't take much to botch a batch of chips.
Thus, it would not be at all surprising if the DRAM stick vendor ended up making some runs with higher quality parts than they originally specified. There's no reason to assume that the DRAM stick vendor knew that the parts were above-spec parts at the time because they would not have been marked as such.
To assume that the DRAM stick vendor did this on purpose from a single sample is a pretty big stretch. Now if you see a pattern of this, it might be worth looking into. As it is, it just sounds like FUD to me.
Check out my sci-fi/humor trilogy at PatriotsBooks.
Humans also perceive sound in a similar manner.
When the frequency of a sound is doubled, it's perceived one octave higher. (The frequency of a note f(n) = F 2^(n/12) where n is the note, 0 signifying the A above middle C, 1 signifying A#, 2 signifying B etc. and typically F = 440 Hz, i.e. the "concert pitch", or the frequency of the A above middle C.)
The loudness of a sound is also perceived in a logarithmic manner. The level of a sound must increase with an accelerating rate in order for us to perceive a linear change in loudness. (The difference in sound pressure level is often measured in decibels. The function is 10 log (p_2^2/p_1^2) dB = 20 log (p_2/p_1) dB where p_1 is the reference sound pressure and p_2 is the measured sound pressure.)
Please excuse any mistakes. It's late, i'm very tired and i don't seem to be able to concentrate at all. :-)
So what you're basically saying is, someone is using the product outside of the product's operating specifications, and then bitching because some other guy was able to use it *further* outside of the operating specifications.
I still don't see the problem here, except perhaps the problem that overclockers are a little too enthusiastic about saving those extra few dollars.
Karma: It's all a bunch of tree-huggin' hippy crap!
I do, its not about that, your missing the point.
The point is, it doesn't matter what the review was about, or what the user was doing. The reviewer of a product is taking the product, using it, or performing tests on it, and then using the information to inform the public, the consumers.
To really do that, they have to have sample of the product that is representative of what you can buy in the store. Its not like you could go out and buy a new memory stick from a retail store that was tested out and deemed to be the best of the best. You buy one that tested out to be "within spec".
The fact is, if... IF they are right, and the RAM manafacturer is testing the ram to a higher standard, and then sending them to reviewers, then they are essentially asking a reviewer to review a product thats not actually available to consumers, and passing it off as one that is (that is ram, tested to a lower standard)
That means they are not only being dishonest, but tricking an independant third party into shilling for them.
-Steve
"I opened my eyes, and everything went dark again"