Slashdot Mirror

InQuest has published a study of the Pentium IV. It found that because of the P4's cache design, it uses 4 times the memory bandwith a PIII does on random access data benchmarks. Even on benchmarks which benefit from the larger cache-line size, the P4 shows no benefit due to its higher clock rate. Furthermore, the 1.5GHz P4's thermal diode throttles the part to effectively 750Mhz as soon as power consumption exceeds 54.7W. Without this limitation, a P4's maximum consumption would be 72.9 watts, similar to a 1.33 GHz Athlon's 73 watts.

General Breetai asks: "This is yet another benchmark showing DDR is faster than RDRAM. I'm not posting this as another DDR vs. RDRAM debate, but rather to question the benchmark methods. When looking through the results, it's clear that the OS and test SW has a huge impact on the results. How can we trust SW benchmarks to test a particular subsystem when it's clear that the SW relies on the entire system? In the case of new memory systems, I propose that the only reliable method is to compare published timing data in a theoretical setting. SW is just too dependent on too many things to be reliable, especially if it is running on top of a complex OS." Are there ways to benchmark hardware components using some measure that isn't explicitly tied to the OS on which said benchmarks run?

"Before all of you SW types out there get all defensive. You'll have to prove to me how the benchmark software is not affected by the entire system. Even between to identically configured systems, there will still be some amount of random error incurred by the OS's management of the system. On top of that, when trying to evaluate a particular subsystem's performance, you cannot show how other subsystems are affecting that performance. For instance, when moving large chunks of data in and out of memory to measure memory performance, how can you prove that no other part of the system is stealing bandwidth? How do you know that the AGP or HDD drivers aren't moving data as well? In addition, since a lot of performance differences are controlled by low level OS drivers and BIOS configurations, how do you know that you are truly measuring the memory performance, and not how well the OS drivers are optimized? BIOS can also play a big part. There have been many cases where performance increases with a new BIOS version because better chipset settings where used. This can be viewed best when two boards with identical chipsets are compared, and one performs better than the other, sometimes significantly. For the above mentioned article, one can ask if the results are skewed by the use of custom Micron OS improvements required to support their Samurai chipset? The conclusion, IMHO, is that benchmarks should be viewed as overall system measurments, even if they are designed to measure a specific part of the system.

The best way to measure a hardware subsystem like the memory channel, is to sit down a simulate transactions based on the published electrical timing specs provided by the vendors. Since these memory busses have very specific protocols, you can critically evaluate the performance based on how well the protocol performs under specific theoretical circumstances. I.E choose a specific amount of data to transfer, and then show how well the two different protocols can move the data. You can then tweak the parameters to find the best settings. In doing so, you can show, down to the picosecond, which protocol can move data the fastest without interference from the rest of the system. However, this all has to be done in a theoretical situation, independent of OS & SW.

By adding the rest of the overall system and the OS which can both steal uncontrolled bandwidth, you are actually yeilding suspect results. Let the flames fly, but what are your thoughts on this? Has anyone seen a hardware/electrical timing comparison between RDRAM and DDR?"