madddog on Linux v NT Benchmarking

BogoMips sent us an interesting tidbit running in Performance Computing currently. Jon "maddog" Hall explains some of the benchmarking issues associates with the DH Brown reports, as well as the ubiquitous Mindcraft tests. Very well written article, IMHO.

Comments

Benchmarks

[oldzoot]

I, for one would like to see a set of benchmark results for Linux that would help a person to make decisions about the configuration of hardware platforms for systems. I would like to see a set of test results for different types of system activity, such as compiling code, raytracing/graphics/visualization, file system access, network bandwidth, combined network/filesystem access etc. This set of measurements could then be run on a variety of hardware types providing the basis of cost/performance decisions in the implementation of systems. One could answer questions like how does a K6/233 compare to a P5/233 ? How does a P5/233 compare to a P6/450 ? How much difference is there between an ULTRA ATA disk system and SCSI? Does the difference change with different processor speeds ? Does 1MB cache make a difference with what I want to do ?
If implementing a cluster, does saving $2K per box make up for the difference ( providing money for more boxes ) if you use ATA and slightly slower CPUs, rather than higher end platforms ?

I would love to see a single ( and evolving ) location for this kind of info. Hell, I would love to work on compiling it. I have seen some sites with benchmark info, but nothing that seems to try to answer specific questions. The sites that I have seen present specific objective numbers but it was hard to derive any context for the differences between systems.