Your Chance To Influence CPU Benchmarking

John Henning writes "When comparing CPUs, is it enough to look at MHz? Of course not; architecture matters, as do caches, memory systems, and compilers. Perhaps the best-known vendor-neutral CPU performance comparison is from SPEC, but SPEC plans to retire its current CPU benchmarks. If you would like to influence the benchmarks that will replace the current set, time is running out: SPEC Search Program entries are due by midnight, June 30."

Comments

Who cares?

[kawika]

I think we're at the point where it doesn't matter what a synthetic benchmark says about the performance of a CPU. The top end of today's processors have plenty of power for what 95% of people use them to do. The workloads of the remaining 5% are specialized enough that a synthetic benchmark is unlikely to be a good predictor.

I would rather have a really big and fast RAID array, 2GB of RAM, or a 2Mbps Internet connection than a faster CPU.

Re:But I don't know anything about CPUs...

[Wesley Felter]

You joke, but SPEC is a consortium made up of IBM, Intel, Sun, HP, etc.

WB/s

[jrpascucci]

Hi,

The most effective benchmark I can think of for typical use is Windows Boots per second (WB/s).

First of all, restarting is the single most used feature of Windows. :-)

But beyond that, what's funny is I'm not kidding: it does more or less everything you want it to do - lots of disk IO, lots of processing, lots of memory access.

WB/s should be measured from power on to 'quiescence' - that is, when the services have finished initializing and are 'ready for action'. This goes beyond gina-time login to actually being able to, for instance, start up an IE and connect to yourself.

This figure has stayed nearly constant for 5ish years, at about 0.005 WB/s (i.e. about 2 and a half minutes between power on and being able to really do stuff). Even 'hibernate' (the ultimate fake optimization for WB/s), is only .06 WB/s.

Ultimately, I'm waiting for a 10 WB/s CPU. Then, I'll be happy. BSOD? Who cares.

J

A spec with multiple marks

[mnmn]

Various SPEC benchmarks should emulate a desktop, CAD workstation, server, game etc computers, and the SPEC results should always be summarized as SPEC(x,y,z,w...) where each of those variables corresponds to the different applications emulated. A game machine uses the CPU in a very different way than a server.. more IO, less task switching. While CAD users compare CPUs for their own applications and do not need other numbers that show the performance for games, servers etc, these values cannot be united either because that will be too general then.