Benchmarking the Benchmarks

apoppin writes "HardOCP put video card benchmarking on trial and comes back with some pretty incredible verdicts. They show one video returning benchmark scores much better than another compared to what you get when you actually play the game. Lies, damn lies, and benchmarks."

Comments

OSS

[Anonymous Coward]

Its no wonder that most modern benchmarks are innacurrate, given that they tend to benchmark propietary, closed source software, running on propietary, closed source operating systems. Where they to run benchmarking software on Open Source operating systems, such as Ubuntu, then their results would not only be more accurate, but fairer. The fact that Open Source software would also have much higher scores then propietary, closed source software goes without saying.

Re:OSS

[snoyberg]

PS yes...release your rage and mod me down.... just makes my post more Insightful.

Translation: if you mod me down, I will become more insightful than you can possibly imagine.

Re:OSS

[edwdig]

Either I misunderstood you, or I don't see how the license can be a metric of performance or accuracy.

Clearly you haven't been drinking enough of your Kool Aid. Please contact the FSF and request more immediately.

back in my day...

[Aranykai]

We used to benchmark a computer by *gasp* actually running things on it. If you wanted to find out how well it would perform running a game, you played the damn game and found out. Course, thats not good enough for these ubernoobs who think they are cool with their benchmark scores on their forum signatures...

Re:back in my day...

[SQLGuru]

And, on top of that, they are on your lawn....

Layne

Re:back in my day...

[IndustrialComplex]

I do remember marveling at my friend's 486 and how fast those cards bounced off the screen.

whatevermark

[Yath]

Crysis, UT3, and COD4 are the three primary games we are using currently, with Crysis performance certainly being the new watermark in the industry.

I have no idea what this means, but it certainly sounds like Crysis has left its mark somewhere or other.

hmm

[nomadic]

Is your benchmark of the benchmarks accurate? We might have to benchmark it.

My old benchmark

[Anonymous Coward]

I used to do this benchmark:
10 PRINT TIME$
20 FOR I=1 TO 9999
30 NEXT I
40 PRINT TIME$

I then improved it to be:
10 A$=TIME$
20 IF A$=TIME$ THEN GOTO 20 !breaks out when the seconds change
30 I=1:A$=TIME$
40 I=I+1:IF A$=TIME$ THEN GOTO 40
50 PRINT I

Ahhh...the good old days... (1970s, early 1980s)

Re:My old benchmark

[sempernoctis]

My favorite benchmark for finding the size of the memory heap:

void doit(int i) { printf("%i\n", i); doit(i + 1); }

worked really well until I tried it in an environment where the call stack could get paged...then it turned into a hard drive benchmark

Re:1st Post

[SQLGuru]

Apparently you were using the wrong benchmark. You just thought you were fast.

Layne

We need international benchmarking standards!

[Thanshin]

...And an international benchmarking committee.

To avoid concentrating all the data management in a single entity, we need a national benchmarking committee for each country and then international elections to get a chief of benchmarking interrelationships or CBI.

To avoid the possible corruption of the CBI, we would need an independent international supervision committee for the review of benchmarking standards.

The IISCRBS would review the actions of the CBI yearly and produce a thorough report.

That report (which would be called the IISCRBS-CBI report) would be the main reference to start any kind of productive debate about who has the leetest rack and who's a lame n00b.

Re:Would like to see a real world comparison for E

[Digital Vomit]

I have what was a "hot" card only eighteen months ago (7800) ago and now it is stuttering on some of the newer content when I'm raiding.

Are you one of those software pirates?