More on Futuremark and nVidia

AzrealAO writes "Futuremark and nVidia have released statements regarding the controversy over nVidia driver optimzations and the FutureMark 2003 Benchmark. "Futuremark now has a deeper understanding of the situation and NVIDIA's optimization strategy. In the light of this, Futuremark now states that NVIDIA's driver design is an application specific optimization and not a cheat."" So nVidia's drivers are optimized specifically to run 3DMark2003... and that's not a cheat.

Comments

Re:How is this news?

[0123456]

"Besides, pick any other industry, any other product, and companies are optimizing their products to run fast"

Yes, but nvidia's "optimizations" would be equivalent to a database engine that improved its benchmark score by simply throwing away half the requests, or a web server which improved its benchmark score by redirecting half the requests to slashdot. Is that an optimization, or a cheat?

Come on people!

[MarcoAtWork]

This is like the emperor's new clothes: in my opinion this is probably what happened:

- NVIDIA for some reason decides to pull out of the FutureMark tax program (aka, 'development program' or whatever the heck they call it)

- FM threatens NVIDIA that either they come back onboard or FM will 'leak' a devel version of FM that can be used to portray NVIDIA in a bad light (and probably -only- NVIDIA, the 8% ATI loss was incidental)

- NVIDIA doesn't budge

- FM leaks it and some 'review sites' go all gung-ho over it

- NVIDIA threatens to sue FM out of existance (presumably they think they have a solid case) and ask for damages due to reputation loss etc. etc. etc.

- FM caves and does some creative spin doctoring about the word 'cheating'.

-every- video card manufacturer cheats on benchmarks, come on, if you were the CEO of ATI/NVIDIA wouldn't you do everything in your power to make your hardware look better than your competitor's?

It should be the benchmark sites that should earn their keep and do 'real' benchmarking of the cards instead of spending 5 minutes setting up a test environment with the usual demos and going back to sleep or something.

When a new card comes out, a reputable test site, instead of using demo001 which can be special-cased in drivers for example, should connect online to a crowded q3/ut2k3 server, play there for 15-20 minutes while recording a demo and using -THAT- demo to benchmark the new card =and= re-benchmark the competing card(s). The site should also obviously take screenshots and compare them to make sure that the drivers aren't cheating by lowering visual quality etc. etc. etc.

Obv. one can't go back and rebenchmark all the way to the TNT, but -at least- the current generation cards should be done, for comparisons to older models then, yeah, the older benchmarks should be used (but still not the canned demo001, but the 'online benchmarks' recorded when the older cards came out).

Just my 2c