Slashdot Mirror
More on Futuremark and nVidia
AzrealAO writes "Futuremark and nVidia have released statements regarding the controversy over nVidia driver optimzations and the FutureMark 2003 Benchmark. "Futuremark now has a deeper understanding of the situation and NVIDIA's optimization strategy. In the light of this, Futuremark now states that NVIDIA's driver design is an application specific optimization and not a cheat."" So nVidia's drivers are optimized specifically to run 3DMark2003... and that's not a cheat.
and i didn't use a cheat sheet, i used a memory priming sheet.
I never said I was smart, I just said I was smarter than you
But then, I'm Bill Gates.
Oh, it seems some careless individual has left this big pile of money on the table! Well, we'll just leave for a few moments and maybe when we come back it will have gone away.
I think [H]ardOCP stated it best as "Futuremark didn't want to get sued by Nvidia". Nvidia has the legal and financial resources to totally ruin Futuremark and they know it.
And now Futuremark has totally invalidated their own benchmark software by declaring it "open season" for hardware manufacturers to distort the "tests" in any way shape or form they desire to make the numbers higher.
N.
"Nothing strengthens authority so much as silence." - Charles de Gaulle
Well, that's excellent... now they can put 'Designed to run 3Dmark2003' on Nvidia product boxes!
I am a filthy pirate.
Cripes already. No one even BOTHERS with #DMark anymore, and after this fiasco no one is ever going to bother with them again. Gamers will use REAL EVERYDAY GAMES to see what runs the fastest again. Looking at some goofy simulation app coming up with scores and people buying into the company and people tricking drivers for particular tests is just crappy and makes 3DMark 100% invalid to any of my concerns in the future. I will only trust reviews that benchmark the latest and greatest games that I will be buying these cards for, whoever can run them fastest at that particular time IS WHAT IM GOING TO BUY. Peroid. Enough of this 3DMark BS.
If a benchmark doesn't measure performance related to real-world applications, what's the point? If a driver is optimized to run a benchmark faster, that SHOULD mean that the real world apps should run faster, too. If not, the benchmark is useless.
It's not an optimization if it does not produce the same results! Recall that the shader code that the driver used did not produce the same visual results as the shader code it replaced.
... based purely on application-specific pattern matching, which by its very nature is fragile. As was demonstrated so aptly by the 'off the rails' mode in Futuremark. This isn't an accidental bug: it is obvious that such mechanisms are highly fragile, and are almost certain to cause bad rendering on these applications when they are modified in small ways.
More tellingly, the driver deliberately flaunts the D3D spec by omitting buffer clears, mucking about with clip planes, etc.
As others have said, Futuremark's statement is just covering their legal arse. If someone modifies their code to get better scores in some benchmarks while introducing deliberate bugs (i.e. incorrect rendering), it's a cheat in my book.
That is right, this is not a cheat.. we are just redesigning the arrow and repainting the target so they match;)
Harpo Tunnel Syndrome--my wrist feels funny.
I think the money quote is:
However, recent developments in the graphics industry and game development suggest that a different approach for game performance benchmarking might be needed, where manufacturer-specific code path optimization is directly in the code source. Futuremark will consider whether this approach is needed in its future benchmarks.
I can sort of see the argument here, but it basically ruins the point of having a standard interface like DirectX. It's also like telling your math teacher, "no, it would be easier for my equations if you made 1+1=3. Now do it because I'm your star student."
Ha, yes and we all know how well that works, don't we? You mark my words, we'll see nVidia's tanks rolling over Poland by Christmas...
I am TheRaven on Soylent News
All video card drivers have algorithms to keep from rendering unneccessary portions of the map. What NVidia did, I believe, was to bypass those algorithms and hard-code into the driver the portions that needed to be rendered. That wouldn't work in a real game, where the card must decide what to render at run-time based on user input. Therefor, it's cheating, no different from including an MPEG of the entire 3Dmark demo and showing it in lieu of actually rendering it.
It's hard to be religious when certain people are never incinerated by bolts of lightning.
Futuremark Statement:
Paragraph 1: We're making a statement.
Paragraph 2: nVidia didn't really cheat.
Paragraph 3: Most computer games cheat.
Paragraph 4: We don't allow companies to cheat in their code.
Paragraph 5: Therefore, we should cheat in ours.
Nvidia Statement: They should have worked with us for a better cheat.
Joint Statement:We should all cheat together.
Footer 1: Futuremark rocks.
Footer 2: Don't steal our IP.
Footer 3: Nvidia rocks.
Footer 4: Really it does.
Footer 5: Of course, we could be lying.
Footer 6: Don't steal our IP.
The ______ Agenda
Don't get me wrong, there is nothing wrong with application-specific optimizations.
But this misses the whole point of 3dmark 2003. Different developers stress the pixel and triangle pipelines in different ways to produce a whole boatload of effects. While major games and engines are often optimized-for, there is no guarantee that ATI or Nvidia will sit down and optimize for the game you just bought.
That said, 3dmark 2003 should be considered a relational tool for generic perfrormance. Consider it a good bet that if two cards perform similarly and acceptably, the two cards should be able to run almost any DX8/DX9 game off the shelf acceptably.
The fact that Nvidia's unopitmized drivers perform significantly behind ATI's unoptimized drivers in 3dmark 2003 raises a significant question:
We all know how well the 5900 does in Quake III, Serious Sam 2, UT2003, etc, but how does it do in ?
I want to know that if I take *insert random DX8 game here* home to play, IT WILL PERFORM WELL. That is the entire point of having a benchmark like 3dmark. To do application-specific optimizations for it is to nullify the entire point of the benchmark.
Man is the animal that laughs.
And occasionally whores for Karma.