Intel Caught Cheating In 3DMark Benchmark

EconolineCrush writes "3DMark Vantage developer Futuremark has clear guidelines for what sort of driver optimizations are permitted with its graphics benchmark. Intel's current Windows 7 drivers appear to be in direct violation, offloading the graphics workload onto the CPU to artificially inflate scores for the company's integrated graphics chipsets. The Tech Report lays out the evidence, along with Intel's response, and illustrates that 3DMark scores don't necessarily track with game performance, anyway."

Comments

If you're too lazy to RTFA...

[Jonboy X]

Just look at the pics. Changing the name of the executable changed the results dramatically. The driver is apparently detecting when it's running a 3DMark (or some other specific apps) and switches to some other mode to boost its scores/FPS markings.

Re:Wonder if AMD plays fair?

[afidel]

Oh, ATI was one of the first to cheat on a graphics benchmark quack.exe anyone?

Re:Eh?

[parallel_prankster]

Effectively dividing tasks among CPUs is not the issue here. They want to benchmark the GPU and they wanna make sure you don't enable optimizations that are targeted specifically for the benchmark which Intel was doing shamelessly.

Re:Good reporting there Ric

[palegray.net]

The driver apparently detects "crysis.exe" and inflates performance metrics by offloading processing, whereas renaming the executable to "crisis.exe" gives realistic performance scores. Please RTFA before replying.

Re:Wonder if AMD plays fair?

[0123456]

It's really hard to get 15% better performance without doing something underhanded unless your previous drivers were beta quality.

I used to work for a video card manufacturer and game and video developers often did totally retarded things which just happened to work on the cards they developed on but made the software run like crap on ours. We routinely had to implement workarounds for individual games to make them run properly on our cards.

One particular example which springs to mind -- I won't mention the developer or the game -- was an engine which used a feature which we supported in hardware but a certain other card manufacturer whose cards they used performed in software. Rather than configuring said feature once as they should have done, retarded developer repeatedly reconfigured it numerous times in the course of a single video frame, which required us to reconfigure the hardware every time -- slow as heck over an AGP bus -- whereas other card manufacturer just had to execute a few CPU instructions. We had to detect the game and disable our hardware support, so that we would fall back to software and run the retarded code much faster; in that instance there were places in the game where, far from a measly 15%, we'd literally be going from seconds per frame to numerous frames per second.

So it's quite possible to need to detect individual games or applications in order to work around retarded coding which cripples performance on your hardware. The line you shouldn't cross -- and which I don't believe we ever did -- was to render something other than what the developer intended, for example by detecting a shader used by a benchmark and replacing it with one that looked similar but didn't do as much work.

Similarly, the issue here is not Intel punting processing to the CPU when the GPU is overloaded, but the fact that they do so by detecting the name of the benchmark rather than by monitoring the GPU loading and dynamically switching between hardware and software so that it would work on any application. General optimisation is fine, workarounds for retarded developers are fine, but special optimisations for benchmarks which don't affect real applications is getting pretty close to the line.

Re:Good reporting there Ric

[Fred_A]

The driver apparently detects "crysis.exe" and inflates performance metrics by offloading processing, whereas renaming the executable to "crisis.exe" gives realistic performance scores. Please RTFA before replying.

Thanks for the tip, I've now renamed all my games to "crysis.exe" and am now enjoying a major speed boost. You've given my laptop a new youth !

I can finally get rid of that cumbersome i7 box with that noisy nVidia !