Slashdot Mirror
Futuremark Replies to Nvidia's Claims
Nathan writes "Tero Sarkkinen, Executive Vice President of Sales and Marketing at Futuremark, has commented on the claims by Nvidia that 3DMark2003 intentionally puts the GeforceFX in bad light, after Nvidia had declined becoming a member of Futuremark's beta program. This issue looks like it will get worse before it gets better." ATI also seems to be guilty of tweaking their drivers to recognize 3DMark.
I suppose you have to expect some poor practices considering that the top 3DMark card will be considered by many gamers to be the best to buy. It's a massive temptation for such a big industry. I find ATI's decision to remove code which it claims boosts over all performance quite funny.
ATI also is crafty at tweaking their drivers to suck. They should be working on decent drivers instead of cheating on stupid benchmarks to get +1.9%.
I bought a Radeon 9700 Pro. The driver issues almost make it not worth the increased FPS over my Ti4400.
The refresh rate problem in XP is annoying as hell. ATI handles it even worse than NVidia, where you set your "maximum" refresh rate and your "highest" resolution, and it assumes that all lower resolutions can only handle that refresh rate.
There's no way to tell your ATI card, "My monitor can do 1280x1024 @ 85hz, but 1600x1200 @ only 75hz." You either get 75hz in everything if you want 1600x1200, or you get 85hz up to 1280x1024, and have to avoid 1600x1200 use lest ye risk getting "frequency over range".
NV handles it better with the driver, allowing you to set maximum refresh rates for every resolution individually.
These refresh rate tweaking programs don't help either, since half the games out there choke when you use them. PC 3d is in a bit of a sorry state right now, and I'm tired of it.
# Erik
This whole episode has turned into a big mess. NVDA seems to be the bad guy in all of this. Their DX-9 product was delayed and their existing products where only DX 8.0*. The benchmark heavily favours DX-9 parts and NVDA's existing lineup was/is getting smoked in the benchmark by it's main (only) competitor. They decided to go on the offensive and try to kill off this benchmark. The 30 person company that produces 3D Mark have stood their ground against the multi-billion dollar NVDA. NVDA instead of admitting that their Pixel Shader is quite slow when running against 2.0 specs insteads tries to decieve and FUD their way out of it. Looks like they got more than just some patents when they purchased 3DFX...
Now they have painted themselves into a corner and how this will turn out is anyone's guess.
*DX8.1 has PS 1.4 which is actually much closer (AFAIK) to PS 2.0 than PS 1.3 (DX8).
AFAIK, ATI displays the graphics on screen properly, the drivers are just optimized for the benchmark. One could still consider this cheating. NVIDIA however does not display the graphics properly, it really does cut corners (literally) to get higher scores. ATI got an extra 3% from cheating. NVIDIA got a whopping 24% higher scores from cheating! take a look at the extremetech screenshots:
, a= 41574,00.asp
http://www.extremetech.com/print_article/0,3998
Hardocp
They do a good job of disecting the benchmark, and I'd have to agree that as a DX9 benchmark it fails.
Whatever, it's still just a synthetic mark and nothing more.
-- taking over the world, we are.
from the what-exactly-are-kid-gloves dept.
Get the answer at Straight Dope
Look there is a clear difference between what NVIDIA and ATI have done here. ATI are not cheating, they look at a sequence of instructions and reorder them if they fit a pattern, but they do exactly the same thing as before. This is central to the kinds of things optimizing compilers and CPUs do. Maybe you thing it's too narrow a path, but it's a minor infraction at best compared to the blatant cheats of NVIDIA, who not only rewrote shaders but did several otehr really heinous things, like disableing screen clear and adding their own hidden clip planes.
It's a real shame that The Register obscured the truth here with an article that attacks ATI for conservatively removing optimizations while giving the real miscreant gets a free pass. ATI should leave their optimizations in IMHO, but maybe you disagree because their mathematically equivalent optimization is not general enough, it's a close call, but they don't deserve what the distorted treatment given in The Register.
This quote is misleading. "DirectX9" alone means nothing.
We need to look at the new shader features offered by DirectX9, these are:
- Pixel and Vertex shaders 2.0 (supported by ATI R3xx line and GeForceFX)
- extended Pixel and Vertex shaders 2.0 (supported only by GeForceFX)
- Pixel and Vertex shaders 3.0 (no support until R400/NV40)
Now let's look at the features which are used by 3DMark03:
- Game 1: no shaders at all, only static T&L
- Game 2: vertex shader 1.1 and pixel shader 1.4 (which isn't natively supported by NVIDIA cards)
- Game 3: vertex shader 1.1 and pixel shader 1.4 (which isn't natively supported by NVIDIA cards)
- Game 4: vertex shader 2.0 and pixel shader 1.4+2.0
This means that:
-DirectX9 offers three new different shaders.
-Three of four 3DMark03 demos don't use new DirectX9 shaders at all
-Three of four 3DMark03 demos use Pixel Shader 1.4 which was introduced with DirectX8.1 and isn't natively supported by NVIDIA cards
-Only one set of new DirectX9 shaders are partially used in one 3DMark03 demo
Thus 3DMark03 shouldn't be called "DirectX9" benchmark. Following quote: "If hardware performs well 3DMark03, it performs well in all applications that use DirectX 9" should be changed: "If hardware performs well 3DMark03, it performs well in all applications that use Pixel Shader 1.4"
Quake 3 vs Quack 3 Go troll elsewhere. They're both guilty from time to time of doing the same bullshit.
This happens so often in grade school I'm surprised the computer industry hasn't caught on to it yet. If you give students a copy of the exam the night before the exam, the only material they are going to bother to study the question-answer pairs on that exam, and may just remember what the answer to #6 is rather than even try to understand the question.
In order for a driver benchmark to be useful at all, it needs to be kept absolutely secret from the chip manufacturers before the test, and then once it is used and revealed that benchmark needs to be retired, because the next generation of testing should be designed to concentrate on the new features that the graphic card developers are expected to put in their next generation of cards that will be used in the next generation of games.
In short, the best benchmark will always be based on "that sure-to-hit game that's just about to come out."
What's going on is that the driver is applying the wrong optimizations to the graphics command stream. Since these hidden tweaks were designed to work on known code, they are very dependent on it. Whatever is used to render the leaves is completely broken by whatever changes are performed to boost Q3 scores, and the leaves and wings both break when it thinks it's running 3DM2K3.
quoting from http://www.gamersdepot.com/interviews/nvidia/3dfx
so nope, nvidia does not have the whole 3dfx dev. team (although a few or some of the engineers might have been employed by them after 3dfx went down)
OEMs make decisions based on cost, feature set and possibly name recognition, but only if it adds value to their product. They care about the business relationships they have with the vendors, and whether they can get price breaks, and whether the vendor's product integrates easily with their own.
Benchmarks that differ by a couple of percent depending on which test is run are not going to make a big difference in the overall decision process. If they made decisions based on benchmarks then ATI would have closed its doors many years ago, since until very recently they have been consistently outclassed by their competitors performance-wise for several years. However, ATI has done VERY well in the OEM market during this time not due to better performance, but due the the factors I listed.
Don't get me wrong, there is nothing wrong with application-specific optimizations.
But this misses the whole point of 3dmark 2003. Different developers stress the pixel and triangle pipelines in different ways to produce a whole boatload of effects. While major games and engines are often optimized-for, there is no guarantee that ATI or Nvidia will sit down and optimize for the game you just bought.
That said, 3dmark 2003 should be considered a relational tool for generic perfrormance. Consider it a good bet that if two cards perform similarly and acceptably, the two cards should be able to run almost any DX8/DX9 game off the shelf acceptably.
The fact that Nvidia's unopitmized drivers perform significantly behind ATI's unoptimized drivers in 3dmark 2003 raises a significant question:
We all know how well the 5900 does in Quake III, Serious Sam 2, UT2003, etc, but how does it do in ?
Man is the animal that laughs.
And occasionally whores for Karma.
Nothing will even use the new kit to its fullest for that long.
I'm rapidly coming to the conclusion that the way to go with video cards is to buy one a year old. It's much cheaper, and typically handles all current and near future games perfectly well. The new gizmos, and speed boosts, on these cards rarely provide worthwhile bang for your buck these days.
Use the money you save to buy a faster processor, more RAM, a RAID array or something else that provides a useful improvement in performance outside of the theoretical. Or if you're buying/upgrading card + monitor together, get an extra couple of inches of screen real estate or go for a nice flat panel. The difference in price really is of that order, yet the difference in ability is irrelevant for almost all real applications.
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
half the dev team according to brian burke
JC is off on this. The specification document for "ARB_fragment_program" does not define a specific precision. The program can set the option "ARB_precision_hint_fastest" or the option "ARB_precision_hint_nicest". However these options are only a guide as the drivers have control as to which precision is used. However "GL_NV_fragment_program" allows the program to use fx12, fp16, and fp32 data types. I would like to know what precision the NV30 path uses and which precision hint Doom III uses for the ARB2 path. I also kind of wonder how long it will take nVidia to "optimize" its drivers for Doom III so that the ARB2 path will use fp16 regardless of which precision hint is specified by Doom III.
Impersonating Tycho from Penny Arcade since before there was a PA.
ATI has played it smart this round. The minimum precision for DX9 is 24 bits and ATI does it.
Hmmm... Pie...