Slashdot Mirror
3D Mark 2003 Sparks Controversy
cribb writes "3DMark 2003 is out, sparking an intense debate on how trustworthy its assessment of current graphics cards is, after some harsh words by nVidia and the reply from Futuremark. THG has an analysis of the current situation definately worth reading. The article exposes some problems with the new GeforceFX previously mentioned in a slashdot article on Doom3 and John Carmack. Alas, here seems to be no end to the troubles with the new nVidia flagship." If you've run the benchmark, post your scores here, and we'll all compare.
792 3DMarks.
System:
Geforce3Ti200 GFX
AthlonXP1700 CPU
256MB SDRAM
ECS K7S5A Mainboard
I don't like it. I'm gonna rely on actual game benchmarks when I compare my system's performance. Some good games to use:
Quake3 (still scales nicely)
UT2003 (the game sucks, but it's a decent CPU benchmark)
C&C: Generals (don't know how it scales, but it cripples most computers)
Doom3 (Will hopefully scale as well as Q3 when it comes out in 2 months)
Synthetic benchies just aren't that reliable anymore...
------- "From bored to fanboy in 3.8 asian girls" ----------
AMD Athlon1400C@1550
512MB Samsung DDR, CL2@147FSB
Geforce4ti4200, clocked@260core, 520memory
a whopping: 1080 points.
Did i mention that this benchmarks makes *heavy* utilization of the otherwis in *no* game used Pixelshader 1.4? Teh exact one, that Nvidia didnt implement in its GF4Ti cards - where only 1.3 and 1.1 is in?
Guess, who has 1.4 - ATI has...
You could also call this benchmark "ATIbench2003", but that was the same in 2000, when 3dmark2000 was favoring Nvidia cards over 3dfx simply because of the lack of 32bit colordepth.
Sheeeshh...
You should read Carmack's comment that pretty much summed up the gist of the debates:
The R200 path has a slight speed advantage over the ARB2 path on the R300, but only by a small margin, so it defaults to using the ARB2 path for the quality improvements. The NV30 runs the ARB2 path MUCH slower than the NV30 path. Half the speed at the moment. This is unfortunate, because when you do an exact, apples-to-apples comparison using exactly the same API, the R300 looks twice as fast, but when you use the vendor-specific paths, the NV30 wins.
The reason for this is that ATI does everything at high precision all the time, while NVIDIA internally supports three different precisions with different performances. To make it even more complicated, the exact precision that ATI uses is in between the floating point precisions offered by NVIDIA, so when NVIDIA runs fragment programs, they are at a higher precision than ATI's, which is some justification for the slower speed. NVIDIA assures me that there is a lot of room for improving the fragment program performance with improved driver compiler technology.
--
Error 500: Internal sig error
The guys at Tech Report also has an article in which they dissect parts of the benchmark and provide what both FutureMark and nVidia's comments on the matter.
IF you've run the benchmark, post your scores here, and we'll all compare.
Or you could just go directly to the futuremark forums instead.
Here is a great paper on the subject. The site is down, but Google has a cache of it.
A quote:
"Michael M, Editor-in-Chief of PC Magazine was looking at the executive report on the latest graphics benchmarks which were to appear in the June 29th issue. As he got deeper into the summary, his face took on a baffled look. He picked up the phone to call Bill M, Vice President for Technology, and asked him to come by his office with the detailed test results. Five minutes later, they were pouring over the data on Bill's laptop."
Source:
Hercules Cheating
It seems like the 3Dmark folks decided to deliberately test DX9 features, even though there are not many cards which support them in hardware yet. Nvidia is pissed because they have not implemented any DX9 features in hardware on the FX, where ATI has them on the 9x00 whatever.
This is a valid benchmark to use to test out how your current hardware will perform in a DX9 environment. I, for one, am glad to see such a tool available so that I can take DX9 performance into account when making my next video card purchase. So my next card may be an ATI - Who knew? The last ATI product I owned was a Number 9, not exactly a 3D monster....
You think emacs is evil?! You've never used VM's XEDIT have you?!! That's evil, baby!
Read about it here. http://www.theinquirer.net/?article=7920
"An Nvidia technical marketing manager confirmed to us that Geforce FX has 4 Pipelines and 2 Texture Memory Units that can results with 8 textures per clock but only in multitexturing.
However, Nvidia did say that there were some cases where its chip can turn out 8 pixels per clock. Here is a quote:
"GeForce FX 5800 and 5800 Ultra run at 8 pixels per clock for all of the following: a) z-rendering b) stencil operations c) texture operations d) shader operations"
and
"Only color+Z rendering is done at 4 pixels per clock"
We talked with many developers and they said me that all games these days use Color + Z rendering. So all this Nvidia talk about the possibility of rendering 8 pixels in special cases becomes irrelevant.
The bottom line is that when it comes to Color + Z rendering, the GeForce FX is only half as powerful as the older Radeon 9700."
This statement is false.
Additional, and IMHO better coverage at Tech Report:
Dissecting the 3DMark03 controversy
Examining graphics card performance in 3DMark03
3d Mark 2003 Score: 1252
Geforce 4 Ti 4600 @ AGP 4x
800 MHz PIII
256 MB RDRAM
Intel VC 820 Motherboard
Windows XP
Games & 3d Mark ran off of 80GB WD 8MB cache Special edition hard drive, alone on a seperate IDE card on the PCI bus.
For Games:
Simcity 4- large maps and pleasing resolutions bring my comp to it's knees. Running SC4 at 1024 & higher resolution is absolutely beautiful, running it at 800 x 600- it looks like ass.
RtoCW runs fine at 1024, haven't tried it higher yet.
Delta Force: Black Hawk down runs fine at 1024, with full effects. Haven't tried it higher yet. The water effects are stunning.
UT2003 ran fine when i had a GF2 in here, haven't tried it since.
my 2 cents
Alcohol, Tobacco and Firearms should be the name of a store, not a government agency.
There is no game that uses directx 9.0 or Pixel Shaders 2.0 but it sure nice to see how these so called direct9.0 graphics cards actually peform. And it has really nice eye candy to boot.
Have you ever been to a turkish prison?
The under-issue here is that nVidia is no longer a "partner" of madonion (I know they changed their name, whoever they are now, futuremark or whatever) but ATI is (IIRC). This is helping fuel suspicion that the benchmark is designed to perform better on ATI hardware than on nVidias. You must pay a fee to be a "partnet" so there is the unspoken idea that what Futuremark is doing might be some kind of extortion.
Where the answer lies is up to you. Personally, I do think that the benchmark is unfair/not a good benchmark. For example, chaning the graphics card in your computer should have next to no effect on the CPU score, if any; yet it has a measureable effect. But all of this is mute, IMHO, since Doom III will be the new Uber benchmark trusted above all else when it comes out. Untill then, argue amongst yourselves.
Comment forecast: Bits of genius surrounded by a sea of mediocrity.
Yes, it does matter (within reason, anyway). While your current card may do well enough at Quake 3 and the new cards may not have a huge margin over it (really, what's the difference between 150fps and 200fps except in the very rare situation where absolutely everything on the screen is blowing up or something), that's old technology. As hardware capabilities increase, software complexity also increases. That card getting you 150fps at 1024x768 in Q3 with 4x FSAA will likely barely break 30fps for Doom 3. (at that point, you tweak -- drop your resolution, turn off FSAA and anisotropic filtering, lower your detail levels, turn off unnecessary effects, etc and get up to a playable 50fps or so) The cards doing 200fps in Q3 will probably run D3 around 50-60fps. While there's little difference between 150-200fps, there's a world of difference between 30 and 60fps.
And just to head off any, "But your eye can only see 24/30/60fps anyway, who needs more?" arguments:
I've got a 9700 pro, p2.53, sis648 and 512 DDR400. Hardly a lowend rig. My 9700 chokes on 3dmark 2k3. At several points in the demos the FPS drops below 10.
If this benchmark is supposedly so horribly biased in favor of ATI, you'd think they might at least get it to run smooth on my 9700.
I think 3dmark may be accurately pointing out that this new wiz-bang high-precision stuff may only start to be gameworthy on the NV35/R350 or even NV40/R400 generations.
roughly what he's saying is:
If you just write an application then it will run twice as fast on the ati card as the geforce fx
But if you write two applications to to the same thing and optimize one for the ati card and the other for the nvidia card then the nvidia card does better
So performance wise nvidia appear to be relying on developers to optimise their applications specificaly for the geforce fx. And they probably will get it too given their current market share.
Clearly he attempted to run this program on a platform which did not support it (guessing UNIX or Linux).
However, there's nothing wrong with running this test from bash (assuming that it's a Windows test and not a DOS-based direct access test which is a safe bet for anything designed to test DirectX9 performance). There are very nice ports of bash to Windows including the one from Cygnus (included in their cygwin package).
What one might also try is:which would be an interesting test of Wine's DirectX support. I'm guessing WineX would be the only thing that could even get close to running this puppy, and even then I don't think WineX has DX9 support yet. Please chime in if you know, as I'm too lazy to check out the WineX site
WRONG!!!! 1.4 *IS* used in many games and many more. You are a fanboy for Nvidia and did not read the articles.
NVidia Fanboys HATE this list of games that use DirectX 8.1 (and 9.0 mandatory) Pixel shader 1.4:
UT2003
Madden 2003
Tiger Woods 2003
Nascar 2003
NeverWinter Nights (actually OGL equivalent).
DOOM3 will have a path that will use the equivalent of PS1.4 as well
It makes them WRONG when they pretend that PS 1.4 is NOT FOUND in modern or upcomming games yet!
That list is factual and not even ehaustive.
Buy ATI and get real 1.4 speed in the games that ARE and WILL use it!
Or wait 9 months for NVidia to catch up.
NVidia's poor performace have anything to do with the recently revealed fact that it does NOT have 8 rendering pipelines as it advertised,but only 4?
Read about it here. http://www.theinquirer.net/?article=7920
"An Nvidia technical marketing manager confirmed to us that Geforce FX has 4 Pipelines and 2 Texture Memory Units that can results with 8textures per clock but only in multitexturing.
However, Nvidia did say that there were some cases where its chip can turn out 8 pixels per clock. Here is a quote:
"GeForce FX 5800 and 5800 Ultra run at 8 pixels per clock for all of the following: a) z-rendering b) stencil operations c) texture operations d)shader operations" and "Only color+Z rendering is done at 4 pixels per clock"
Nearly ALL games use Color + Z rendering in 2002,2003 so except for CAD and Architectural walkthroughs, NVIDIA runs half as fast as ATI Radeon 9700
That is another reason why Apple promotes ATI
IT didn't just make it run faster. It turned down the Visual Quality lower than what you were asking for in order to make it run faster.
If I ask for Highest Visual Detail in a game, I expect Highest Visual Detail. I don't expect the Video Card Drivers to internally decide that I really meant Pretty High Visual Detail so that it can run it faster.
AMD XP TBred-B 2100+ OCed to 2700+ (166fsb x 13)
2x265MB DDR400 Clocked at 333Mhz, with 2-2-4-2 Timings (Dual Channel A7N8X Deluxe)
ATI Radeon 8500 Default Clocking
My Score was a wopping 1173 3DMARKS with
Program Version 3DMark03 Revision 1 Build 3
Resolution: 1024x768@32 bit
Texture Filtering: Optimal
Pixel Processing: None
Vertex Shaders: Optimal
Uh, PS1.1 and 1.4 are part of DX9; they just happen to be available on DX8 cards too. There's no reason why you _have_ to use PS2.0 on DX9 cards if earlier versions will work just as well... and it's likely that game developers will use the earlier versions where possible, for best backward compatibility with older cards.
Just imagine if every test had required DX9: people would be whining that their DX7 and DX8 cards couldn't run anything.
Uhh.. You didn't read the reply, did you? OK, I thought so. Here's an excerpt from it:
(emphasis mine)Do you think your web browser should use DirectX 9 pixels shaders to render text, too?
_________________________
Spelling and grammar mistakes left as an exercise for the reader.
But this is different. It's an apples to oranges situation. Essentially what Carmack has said is that when the NV30 runs in ARB2 mode it is doing 32 bit calculations and the ATI is running 24 bit calculations. Bandwidth alone will seriously affect the benchmarks and theoretically the NV30 has a more accurate picture. Now switch the NV30 to the NV30 path which runs at 16 bit and it beats the ATI, but now the ATI probably has better image quality. The problem with this 'neutral' isn't meaningful since both cards run different settings. Now when quoting 3dmark scores for ATI and NVidia we don't know that image quality maybe be lower on one card or that if you were ok with lower quality the lower 3dmark score card is actually faster. In other words no kidding the ATI runs faster when it has less data to shuffle and then the NV30 runs faster when it has less date to shuffle.