Slashdot Mirror
3D Mark 2003 Sparks Controversy
cribb writes "3DMark 2003 is out, sparking an intense debate on how trustworthy its assessment of current graphics cards is, after some harsh words by nVidia and the reply from Futuremark. THG has an analysis of the current situation definately worth reading. The article exposes some problems with the new GeforceFX previously mentioned in a slashdot article on Doom3 and John Carmack. Alas, here seems to be no end to the troubles with the new nVidia flagship." If you've run the benchmark, post your scores here, and we'll all compare.
Now, it's the video card makers slagging the benchmark makers.
Anybody remember the early 90s (93?) when Hercules got itself into hot water by hard-coding a super-fast result for the PC Magazine video benchmark? Whoo hoo, that made for some good press. Got their awards pulled and everything.
Do daemons dream of electric sleep()?
NVidia missed a manufacturing cycle and now it's coming back to haunt them. They really need to drop the FX and concentrate on whatever new architecture is currently being tossed around in R&D.
Originally I was planning to buy the successor to the NV30 for a great experience in Quake III and better framerates in older games. But now it looks like I'll be laying out the dough on whatever ATI brings out early next year.
Benchmarks are generally too isolated to be of much use. They might be okay for getting a rough picture, but a high scoring 3d benchmark might not directly translate into good 3d performance.
Even so-called 'real world' benchmarks that test stuff like file opening and scrolling documents don't really get into the meat of the everyday user experience.
Using benchmarks to decide what computer to buy is like macking on the girl with the big boobs. She might look nice, but she could be horrible in bed. Also she might have crabs.
(-1, Raw and Uncut is the only way to read)
Am I the only one who saw "THG" in the post and thought, "The Humble Guys? They're still around? And they care about graphics??"
I had to mouseover to realize that they meant Tom's Hardware Guide and not "The Humble Guys" of 1980s BBS piracy. Hrm, I guess I'm showing my age.
Heh, for a trip down memory lane, check this out:
http://www.textfiles.com/piracy/HUMBLE/
Using your sig line to advertise for friends is lame.
Personally, I think that a good benchmark is just doing whatever you are going to be doing and timing that.
Are you going to be playing much of the 3D-Mark benchmark ? If the answer is yes, then you should use it, otherwise it's pure masterbation. Their site claims that the purpose of the benchmark is to give you an idea of what a typical DX7-DX9 game will give you in performance. However, the 'games' they use to test it are not games you can actually play. It's basically a graphics demo. Wow.
The only benchmarks even worth considering are the Quake, Unreal, etc. benchmarks that test real games being played. And even those results should be taken with a grain of salt. They are 'real world' results, but you have to take into account many factors to actually derive useful information from them. Such as RAM, CPU, resolution that marks were run at, etc.
If you are smart, then you will buy your graphic card from a place like Fry's that will let you return it if the performance is unsatisfactory. In this day and age where the graphics card costs more then a computer, you had better get your money's worth.
Just test SimCity 4. It kicks the snot out of my P4-2.26/1GB DDR/4200.
This is not the greatest sig in the world, this is just a tribute.
Sometimes people scratch their heads about benchmarks and wonder "how did they come up with that number?" If the benchmark itself was Open Source you'd have at least a partial answer. Not to mention you'd have the eyes of many people looking over the code to make sure it was executing draws in the right and consistent manner.
So why aren't benchmarks open? What do the makers of benchmarks have to hide? Are they under NDAs from the card vendors?
Actually, this is NOT a very valid DirectX 9 test at all. Only one benchmark even tests DirectX 9 features, the Mother Nature game, and even then, it is only a partial DirectX 9 test, not a full one. The first three game tests are DirectX 7 and DirectX 8 benchmarks.
I think there are several uses for a benchmark. One is to measure compatibility with the features offered by today's game engines and gaming API's (OpenGL, DirectX). The second is to measure real-world performance for current gaming titles and technologies. I think 3DMark `03 looks nice, is perhaps a partial measure of current featuresets at best, but is not a good measure of real-world performance at all.
Never look down your nose at others. Someday, someone is bound to see your boogers.
Carmack says:
It seems that the NV30 architecture requires a good deal more optimization to run shader code optimally (read: fast), while R300 deals with standard code much better. This would explain why NVIDIA is so harsh and aggressive in its criticism of the new 3DMark 2003, since the GeForce FX (NV30) seems to have a problem with non-optimized shader code, a trait that its mainstream siblings NV31 and NV34 will obviously share. If word got around - and in this community, it does - this could seriously hurt NVIDIA's sales.
To be fair, in real games this "handicap" will most likely not be nearly as pronounced as in the 3DMark test. After all, NVIDIA is very good at convincing game developers to optimize and adapt their code for their hardware.
So NVidia only runs well with optimized code huh? That's going to be a problem for them I think. It means we won't know how well it works until we get some games to benchmark it with. Sure, we could benchmark it with UT2003 or something; but that doesn't mean much. I don't care about UT2003. My current card runs that fine. I (and other people who buy these cards) care about how they will run the next gen games. We could wait until those games come out, but a lot of people don't have that patience. For those people it might be safer to get the ATI. If you go with NVidia you have to really trust that the games you want are going to be well optimized for it, though as Carmack said, they probably will be. Personally I'm still on the fence about which card I will eventually get.
Aw crap, ninjas!
Looks like we'll have to wait and see.
Comment forecast: Bits of genius surrounded by a sea of mediocrity.
Your opinion on smearing the motives of the benchtest are noted, but the test merely tests 1.4 more agressively in DirectX without cheating.
Did you read the articles? I have. The benchmark is valid and works well.
The updated benchtest merely uses PS 1.4 in some tests and ATI uses 24 bit floats and Nvidia can only choose between 16 bit or 32 and in 32 it suffers 33% slowdown over ATI cards.
NVidia Fanboys HATE this list of games that use DirectX 8.1 (and 9.0 mandatory) Pixel shader 1.4
UT2003
Madden 2003
Tiger Woods 2003
Nascar 2003
NeverWinter Nights (actually OGL equivalent).
DOOM3 will have a path that will use the equivalent of PS1.4 as well
It makes them WRONG when they pretend that PS 1.4 is NOT FOUND in modern or upcomming games yet!
That list is factual and not even ehaustive.
Buy ATI and get real 1.4 speed in the games that ARE and WILL use it!
Or wait 9 months for NVidia to catch up.
NVidia's poor performace have anything to do with the recently revealed fact that it does NOT have 8 rendering pipelines as it advertised,but only 4?
Read about it here. http://www.theinquirer.net/?article=7920
"An Nvidia technical marketing manager confirmed to us that Geforce FX has 4 Pipelines and 2 Texture Memory Units that can results with 8
textures per clock but only in multitexturing.
However, Nvidia did say that there were some cases where its chip can turn out 8 pixels per clock. Here is a quote:
"GeForce FX 5800 and 5800 Ultra run at 8 pixels per clock for all of the following: a) z-rendering b) stencil operations c) texture operations d)
shader operations"
and
"Only color+Z rendering is done at 4 pixels per clock"
Nearly ALL games use Color + Z rendering in 2002,2003 so except for CAD and Architectural walkthroughs, NVIDIA runs half as fast as ATI Radeon 9700
That is another reason why Apple promotes ATI
The 3dfx 3dmark2000 issue was completely diffrent. At the time time most games did have support for 32 bit color(and could be impelemented with little to no performace hit on nvidia cards no less), and the fact that 3dfx lacked it was a major disatvantage in 3dfx cards and they deserved to be docked for it. This current situation isn't about the geforce4 at all. Nvidia dosen't care that it performs badly in it, as it is supposedly a directx 9 benchmark, and the geforce 4 is not a directx 9 part. What nivida is upset about is that their brand new part, the FX(a DX9 part) only perfoms to parity against the 9700 pro which is 6 month old card. So the real question is,does nvidia have a point? IMO, not really. while I agreem that 3dmark 2003 has some strange rendering techniques, and that they are probably biased towards the 9700, it wouldn't of happened if nvidia haden't dropped it's 3dmark subscrition, which they did becuase no matter what, they couldn't make the FX beat the 9700 by any particularly great margin. The only way to save face was to drop the subscription and cry foul. Does that make 3dmark2003 any more legit? Nope. But it does explain why nvidia is so pissed off. Their new part just isn't up to snuff.
There's a substantial thread on Ars Technica's forums that contains a ton of benchmark results. What it boils down to is that if you have a decent processor (Athlon XP 1600+ or better) and an NVidia GF4 Ti4600, you'll end up with something like 1500-1700 3DMarks. If you pull the GF4 out and slap in a Radeon 9700 Pro (and get the appropriate drivers installed, of course), your score would shoot up to over 4000 3DMarks.
I've got a Ti4600, and 3DMark 2003 runs like ass. Fortunately, Splinter Cell plays just fine, so I'll ignore the benchmark and get on with actually using the computer.
This is what is scaring me about Nvidia, Espicially since just about everything Nvidia makes is running my PC right now and it would be a cold day in hell before an ATI product touches my PC.
From what I've seen so far, Nvidia is doing the exact same thing that 3dfx Did when the voodoo3 came out, and whats more disturbing is that the're following the 3dfx downward spiral so close that you could praticially mirror the two, a sort of NVfx if you will.
Making Video cards, Pushing their Rendering Format harder than ever, Bashing Benchmarks, claiming that their hardware is limited for a reason, ETC. All of this failed miserably when 3dfx did it, and it's going to fail for Nvidia as well.
It wouldn't suprise me to see a dual GeforceFX board this year, or even a quad version. It's what 3dfx did before they went under.
In Soviet Russia, Trojan exploits YOU!
All the more interesting, since by everything I'VE heard, the FX was designed by engineers formerly from 3DFX. It sure shows in the design-- it has all the hallmarks of 3DFX bad design. High heat, high power consumption, brute force design, and a huge card.
At this rate, unless Nvidia gets up off their collective asses and designs a card that can actually show superiority, they're going to lose. Okay, fine, the FX is a bit faster than the 9700 Pro.. but I see that the 9700 Pro hasn't been pushed to ITS limits yet. I expect a newer revision of the Pro with a higher clockrate, and I really expect to see it blow away the FX.
How can I be so sure of this? Because there are already pre-overclocked 9700 Pro cards on the market (at about the same price as the FX) that blow away the FX.
I'm no ATi fanboy, but this doesn't look good for Nvidia.
When AMD's K6-2 processors were getting stomped by the Pentium II, it turned to 3DNow, leaning heavily on 3DNow-optimized Voodoo2 drivers and a 3DNow-optimized version of Quake 2. Anand's Monster 3D-2 review shows 3DNow improving a last-place 44 FPS to a competitive 76 FPS. Quake 2 played better because of the efforts of AMD and 3dfx. However, the results weren't representative, as the Turok and Forsaken benchmarks show.
I played System Shock 2 on a Voodoo3. At the time, 3dfx had Quake 3 on the brain, struggling to tweak its drivers to keep up with the GeForce. Those efforts were small consolation to me, as each new driver release would break something in System Shock, like making the weapon model sporadically disappear.
The problem with a marquis game like Quake is that it encourages short cuts. The testing is done when Quake runs (a little faster). I, for one, am glad that Quake 3 put an end to the miniGL nonsense. Give me a card with decent, reliable performance in standard APIs like OpenGL and Direct3D. Put it this way: would you buy a TV that was optimized for Friends?
3dfx started losing around the time the tnt2 came out. Or maybe a little before that. Why? Quake.
Quake3 used full on 24bit color and big (256x256) textures. The 3dfx cards, including the voodoo3 and banshee, couldn't handle these larger textures OR the higher bit depth rendering. Nvidia beat them on features, and later framerates.
I traded my Banshee card for a TNT card back in the days when Quake3 first came out. After my friend saw Quake3 on the TNT card he wanted to trade back. Yes, it WAS that much of a difference.
These days, it's more about drivers and price/performance. Visual quality is pretty much the same to the human eye on all the high end cards, and framerates like 90fps+ are about all you can see. Anything more than that is just extra horsepower for higher details or the latest and greatest game.
Nvidia won't end up like 3dfx at this point in the game. It's just the two headed monster of ATI and NVIDIA, each with their own fans and pluses/minuses.