3D Mark 2003 Sparks Controversy

cribb writes "3DMark 2003 is out, sparking an intense debate on how trustworthy its assessment of current graphics cards is, after some harsh words by nVidia and the reply from Futuremark. THG has an analysis of the current situation definately worth reading. The article exposes some problems with the new GeforceFX previously mentioned in a slashdot article on Doom3 and John Carmack. Alas, here seems to be no end to the troubles with the new nVidia flagship." If you've run the benchmark, post your scores here, and we'll all compare.

Comments

Well...

[Geekenstein]

It stands to reason that a benchmark should fairly and accurately depict the widest range of common capabilities possible to determine a clear winner. Of course, this can be very hard to do. It does seem in this case though that 3DMark got caught up in the whiz-bang marketing side of things by supporting the latest and greatest(?) features and ignoring the very compatibility that would give it any real meaning.

Sorry guys, you goofed.

Re:Well...

[WiPEOUT]

This reminds me of a certain other graphics vendor (now departed), who relied upon developers optimising specifically for their chipset. Then came a new entrant, who provided a chipset that outperformed it when using standard APIs like Direct3D and OpenGL.

It's ironic that I'm referring to 3dFX and the then-incumbent nVIDIA, where now it's nVIDIA expecting developers to optimise for it's cards, while ATI makes sure their card is fast without specific optimisations.

I hope nVIDIA sees the parallels, and wakes up to itself. I'd hate to see the heated competition in the graphics market come to an abrupt end due to nVIDIA's arrogant assumptions on how developers should do their thing sending it under.

Is there such a thing as a dependable benchmark?

[Shayde]

With the level of complexity in current hardware, I can't imagine anyone will come up with a benchmark that -can't- be labelled as skewed, inaccurate, or 'not giving justice'.

If I spend a million dollars developing a cool board that does zillions of sprigmorphs a second (a made up metric), and someone does a benchmark that doesn't test sprigmorph rendering, does that mean my board sucks? No, it just means the benchmark doesn't check it.

However, if Competitor B makes a board that doens't have sprigmorph rendering, but scores higher on this benchmark, which is the 'better card'?

The days of simple benchmarks, alas, are past. It used to be "how many clock cycles a second". Nowadays, whether one piece of hardware is better than another simply comes down to "Can it do what I'm doig right now any faster or cheaper than another unit?"

Silly arguments...

[klocwerk]

Ok slashies.
3DMark 2001 measures performance for directx 7 and 8 hardware platforms.
3DMark 2003 was built from the ground up to measure performance for directx9 platforms, it is not DESIGNED to be a broad range benchmark. it isn't meant to give good scores to your computer that does what you need it to.

It's a high end performance measurement tool, which UNLESS USED IN THE PROPER CONTEXT gives you useless measurements.

Sorry for the pissiness, but jeeze. for geeks who claim to love specialized tools and hate bloat, this is the perfect tool. it does one thing specifically and doesn't throw in the kitchen sink, or support for ancestral hardware.

They aren't microsoft, they're fully supporting 3DMark 2001 for the platforms that it was designed for.

I'll hush now.

Re:Silly arguments...

[l33t-gu3lph1t3]

One of the primary reasons for the criticism of 3DMark2003 is the fact that it *DOESN'T* use DX9 extensively. Pixel shader 1.1 and 1.4 are primarily used, which is absolutely laughable, and only in ONE benchmark are there SOME PS2.0 and VS2.0 paths used. The first test is DX7 for chrissakes...

the point

[cribb]

what bothers me is that the geforceFX, being very slow with unoptimized code, needs code specially rewritten so it works fast. directx was created with the idea that it will be the standard 3D engine, eliminating the need of a each game developing its own.

now nvidia are introducing a new factor in the equation: now you have to write different code for each videocard. just as there used to be 3dfx-only games.

isn't this against the idea of directx? seems very counterproductive to me, and an attempt by nvidia to monopolize the gaming industry.

Re:the point

[olethrosdc]

In a nutshell: You don't need to write different code for different cards. Your program will work everywhere. You might improve the performance if you write special code. But that should be handled by the directx driver, so you would not have to.

Carnack is doing a bit deeper programming than just using the top-level opengl API, he's actually coding shaders and stuff.. I guess in that case you might need to go do vendor-specific stuff. But the top-level API is the top-level API You just use it and it's the same for all cards, the driver inbetween does its job and you dont need to write extra code.

Correct me if I'm wrong.

See if they *learn* from 3dfx

[siskbc]

If nVidia put out the FX ASAP, drop the price on it, take as much of a bath as they have to financially, they might be OK. The longer it takes to get it in the market, the less time until ATI's next one (at which point FX sells for $75). They need to reload for the next one (as you say). Problem is, they can't rush the next one (or delay the FX to slap a new capability on it). That's what 3dfx did, and it kept them behind the curve set by nVidia, and ensured their doom.

nVidia needs to learn that you can stay alive as a company with the #2 video card, as long as you can price it competitively - hell, that's what ATI did for years. But they do need to make sure they eventually get a winner. Since FX obviously ain't it, maybe they can win one next year. And making better decisions is part of it - don't skimp on pixel shaders like 1.4 when the competition will be able to kill you with it.

They definitely need to catch back up to ATI - competition on this front is good for all of us.