New DX10 Benchmarks Do More Bad than Good

NIMBY writes "An interesting editorial over at PC Perspective looks at the changing status between modern game developers and companies like AMD and NVIDIA that depend on their work to show off their products. Recently, both AMD and NVIDIA separately helped in releasing DX10 benchmarks based on upcoming games that show the other hardware vendor in a negative light. But what went on behind the scenes? Can any collaboration these companies use actually be trusted by reviewers and the public to base a purchasing decision on? The author thinks the one source of resolution to this is have honest game developers take a stance for the gamer."

Comments

John Carmack

[dsanfte]

John Carmack used to be pretty good at cutting through the marketing crap and telling it like it was. Let's ask him.

Re:John Carmack

[Applekid]

He seems to be less anti-DirectX these days:

"JC: DX9 has its act together well. I like the version of DirectX on the 360. Microsoft is doing well with DX10 on tightening the specs and the exactness."

Of course, he's still calls it like it is:

"The new features are not exactly well-thought-out. Most developers are pretty happy with DX9. The changes with DX10 aren't as radical. It's not like getting pixel shaders for the first time. Single-pass shaders are nice with DX10, but it's a smaller change. "

Re:DX10

[Anonymous Coward]

Really? Based on what I read, I assumed it was DirectX 10.

separate drivers for high-end gpu?

[CaptainNerdCave]

would it make sense to use different drivers for the top o'the line gpus? instead of putting together a massive pile of mush that everyone "needs", perhaps just make up new sets of drivers for each new card that comes out; thus, these drivers can be specialized and optimized on a per card basis. unfinished drivers would mean unfinished card - problem solved.

i guess this may take away the "poor drivers" back door, eh?

Re:quit already with 'optimized' drivers

[Kamokazi]

Quite hard actually.

What happens when a better way than the "known standard" comes around. Are we supposed to wait for some updated standard then updated hardware for that standard but by then don't you think some part of that standard will be obsolete?

Tweaked drivers, in most cases, only provide marginal benefits that many users would hardly notice. Yes, there are some stark exceptions where a different driver can have substantial impact, but this is often the game developer's fault as much as the hardware developer.

Re:quit already with 'optimized' drivers

[ChronosWS]

This is only a problem if in the course of 'optimizing' for a particular use case they degrade performance in all of the other cases. There may be times, if the case is particularly widely used, that it might even be worth a small perf hit in one area to gain a large benefit in another.

You've got to remember, these guys live and die by sales. They *have* to look good in the numbers because that's what sells their cards at the top end. At the low end, no consumer cares either way as price dominates, but like automobiles, people assume that the tech from the top end trickles down to their lowly mass-market video hardware in some fashion, so it ends up still being relevant, if less directly so.

Also, if you have looked at most of these benchmarks, the difference between best and 2nd best is usually quite small, on the order of a couple percent. The bragging rights of being able to claim you can run your game at 150fps while other plebeans can only run at 140fps is just that - bragging rights. There is no practical effect on game play until framerates drop below 30fps. And the top end graphics hardware these days is not the bottleneck at resolutions of 1280x1024 and below, so really, these guys are chasing numbers in the rarified air of super high resolution monitors and games which use every trick in the book, which is an extremly small set of games actually played.

But that is what sells. And in any case, the competition between ATI and nVidia is good even if those of us who 'know' see their number-chasing as pointless. Let them do their thing, and reward or punish them at the counter as you see fit.

Re:quit already with 'optimized' drivers

[Kamokazi]

Well let's first assume OSX and Windows were on equal footing with PC Games, and upgrading video hardware was common for users of both OS's.

If Apple were to mandate absolute perfection, you'd see a lot fewer driver releases for OSX...because they require more QA time.

So on the other hand, Windows users would be getting better performing drivers more quickly that may have a hitch here and there in select titles, while OSX users would have inferior performance, all because Apple mandates perfection.

Personally I can handle a few bumps, so I'd take the performance any day of the week.

Honestly as soon as I hit a diver issue, I just check for an update and that usually clears it right up. The only time I really have to experiment with different drivers is when I'm beta testing a game.