Slashdot Mirror
Carmack On ATI's Driver Modifications
CitizenC points out that John Carmack's .plan file has been updated to discuss ATI's driver optimizations. If you weren't paying attention, ATI put code in their drivers to optimize for Quake3, based on the name of the executable - so when running Quake3, you'd get a (good) set of optimizations for the game, but when running the same game after changing the name of the executable, you'd get a default set of optimizations with lesser performance. Some people called this cheating since Quake3 is a typical benchmark application these days.
Full Plan:
Welcome to id Software's Finger Service V1.5!
Name: John Carmack
Email: johnc@idsoftware.com
Description: Programmer
Project:
Last Updated: 11/16/2001 23:22:17 (Central Standard Time)
------
Nov 16, 2001
-----
Driver optimizations have been discussed a lot lately because of the quake3
name checking in ATI's recent drivers, so I am going to lay out my
position on the subject.
There are many driver optimizations that are pure improvements in all cases,
with no negative effects. The difficult decisions come up when it comes to
"trades" of various kinds, where a change will give an increase in
performance, but at a cost.
Relative performance trades. Part of being a driver writer is being able to
say "I don't care if stippled, anti-aliased points with texturing go slow",
and optimizing accordingly. Some hardware features, like caches and
hierarchical buffers, may be advantages on some apps, and disadvantages on
others. Command buffer sizes often tune differently for different
applications.
Quality trades. There is a small amount of wiggle room in the specs for pixel
level variability, and some performance gains can be had by leaning towards
the minimums. Most quality trades would actually be conformance trades,
because the results are not exactly conformant, but they still do "roughly"
the right thing from a visual standpoint. Compressing textures automatically,
avoiding blending of very faint transparent pixels, using a 16 bit depth
buffer, etc. A good application will allow the user to make most of these
choices directly, but there is good call for having driver preference panels
to enable these types of changes on naive applications. Many drivers now
allow you to quality trade in an opposite manner -- slowing application
performance by turning on anti-aliasing or anisotropic texture filtering.
Conformance trades. Most conformance trades that happen with drivers are
unintentional, where the slower, more general fallback case just didn't get
called when it was supposed to, because the driver didn't check for a certain
combination to exit some specially optimized path. However, there are
optimizations that can give performance improvements in ways that make it
impossible to remain conformant. For example, a driver could choose to skip
storing of a color value before it is passed on to the hardware, which would
save a few cycles, but make it impossible to correctly answer
glGetFloatv( GL_CURRENT_COLOR, buffer ).
Normally, driver writers will just pick their priorities and make the trades,
but sometimes there will be a desire to make different trades in different
circumstances, so as to get the best of both worlds.
Explicit application hints are a nice way to offer different performance
characteristics, but that requires cooperation from the application, so it
doesn't help in an ongoing benchmark battle. OpenGL's glHint() call is the
right thought, but not really set up as flexibly as you would like. Explicit
extensions are probably the right way to expose performance trades, but it
isn't clear to me that any conformant trade will be a big enough difference
to add code for.
End-user selectable optimizations. Put a selection option in the driver
properties window to allow the user to choose which application class they
would like to be favored in some way. This has been done many times, and is a
reasonable way to do things. Most users would never touch the setting, so
some applications may be slightly faster or slower than in their "optimal
benchmark mode".
Attempt to guess the application from app names, window strings, etc. Drivers
are sometimes forced to do this to work around bugs in established software,
and occasionally they will try to use this as a cue for certain optimizations.
My positions:
Making any automatic optimization based on a benchmark name is wrong. It
subverts the purpose of benchmarking, which is to gauge how a similar class of
applications will perform on a tested configuration, not just how the single
application chosen as representative performs.
It is never acceptable to have the driver automatically make a conformance
tradeoff, even if they are positive that it won't make any difference. The
reason is that applications evolve, and there is no guarantee that a future
release won't have different assumptions, causing the upgrade to misbehave.
We have seen this in practice with Quake3 and derivatives, where vendors
assumed something about what may or may not be enabled during a compiled
vertex array call. Most of these are just mistakes, or, occasionally,
laziness.
Allowing a driver to present a non-conformant option for the user to select is
an interesting question. I know that as a developer, I would get hate mail
from users when a point release breaks on their whiz-bang optimized driver,
just like I do with overclocked CPUs, and I would get the same "but it works
with everything else!" response when I tell them to put it back to normal. On
the other hand, being able to tweak around with that sort of think is fun for
technically inclined users. I lean towards frowning on it, because it is a
slippery slope from there down in to "cheating drivers" of the see-through-
walls variety.
Quality trades are here to stay, with anti-aliasing, anisotropic texture
filtering, and other options being positive trades that a user can make, and
allowing various texture memory optimizations can be a very nice thing for a
user trying to get some games to work well. However, it is still important
that it start from a completely conformant state by default. This is one area
where application naming can be used reasonably by the driver, to maintain
user selected per-application modifiers.
I'm not fanatical on any of this, because the overriding purpose of software
is to be useful, rather than correct, but the days of game-specific mini-
drivers that can just barely cut it are past, and we should demand more from
the remaining vendors.
Also, excessive optimization is the cause of quite a bit of ill user
experience with computers. Byzantine code paths extract costs as long as they
exist, not just as they are written.
I'm not implying that ATI did it in a selfless manner; enlightened self interest is a good thing. ATI does well in reviews and Quake III players that buy ATI cards get faster operation. Other than the competition, who loses here?
It's not as if ATI contracted with id to make other games slower. They just chose to optimize for the common case. There's a phrase to describe that type of choice: Good engineering.
There's another as well, that I suspect may be a part of the "controversy" here: Good business. And as we all know, business is bad.
<sigh>
Maybe I'm wrong, and folks just haven't taken the time to think about this issue and instead are reacting w/o understanding. Frankly, I'm not sure which thought depresses me more.
The issue wasn't that it made Quake 3 faster and nothing else, the issue was that it made it faster by degrading it's visual quality, and did it without informing the user.
0 4&mode=thread
old article:
ATI Drivers Geared For Quake 3?
Or if you hate clicking and want to cut-n-paste:
http://slashdot.org/article.pl?sid=01/10/24/16432
-- Dan
One small comfort is that no one buys ATI cards for performance. Any cards of the GeForce (Nvidia's brand) variety handily whip anything at the same price point with ATI.
This really isn't true any longer. ATI finally released better drivers for the 8500 this week and it keeps up fine with a GeForce3 Ti 500, for $50 less. Here's a review at Tom's Hardware.
A lot of people still don't realize, what ATI's "optimizations" are about. It's not a optimization specific to Quake 3 which noone could complain about. ATI's drivers forced some features to be disabled massively sacrificing image quality only when Quake 3 is run. It's not clear what they did exactly, but it looks like they were forcing 16-bit textures and/or using lower-resolution mipmaps. See the results here. (The site is in German, but they have some nice details of screenshots, showing the difference onmouseover.)
If you'd read the analysis articles at all, you'd note that the trick ATI is trying to pull is NOT driver (good)optimization for quake 3, but instead is intentionally degrading image quality to improve benchmarks. Quake3 looks like total crap with the ATI drivers, that's why it's fast.
It's essentially of forcing any game with the name quake3 to run at below the minimum detail levels, regardless of what the user has selected, just in order to manipulate benchmarks.
You can debate whether optimizing for a certain game is good or not, but that is a totally different question from what ATI is actually doing, which is intentionally manipulating benchmarks.
I know it's hard to keep track of all the news, but before saying that some driver changes are "good optimizations", you should really check out the facts first. You can look through the comments for this article and see that most of the slashdot readers only read the headlines and initial blurb. Because of this, a lot of people are misinformed about what is really going on.
If you missed the discussion in the first place.
:Our goal for the RADEON 8500 is and always has been to deliver the best possible gaming experience to our customers.
The Register : As we say, if you like the 8500's Quake III frame-rate but aren't willing to put up with the dip in image quality, buy a different card. Or wait for ATI to change its drivers, which, we understand, it's in the process of doing.
HardOCP was the first to publish about this: The Facts As We See Them: It certainly seems to us here at [H]ardOCP that ATi has in fact included application specific instructions in their version 5.13.01.3276 Win2K drivers that make Quake 3 arena benchmarks faster by up to over 15%. Either way, the driver optimisations for Quake III are just one of the (many) factors that differentiate different vendors' products. ®
firingsquad show show some details how the quack.exe is made and concludes:
To some of us, it seems like the evidence points towards intentionally deceptive code designed not only to inflate benchmark scores, but also to keep anyone from finding out. To others, this is nothing more than an overreation to a perfectly legitimate game optimization. In our eyes, anyone who vehemently peddles either of these explanations is either naive or pushing an agenda of their own.
there later in Q&A with ati explains in 2 pages that
yeah right.!
Anand did a new look at the 8500 with the new ATI drivers. He claims that the image quality is now identical to the Geforce3 and that the driver optimizations work for all Quake3 engine based games. Anandtech article
Well, actually, while ATI does not actively support Linux themselves, they do release more information about their chips than nVidia does. Because of this, third party driver developers can develop open-source, cross-platform drivers, which with nVidia, you are stuck with Linux on x86. So while nVidia invests more development time, ATI gives more information and therefore has more well-developed drivers on more platforms. Their linux driver implementation may not be as good as nVidias, but will be around even if ATI went out. If nVidia goes, and all we have are the binary drivers compiled for Kernel 2.4.x and XFree 4.1.x, then nVidia owners are in a bit of a fix....
XML is like violence. If it doesn't solve the problem, use more.