Slashdot Mirror
Carmack On ATI's Driver Modifications
CitizenC points out that John Carmack's .plan file has been updated to discuss ATI's driver optimizations. If you weren't paying attention, ATI put code in their drivers to optimize for Quake3, based on the name of the executable - so when running Quake3, you'd get a (good) set of optimizations for the game, but when running the same game after changing the name of the executable, you'd get a default set of optimizations with lesser performance. Some people called this cheating since Quake3 is a typical benchmark application these days.
The solution is simple: when benchmarking, don't use the executable by the default name. No reason to panick about this "cheating".
Man is born free; and everywhere he is in chains.
It's an excuse for their normally sub-par driver quality, they can't get their generic all purpose drivers to deliver the goods so they instead do what they have done. They'll never live down the driver thing until they get their act together and just build some quality drivers that fully exploit their hardware.
Basically, here's the gist of what Carmack said:
Uhh, it's bad, but don't quote me on that.
-- Dan
"Making any automatic optimization based on a benchmark name is wrong. It subverts the purpose of benchmarking, which is to gauge how a similar class of applications will perform on a tested configuration, not just how the single application chosen as representative performs.
It is never acceptable to have the driver automatically make a conformance tradeoff, even if they are positive that it won't make any difference. The reason is that applications evolve, and there is no guarantee that a future release won't have different assumptions, causing the upgrade to misbehave. We have seen this in practice with Quake3 and derivatives, where vendors assumed something about what may or may not be enabled during a compiled vertex array call. Most of these are just mistakes, or, occasionally, laziness."
Carmack seems pretty well decided on this one, and not in favor of it. He *does* show a bit of support for having super-ultra-tweak control panels on driver config screens, but that's (almost) an entirely different matter.
It seems pretty cut and dried, at least from his perspective. I for one have got to agree with this viewpoint. Of course, to anyone who has said or will say that you should just rename the executable, I completely agree that this is the only real way of getting an objective test out of it.
Card manufacturers who do this sort of thing *will* get egg on their faces when they start hearing all about the crappy performance of their cards after a new game version comes out with different thinking on what's important visually. Unfortunately, as John points out, a lot of the flack will end up on the developers' doorsteps (misplaced, but a lot of gamers won't know that).
This created a big stink at the time.
Of course, there is the question of how much of that sort of thing was really hardware dependant, vs a minor feature update that justified another round of hardware upgrades.
And so the tactics of the marketing monster enter the picture once again. What can you do, what can you get away with, and how much does it cost ...
"It is a greater offense to steal men's labor, than their clothes"
The driver is just software. When Quake3 asks it to do something for the first time, it simply checks the filename attached to the process handle and sets a flag accordingly.
If a job's not worth doing, it's not worth doing right.
It's not surprising to see ATI do something like this. In the business world, this kind of thing may pass off as slightly unethical, but for the target market of graphics card vendors, this is just plain #$*&'ed up. It's quite true that Quake 3 Arena is a very standard benchmarking application nowadays, so any performance gain in the driver for that specific application that does not apply to all programs in general will mislead the consumer in making a purchasing decision. An analogy: no one would appreciate it if a company said their car goes from 0 to 60 in 3.2 seconds and in reality it takes 3.2 seconds just to make it up to 20.
Carmack's right, this whole driver situation in general is a slippery slope. One of the biggest hassles of his job since Quake (#1) came out has been to get all the graphics card companies to play nice and write good drivers. This has not happened for one second. Some companies are better than others (Nvidia is an example of one of the better companies in the field all around), and the situation right now is FAR better than it was. But, between the race for speed, trying to keep up with the most popular applications, and having to support various APIs being pushed by different 500 lb gorillas in the field... it's a mess. ATI was ALWAYS a mess with their drivers, so this doesn't really surprise me. But this isn't right, cause it'll lead to a day when nothing but the lastest Quake game works on new graphics cards. I mean, nothing works AT ALL. And if it's easy enough to get to that situation, I can just see bribes and payoffs... ahem, "strategic partnerships" being made to accomodate game vendors who want to publish a working 3D game at some point in time...
But anyway...
One small comfort is that no one buys ATI cards for performance. Any cards of the GeForce (Nvidia's brand) variety handily whip anything at the same price point with ATI. The big thing ATI has is OEM agreements, and they also sell some really exotic TV/Video Capture/MPEG recording cards that are really snazzy sometimes. I know cause I'm using one of em right now. But their drivers suck in a lot of ways, they were never the fastest, and I'd love to see them stick to a product release schedule EVER. This is the kind of company that gets wiped out when someone new on the scene releases something better/faster/cheaper.
Unfortunately, the last time we saw better/faster/cheaper in the graphics industry was five years ago. Nowadays, you usually get one of those improvements in a new release video card... never all three.
I'm not implying that ATI did it in a selfless manner; enlightened self interest is a good thing. ATI does well in reviews and Quake III players that buy ATI cards get faster operation. Other than the competition, who loses here?
It's not as if ATI contracted with id to make other games slower. They just chose to optimize for the common case. There's a phrase to describe that type of choice: Good engineering.
There's another as well, that I suspect may be a part of the "controversy" here: Good business. And as we all know, business is bad.
<sigh>
Maybe I'm wrong, and folks just haven't taken the time to think about this issue and instead are reacting w/o understanding. Frankly, I'm not sure which thought depresses me more.
So I need to rename all my games, Simcity, Theme Hospital, UT, to quake3.exe to get a optimized version ? :o)
cd \sc2000
ren simcity.exe quake3.exe
cd \hospital
ren theme.exe quake3.exe
cd \ut
ren unreal.exe quake3.exe
...
(already can see it happen!)
--- I am known for the ones who want to find me on the net. Is that a privacy risk or a privilege? One might wonder..
The issue wasn't that it made Quake 3 faster and nothing else, the issue was that it made it faster by degrading it's visual quality, and did it without informing the user.
0 4&mode=thread
old article:
ATI Drivers Geared For Quake 3?
Or if you hate clicking and want to cut-n-paste:
http://slashdot.org/article.pl?sid=01/10/24/16432
-- Dan
ATI put code in their drivers to optimize for Quake3, based on the name of the executable
That's not how I'd describe it. As I understand it, they made what Carmack called a "conformance tradeoff", arbitrarily reducing image quality to increase speed. That's not a Quake3-specific optimization, that's a Quake3-specific dirty, sneaking, benchmark-busting hack.
Note to ACs: I usually delete AC replies without reading them. If you want to talk to me, log in.
To all you OSS zealots out there, *this* is why NVIDIA's drivers are closed source. You can bet ATI would love to put this whole driver fiasco behind them and just steal the high level OpenGL code (an OpenGL driver has to implement the whole GL API, not just hardware interfacing) from NVIDIA's ICD.
A deep unwavering belief is a sure sign you're missing something...
There are unknowns like perhaps these optomizations impact overall performance? So they are selective? What would you rather have? Faster Quake performance at the expense of over all performance or better Quake performance with the best overall performance you can get?
This is just nitpicking to try and make news. Everyone knows the HW makers tune their systems around benchmarks - hell even the CPU makers try to do it I'm sure.
Top Most Bizarre/Disturbing Error Messages
I'm sure those people who play Quake 3 will be VERY unhappy about getting better performance. :-)~
Why would they need the high-level OpenGL code (Besides, here's a clue for you- they probably DO happen to have the same high-level code, licensed from SGI...) when it's the innards of the driver, working with the low-level pieces that makes the card fast or slow? Since the low-level pieces are not going to be the same (nor, would they likely ever be so) the low-level stuff would be of little help for either company.
I am not merely a "consumer" or a "taxpayer". I am a Citizen of the State of Texas
A lot of people still don't realize, what ATI's "optimizations" are about. It's not a optimization specific to Quake 3 which noone could complain about. ATI's drivers forced some features to be disabled massively sacrificing image quality only when Quake 3 is run. It's not clear what they did exactly, but it looks like they were forcing 16-bit textures and/or using lower-resolution mipmaps. See the results here. (The site is in German, but they have some nice details of screenshots, showing the difference onmouseover.)
Back in the day, there were cards optimized to speed up ONE application. Radius (before it became just another name on cheap monitors) used to make cards that were specfically optimized to speed up photoshop filters. And they sold them in droves for $500+ Others did the same thing. The difference here is that Radius TOLD you beforehand that the card was basically designed to give you 24 bit graphics (when most computers were 8 bit) and to speed up photoshop.
Evidently ATI never told anyone they were buying a "Quake Accelerator". And therein lies the problem.
If you'd read the analysis articles at all, you'd note that the trick ATI is trying to pull is NOT driver (good)optimization for quake 3, but instead is intentionally degrading image quality to improve benchmarks. Quake3 looks like total crap with the ATI drivers, that's why it's fast.
It's essentially of forcing any game with the name quake3 to run at below the minimum detail levels, regardless of what the user has selected, just in order to manipulate benchmarks.
You can debate whether optimizing for a certain game is good or not, but that is a totally different question from what ATI is actually doing, which is intentionally manipulating benchmarks.
I know it's hard to keep track of all the news, but before saying that some driver changes are "good optimizations", you should really check out the facts first. You can look through the comments for this article and see that most of the slashdot readers only read the headlines and initial blurb. Because of this, a lot of people are misinformed about what is really going on.
They no longer have trade off optimisations (sacrificing quality for more performance) for quake.exe.
Instead they now have genuine optimisations for the quake engine, so any program (no matter what the executable is called) that uses the quake engines gets genuine optimisations.
I read a couple of reviews of the latest drivers, & they said that the Radeons (using the new drivers) do perform better compared with GeForce 3, with all (tested) quake engine games, relative to how the radeon compares with the GeForce 3 as far as non quake engine games are concerned.
Plus there's no image compromise this time.
If you missed the discussion in the first place.
:Our goal for the RADEON 8500 is and always has been to deliver the best possible gaming experience to our customers.
The Register : As we say, if you like the 8500's Quake III frame-rate but aren't willing to put up with the dip in image quality, buy a different card. Or wait for ATI to change its drivers, which, we understand, it's in the process of doing.
HardOCP was the first to publish about this: The Facts As We See Them: It certainly seems to us here at [H]ardOCP that ATi has in fact included application specific instructions in their version 5.13.01.3276 Win2K drivers that make Quake 3 arena benchmarks faster by up to over 15%. Either way, the driver optimisations for Quake III are just one of the (many) factors that differentiate different vendors' products. ®
firingsquad show show some details how the quack.exe is made and concludes:
To some of us, it seems like the evidence points towards intentionally deceptive code designed not only to inflate benchmark scores, but also to keep anyone from finding out. To others, this is nothing more than an overreation to a perfectly legitimate game optimization. In our eyes, anyone who vehemently peddles either of these explanations is either naive or pushing an agenda of their own.
there later in Q&A with ati explains in 2 pages that
yeah right.!
If they would have DOCUMENTED the changes, and put a button in the control pannel (like where all the antialiasing on off, buffer level and all is) it would have been legit, they could have put it in their press release and send them around WITHOUT getting any flames.
There are ways to do things, you can bend the rules, but breaking it will only get fire back. I'm all for specific-engine optimization, and like carmack says, Conformance trade-offs shouldn't be made at the driver level in general, but to this I'll say, if it's documented and available as an OPTION somewhere, it won't apply only to techies, it can be applied to joe schmoe as well "if you click here, game will run faster with a small quality degradation (that you will probably not even notice because at the speed of events in quake 3)". Heck if they can put something like Quancun Aliasing in the Panel options, you can be sure people will understand specific game optimization as well.
Anyways forcing something to the users without their knowledge is just plain bad and lame. Of course people will end up finding out, and of course it will backfire... what is gain from doing that!? there sure is poor judgement at ATI, and there's probably a bunch of people there right now saying "see? told you so!". Management, you should listen to the people in the lower part of your food chain (i.e. QA testers), they might know more about the end-user market and hardware benchmarkarking scene (not the OEM of course) than you do!).
--- Metamoderating abusive downgraders since my 300th post.
Anand did a new look at the 8500 with the new ATI drivers. He claims that the image quality is now identical to the Geforce3 and that the driver optimizations work for all Quake3 engine based games. Anandtech article
Carmack makes a good case for why this would be reasonable to do in certain cases.
ATI says that they optomise their driver for some other popular games. I would be interested to know which other games. I doubt they could list them all for pr reasons but it would be cool if they would just list 2 or 3 other games that they optomise for.
1) It is a web site, hence the URL in the story worked.
.plan file they way they are supposed to.
2) I think it's kind of cool that somebody is still using their
It's the magazine shootouts that are flawed, and you if you buy video cards based on magazine shootouts that only involve Quake3.
Is that all you play?
So why is that all you care about?
STOP . AMERICA . NOW
While many companies focus upon optimising for benchmarks, most simply optimise the driver paths for the specific cases that those benchmarks use. Any other app that uses similar settings can gain performance from that work. But ATI have made their optimisations dependant upon the name of the app, so no other apps can benefit from their work. While a Quake3 player might not mind, Q3 isn't as widely played today as it once was, and that same player might be less pleased when the card fails to perform to the same standard on any other of their games.
But what makes this particular "optimisation" underhanded is that it's not better or more tuned code, it trades off quality. Have a look here to see the mess that it makes of textures!
Now, if players wanted to see blurry textures in exchange for more performance, they'd simply lower the texture quality slider in the Quake3 game. ATI's drivers do this for them; they're forcing the mipmaps two levels down - a 16x reduction in texture detail - to get the extra speed. And this isn't optional. You can't turn it off, short of using a hex editor.
Particularly, a reviewer running the standard Quake3 High Quality benchmark will never notice the difference (as the frames run by way too fast). There's no extra sliders in the driver, no other indications or switches, and of course no notification in the driver documentation. All the reviewer sees is higher framerates, because the drivers are, quite literally, cheating. They're giving low-texture numbers on the high-texture setting.
I for one applaud ATI's renewed efforts to improve their drivers (and I still plan to buy an 8500DV alongside my existing QuadroDCC), but I feel ATI really have attempted to subvert the benchmark process, and shot themselves in the foot. The strong implication is that, even though their hardware is fine, they don't feel they can compete with nVidia's driver team so they have to resort to methods like these.
Very unwise - they've lost a lot of the support they had as the underdog to nVidia, through these tactics.
Why would anyone engrave "Elbereth"?
Bad idea. Too much of that already. The release notes for NVidia's drivers now require a 6.62 megabyte PDF file. 154 pages of manual. For a driver. And that's a document for end users, not programmers.
This guy has OBVIOUSLY not looked at comparision screenshots, which will blow your mind. Quake3 with ATI has enourmously lower quality than it should, they didn't practice 'Good Engineering' or whatever this idiot is blabing about. They cheated and got caught.
That this is modded as 'informative' is rediculous, this guy is barely informed
We can all agree that nVidia has the best drivers in the industry. ATI is #2.
:-)
Sometimes when you are #2 you try harder. ATI, being behind on drivers, has been much more open about their cards' architectures, and so Xfree86 has fully open 3D drivers for Radeon but not for nVidia.
Even under Linux, the nVidia drivers perform better than the ATI drivers. But there is a chance that the open source community will improve the ATI drivers. It may take a long time -- Mozilla took years before it became really good -- but at least under Xfree86, ATI may catch up or even pull ahead.
If I were ATI, I would be paying money to one or more Xfree86 developers. Not only would that mean one developer would be working to improve the drivers, but that developer would also be collecting patches and integrating them. Long-term it could pay off very well. (The free software community will improve the ATI drivers no matter what, but it would go faster if they paid one or more people to work on the drivers full-time.)
In fact, if I were ATI I think I would release the Windows drivers as open source! It's not as if they need to worry about nVidia stealing their code.
P.S. If you feel strongly about free software, then a Radeon is the card for you. With a decent computer you get enough performance to play games. (Tux Racer runs great.) Maybe ATI should get Richard Stallman to endorse their 3D cards?
steveha
lf(1): it's like ls(1) but sorts filenames by extension, tersely
There wouldn't be consumer level vector graphic cards. You just don't get the economies of scale to be able to produce a 100 dollar 3D card.
Games are the reason that you're not spending 10's/100's of thousands of dollars on SGI equipment.
Trolls throughout history:
Jonathan Swift
Finger is a very bad thing from a security perspective. Pretty stupid to be allowing it on the public internet these days.
...what ATI would say if the next ID title was designed to run slower on an ATI card?
I get the feeling they wouldn't like it much, but I'd love to see it happen.
They could alway patch it, so you suckers that continue to buy ATI products shouldn't whine to loudly.
The Kruger Dunning explains most post on
The real problem here is that so many of the cry babies in the world of 3D benchmarking want to benchmark xyz 3D app by proxy, accepting the Quake3 benchmark as the golden rule. Obviously, this is their own folly, and they really ought to accept responsbility for THAT rather than blaming ATI for optimizing performance for what is arguably the most popular 3D app on the market. Reading through Carmack's diatribe on the topic, I hear him saying that different apps and even different versions of the same app will perform differently. To me, this strongly suggests each pertinent application needs to be benchmarked individually, rather than by proxy. Well, if one really wants to know what is going on, that is.
The problem is accepting the Q3 benchmark alone. Savvy buyers should also take other raw benchmarks into consideration before plunking down their cash.
cat
Well, actually, while ATI does not actively support Linux themselves, they do release more information about their chips than nVidia does. Because of this, third party driver developers can develop open-source, cross-platform drivers, which with nVidia, you are stuck with Linux on x86. So while nVidia invests more development time, ATI gives more information and therefore has more well-developed drivers on more platforms. Their linux driver implementation may not be as good as nVidias, but will be around even if ATI went out. If nVidia goes, and all we have are the binary drivers compiled for Kernel 2.4.x and XFree 4.1.x, then nVidia owners are in a bit of a fix....
XML is like violence. If it doesn't solve the problem, use more.
Let's put it this way. Many companies cheat when it comes to "demoing" their products. And benchmarks are, for pretty much all intents and purposes, a demo of the product being benchmarked. So they're cheating. OH WELL.
ATI's drivers are OSS (on Linux anyway) to begin with.
A deep unwavering belief is a sure sign you're missing something...
Telnet is far from obselete. I use it very frequently with my Netgear RT314 router. It's a far better interface than the web interface it supports. In fact it bugs me that they've even felt the need to have a web interface! It's bloated, it's slow, I have to wait for the web browser to load, and some functionality just isn't suited to that interface.