Carmack On ATI's Driver Modifications

CitizenC points out that John Carmack's .plan file has been updated to discuss ATI's driver optimizations. If you weren't paying attention, ATI put code in their drivers to optimize for Quake3, based on the name of the executable - so when running Quake3, you'd get a (good) set of optimizations for the game, but when running the same game after changing the name of the executable, you'd get a default set of optimizations with lesser performance. Some people called this cheating since Quake3 is a typical benchmark application these days.

Comments

Carmack's Conclusion

[mESSDan]

Basically, here's the gist of what Carmack said:

Uhh, it's bad, but don't quote me on that.

Poor behavior in a rough industry

[brianvan]

It's not surprising to see ATI do something like this. In the business world, this kind of thing may pass off as slightly unethical, but for the target market of graphics card vendors, this is just plain #$*&'ed up. It's quite true that Quake 3 Arena is a very standard benchmarking application nowadays, so any performance gain in the driver for that specific application that does not apply to all programs in general will mislead the consumer in making a purchasing decision. An analogy: no one would appreciate it if a company said their car goes from 0 to 60 in 3.2 seconds and in reality it takes 3.2 seconds just to make it up to 20.

Carmack's right, this whole driver situation in general is a slippery slope. One of the biggest hassles of his job since Quake (#1) came out has been to get all the graphics card companies to play nice and write good drivers. This has not happened for one second. Some companies are better than others (Nvidia is an example of one of the better companies in the field all around), and the situation right now is FAR better than it was. But, between the race for speed, trying to keep up with the most popular applications, and having to support various APIs being pushed by different 500 lb gorillas in the field... it's a mess. ATI was ALWAYS a mess with their drivers, so this doesn't really surprise me. But this isn't right, cause it'll lead to a day when nothing but the lastest Quake game works on new graphics cards. I mean, nothing works AT ALL. And if it's easy enough to get to that situation, I can just see bribes and payoffs... ahem, "strategic partnerships" being made to accomodate game vendors who want to publish a working 3D game at some point in time...

But anyway...

One small comfort is that no one buys ATI cards for performance. Any cards of the GeForce (Nvidia's brand) variety handily whip anything at the same price point with ATI. The big thing ATI has is OEM agreements, and they also sell some really exotic TV/Video Capture/MPEG recording cards that are really snazzy sometimes. I know cause I'm using one of em right now. But their drivers suck in a lot of ways, they were never the fastest, and I'd love to see them stick to a product release schedule EVER. This is the kind of company that gets wiped out when someone new on the scene releases something better/faster/cheaper.

Unfortunately, the last time we saw better/faster/cheaper in the graphics industry was five years ago. Nowadays, you usually get one of those improvements in a new release video card... never all three.

Re:Tell me again: How is this bad?

[Mark J Tilford]

What they did was not "Have Quake run in mode which is faster" but "Have Quake run in mode which has lower quality than it should so it will run faster"

just rename everything ...

[freaker_TuC]

So I need to rename all my games, Simcity, Theme Hospital, UT, to quake3.exe to get a optimized version ? :o)

cd \sc2000
ren simcity.exe quake3.exe
cd \hospital
ren theme.exe quake3.exe
cd \ut
ren unreal.exe quake3.exe
...

(already can see it happen!)

Re:Tell me again: How is this bad?

[Penrif]

Let me get this straight. ATI takes the time to create optimizations in their drivers that make Quake III run faster, and people are unhappy about it?

You're damn right I am. As someone who uses graphics hardware for scientific visualization, any time hardware manufactures spend on making games faster is a complete waste. With the latest cards and drivers from ATI, we've seen this problem en masse. Games run great, but code that we write (that's general OpenGL and works great on GeForce cards) runs slower than hell on the 8500.

So, if all ATI wants to do is sell cards to gamers (and that's certainly a good market), they can go right on ahead and do that. Nvidia takes more time with their drivers and makes them much more optimized for the general case...so they've got my buisness for now.

Just my thoughts. I'm going to skip the part where I tell everyone else that they're being knee-jerk.

LINKS.

[leuk_he]

If you missed the discussion in the first place.

The Register : As we say, if you like the 8500's Quake III frame-rate but aren't willing to put up with the dip in image quality, buy a different card. Or wait for ATI to change its drivers, which, we understand, it's in the process of doing.

HardOCP was the first to publish about this: The Facts As We See Them: It certainly seems to us here at [H]ardOCP that ATi has in fact included application specific instructions in their version 5.13.01.3276 Win2K drivers that make Quake 3 arena benchmarks faster by up to over 15%. Either way, the driver optimisations for Quake III are just one of the (many) factors that differentiate different vendors' products. ®

firingsquad show show some details how the quack.exe is made and concludes:
To some of us, it seems like the evidence points towards intentionally deceptive code designed not only to inflate benchmark scores, but also to keep anyone from finding out. To others, this is nothing more than an overreation to a perfectly legitimate game optimization. In our eyes, anyone who vehemently peddles either of these explanations is either naive or pushing an agenda of their own.

there later in Q&A with ati explains in 2 pages that :Our goal for the RADEON 8500 is and always has been to deliver the best possible gaming experience to our customers.

yeah right.!

Quality is lowered to boost benchmarks

[Namarrgon]

If ATI had just optimised for a specific game, I think nearly everyone would be fine with that. Perhaps the only objection you could make is that they might have spent their time optimising for the Quake3 engine instead, covering a wider range of games and benefiting more users. When that game is a benchmark (or rather, THE primary benchmark used to determine real-world performance under a set of well-known and duplicatable conditions), then optimisations specific to that are a little more questionable.

While many companies focus upon optimising for benchmarks, most simply optimise the driver paths for the specific cases that those benchmarks use. Any other app that uses similar settings can gain performance from that work. But ATI have made their optimisations dependant upon the name of the app, so no other apps can benefit from their work. While a Quake3 player might not mind, Q3 isn't as widely played today as it once was, and that same player might be less pleased when the card fails to perform to the same standard on any other of their games.

But what makes this particular "optimisation" underhanded is that it's not better or more tuned code, it trades off quality. Have a look here to see the mess that it makes of textures!

Now, if players wanted to see blurry textures in exchange for more performance, they'd simply lower the texture quality slider in the Quake3 game. ATI's drivers do this for them; they're forcing the mipmaps two levels down - a 16x reduction in texture detail - to get the extra speed. And this isn't optional. You can't turn it off, short of using a hex editor.

Particularly, a reviewer running the standard Quake3 High Quality benchmark will never notice the difference (as the frames run by way too fast). There's no extra sliders in the driver, no other indications or switches, and of course no notification in the driver documentation. All the reviewer sees is higher framerates, because the drivers are, quite literally, cheating. They're giving low-texture numbers on the high-texture setting.

I for one applaud ATI's renewed efforts to improve their drivers (and I still plan to buy an 8500DV alongside my existing QuadroDCC), but I feel ATI really have attempted to subvert the benchmark process, and shot themselves in the foot. The strong implication is that, even though their hardware is fine, they don't feel they can compete with nVidia's driver team so they have to resort to methods like these.

Very unwise - they've lost a lot of the support they had as the underdog to nVidia, through these tactics.