Slashdot Mirror
FutureMark Confirms nVidia's Benchmark Cheating
jlouderb writes "As first reported by ExtremeTech, Futuremark has confirmed that nVidia is cheating on its 3DMark2003 benchmark through eight driver optimizations. The 3D graphics performance war just keeps getting more and more interesting!" See our previous story.
Test with the applications/games people really use, and they can't optimize for them without, well, optimizing for them! If they want to make Quake III faster, great.
You don't base your findings on one benchmark. Whenever I go to a site like tomshardware.com they have several different ways to benchmark. Each card has its own strengths, and if a card has cheated it will show up like that.
Paint.NET, a Free Image Editor, with Source Code Available!
eh, no such thing. Software and hardware companies have been cheating on benchmarks for decades. This is nothing new.
Do it doug.
WHAT?? My FX 5800 Leaf Blower only has a range of five feet and not six? I want a refund!
/^[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,4}$/i
There a lies, damm lies and statistics .
I remember SPEC benchmarking ment something, and companies putting special routines to make chips seems faster than they were.
Thats why "Real world testing" is important. While not always the greatest comparison, its much better in most cases.
you can't say nVidia doesn't know how to write effective drivers ! :-|
That's one area that ATI has traditionally been weak in, although that's changing.
I wonder if ATI will attempt to sue.
Look back up at my post, now look back down, you're on the Internet. Now look back up. I'm a signature.
While this isn't a huge suprise, I am happy that there are smart folks out there who spend time to uncover this kind of information. Kudos to you for your efforts!
Videocard Benchmarks are about as believable as the the 'World's Best Grampa' award.
-n
http://www.remix.net/
How can company proceed to do its business while blatantly lying to its customers!!??
Oh wait, my medication just kicked in. It's just business as usual. I will just go on checking my MSN e-mail, while watching MSNBC, drinking my Coke and eating my McDonalds burger.
Never mind.
Wearing pants should always be optional.
anyone have a mirror of that pdf? all I get is a 7 page grey box :/
No prom tickets for them.
They won't get into a good college.
Their grandparents and I are so ashamed.
So it's quite likely that NVidia was just anticipating optimizations and not outright "cheating."
In other news, Futuremark's site was 'slashdotted' by terrorists today. Similar to a DDOS attack, slashdotting is a system admins nightmare!
Buy my new anti-slashdot software today!
Comment removed based on user account deletion
Do it, Doug!
Best sig ever.
Love,
Rob Feature
How are you going to keep them down on the farm once they've seen Karl Hungus?
Calling them optimizations gives what nVidia is trying to do a level of legitimacy which is undeserved. If you read the Futuremark paper, you will see that they are clearly cheating.
It would be as if a CPU manufacturer substituted its own algorithms stealthily in a CPU performance benchmark and only when running that benchmark.
Sure, you get a higher number, but you aren't measuring what the benchmark designer intended to measure.
Thank you for submitting this to Slashdot. With Futuremark slashdotted to death, NOBODY will be able to get the evidence! *manical laughter*
This has been done for many years, even the last decade. A good friend of mine works and has worked for almost every major video card company in the buisness for the last decade. What is his job? Make sure THEIR video card gets the best scores on the latest and greatest video cards.
I am sorry to tell you all, but just because Nvidia was CAUGHT this time, doesn't mean they haven't been "cheating" (by optimizing for a specific benchmark) for the last 6 years.
I would bet every driver release contains code to help out benchmarks and even specific games. Why do you think Nvidia just said with there latest driver release " *Up to 30% faster frame rates ( *With Unreal Tournament 2002)".
Its just once in a great while someone notices a performance jump TOO big, or just wants some news worthy-ness and decides to put out a nice PDF file.
- Jeff
Modesty is one of life's greatest attributes
One player "cheated" by whopping 1.9% (withing the margin of error actually). Other cheated by 24%.
Lesbian Nazi Hookers Abducted by UFOs and Forced Into Weight Loss Programs - -all next week on Town Talk.
From what I read from [h]ardOCP's benchmark with doom3 It kills nvidia's card. And who cares aren't you suppose to optimize your card?
They also have another benchmark here where they compare the 5900 ultra and the radeon 9800 pro. In that article it says that NVIDIA told them not to use 3DMark03 I recommend reading that article
9th grade, you told me cheaters never make money
well 'pbhtbhtbthbth'
I thought that ATI did the same with their Radeon 8500 drivers 2 years ago, making their Quake 3 scores look better by "cheating". Isn't that just status quo in the video card manufactoring world.
Our investigations reveal that some drivers from ATI also produce a slightly lower total score on this new build of 3DMark03. The drop in performance on the same test system with a Radeon 9800 Pro using the Catalyst 3.4 drivers is 1.9%. This performance drop is almost entirely due to 8.2% difference in the game test 4 result, which means that the test was also detected and somehow altered by the ATI drivers. We are currently investigating this further.
It not about cheating... but about how much you cheat.
"Engineers do the work of man, Physicists do the work of God"
It's still faster than my trusty GeForce2, which keep because it still renders games faster than I can physically detect...
It's still stupid though. If nVidia really wants to appear the best then there's not much we can do to stop them. The people who buy nVidia cards are not going to drop everything they're doing and boycott because of some pixel shaders or something.
I read a while back when they altered the clipping planes of some test to enhance the score, but it left hideous effects on the screen. The pdf won't load but if this is the same thing then it's old news. Add that to the fact that benchmark alteration is quite common. Who plays fair and makes money anyway?
http://198.3.92.62/3dmark03_audit_report.pdf Just don't kill me now. ;-)
A test system with GeForceFX 5900 Ultra and the 44.03 drivers gets 5806 3DMarks with
3DMark03 build 320.
The new build 330 of 3DMark03 in which 44.03 drivers cannot identify 3DMark03 or the tests in
that build gets 4679 3DMarks - a 24.1% drop.
Our investigations reveal that some drivers from ATI also produce a slightly lower total score on
this new build of 3DMark03. The drop in performance on the same test system with a Radeon
9800 Pro using the Catalyst 3.4 drivers is 1.9%. This performance drop is almost entirely due to
8.2% difference in the game test 4 result, which means that the test was also detected and
somehow altered by the ATI drivers. We are currently investigating this further.
Who needs features! Everyone complains, but with a few more high-performance bugs, frame rates should shoot through the roof!
-1 Uncomfortable Truth
If I follow your line of thought, then murder isn't a big deal since it has been done since the beginning of the human existence?
You better make your cards way cheaper by the time Half-life 2 comes out.. or else!!
We should have a constant for each 3d company that we can multiple their benchmarks agains...
...
Maybe nvidia is 0.80 and ATI is 0.90
so then 100fps on a geFrorce card, is really 80 fps, and it would be 90 on an ATI...
The "optimization" relied on the benchmark camera being on 'rails'. It always shows the exact same angles, and there are some things that the benchmark would have the graphics card render, even though it's impossible for the viewer to see.
HOWEVER, in the development version of 3dmark 2k3, you can take the camera "offroading". When you do that, it becomes apparent that things are being drawn incorrectly -- that there are hard-coded limits that result in the video card doing less work than the program requests.
For those of you whining about how they should use "real life" games for benchmarks, this technique could be applied to anything where the camera path is predetermined. It has nothing to do with 3dmark 2k3 specifically.
No, because ATI did a much worse job of cheating. Nvidia got a 24% boost out of some of the benchmarks while the best ATI could do was a measly 8%. This clearly shows that ATI must cheat harder if they want to keep up with Nvidia.
I read the internet for the articles.
Different graphics cards have different strengths and weaknesses - much moreso than in previous years.
eg. Fillrate, Vertex manipulation, Texture rasterizer, Shader technology, Texture sampling techniques, Shadow buffering etc.. etc...
Some cards will be better than other at these tasks, and some games will take advantage of differing ratios of these technologies.
The unreal engine has a reliance on poly-count and texture resolution, and it looks like the doom engine will tend to tax shader, and multitexture units more than the polygon throughput side of things.
In other words, gfx cards are now so flexible that their abilities in these individual areas must be assessed in isolation depending on your choice of game/engine/technology.
As little as 2 years ago all that mattered was fillrate, and this was essentially what the direct3d/opengl api's could stress in hardware.
IMO, price seems to be the most useful benchmark for the newest cards.
\\ Mitch
Not entirely true. They were only looking for nVidia specific cheats, and they happened upon an ATI cheat as well. Who knows how many ATI cheats they will find once they start investigating ATI's drivers?
Let me just say that this occurs not just on this test, but on all imaginable tests, as well as all games that are somewhere used as benchmarks. Many of the cheats are hard to detect because they don't break the test in the way that this cheat did. For instance, at some point there was a trick for a test with lots of occlusion to clip (discard) polygons that would eventually be occluded. However, these discarded polygons were actually calculated at run-time and not precomputed, so if you changed the test, it would still work right. For Quake (I or II, can't remember) they had a hack where they wouldn't need to clear the framebuffer. That version of Quake would do a glClear at each frame, which takes some time, and prior to framebuffer compression, there was a hack where you wouldn't need to clear the framebuffer if you swapped the Z-check and only used half of the Z span every frame. That hack's probably been backed out now because with framebuffer compression, you're actually better off doing the glClear each frame.
Anyway, I'm posting this as an AC for obvious reasons.
not that we didnt already know this , but it nice to have the company confirm that there was cheeting going on. Nvidia , at least from my POV , was consider by me to be one of the best graphics card companies , now they have lost that priviledge after trying to ly to us . Boycot Nvidia (unless they make something really cool). P.S. Article has some really interesting stuff about why it is easier to detect cheeting in synthetic benchmarks , so if like most slashdot people you didnt read the article you might want to.
Muah. Nice to see that some people are willing to stand by their "Cheatin'" display adapter supplier. Kewl, says I.
They've never done you any harm. And except for recent accusations of revenue massaging, they don't lie.
the pdf for bittorrent
YHABT, HAND!
nVidia Rep: Just look at how fast Quake III is running!
Reviewer: Sure but why is it just in wireframe?
Trolling is a art,
My hairs starting to get a bit funky.
Thats why "Real world testing" is important. While not always the greatest comparison, its much better in most cases.
:)
"Real world testing" is great, if you're just a gamer. The problem is with independent developers who want to know about the performance of a card. Not only are features important but also how well can the card perform. I put both in consideration when I'm looking for a card. I don't consider current game benchmarks much because those games won't matter in 6 months, or by the time I finally finish my game.
-B
I made a good point about this sort of thing. See post 65272
Support Israeli punk bands. Man Alive.
3DMark03 is such an unreliable and poorly programmed piece of code that it is all but completely useless as a benchmark. This was not an attempt at artificially inflating 3DMark scores it was simply a bug in the drivers that affected the calculations when clipping and culling.
I think that Kyle at HardOCP said it best when discussing this issue he said "Finding a driver bug is one thing, but concluding motive is another." Look up his comments on Thursday May 15, 2003 if you want to read more.
Doesn't everybody fuck a rubber ducky in the shower?
The preceding post was not a Slashvertisement.
Page 1 of 7
,00.asp
Futuremark Corporation
May 23rd, 2003
To the Media, Futuremark's customers and business partners:
Audit Report: Alleged NVIDIA Driver Cheating on 3DMark03
After the launch of 3DMark03 Build 320 Futuremark has received reports from the members of its BETA Program concerning certain anomalies with 3DMark03 and Nvidia drivers. ExtremeTech (www.extremetech.com) has published an article1 on suspecting NVIDIA drivers to improperly boost scores on Futuremark's 3DMark®03. Some of these anomalies have also been reported by Beyond3D2. Alarmed by all these reports Futuremark has conducted a thorough internal audit regarding this matter and has verified that certain NVIDIA drivers indeed seem to have detection mechanisms, which are triggered by components of the 3DMark03 program. We have identified eight such mechanisms.
In our testing, all identified detection mechanisms stopped working when we altered the benchmark code just trivially and without changing any of the actual benchmark workload. With this altered benchmark, NVIDIA's certain products had a performance drop of as much as 24.1% while competition's products performance drop stayed within the margin of error of 3%. To our knowledge, all drivers with these detection mechanisms were published only after the launch of 3DMark03. According to industry's terminology, this type of driver design is defined as 'driver cheats'.
We are publishing this document to report to our customers in detail about our findings. Main reason behind publishing this document is to answer the criticism presented against synthetic benchmarks and their reliability when testing hardware performance. The document follows a question/answer format.
How Were These Driver Cheats Found?
Members of Futuremark's BETA program3 first noticed how parts of the tests in 3DMark03 were rendered differently on different hardware. When testing NVIDIA hardware on 3DMark03 with socalled developer's version's free camera enabled, they noticed how some parts of tests were rendered strangely, and informed Futuremark of their findings. Futuremark investigated further and our findings show that certain NVIDIA drivers seem to detect when 3DMark03 is running and then replace the 3DMark03's rendering requests with manually implemented alternative rendering operations. These alternative rendering operations reduce the amount of rendering work and thereby increase the obtained benchmark result.
1 Extremetech: http://www.extremetech.com/article2/0,3973,1086025
2 Beyond3D: http://www.beyond3d.com/#news5856
3 Futuremark's BETA program is an open, fee based
Page 2 of 7
Why Does This Matter - It Is Just a Synthetic Benchmark?
We acknowledge with great pride how big a role 3DMark has in the PC industry. 3DMark score has become perhaps the most influential metric of PC performance. Enthusiasts, professional hardware reviewers and OEMs all depend on 3DMark results to a great extent.
We have a tremendous responsibility towards our users, who count on us and on our products when making important decisions. Thus, it matters a great deal that no one is able to take advantage of 3DMark - or any other significant benchmark - with unfair means.
Well designed synthetic benchmarks are excellent tools to objectively compare performance and to reveal different architectures' strengths and weaknesses. Some commentators have argued for only using benchmarks based exclusively on games; however there are severe problems with relying only on this approach:
Game benchmarks only demonstrate how the hardware performs for that particular game and do not indicate the overall performance of the hardware.
Any cheats potentially included in drivers are much easier to hide in game benchmarks.
Finally, synthetic benchmarks can stress the hardware in a variety of ways, allowing reviewers to explore performance in particular areas and extrapolate performance of a hardware f
yeah, i'll bet *your* "backdoor" has been widely used too
> just keeps getting more and more interesting!
Uhh, no it's not.
Antiquated competence won't be a job skill forever.
What in the hell are you talking about? Do you realize what you're replying to?
Are you drunk?
What site are you talking about? Please post the address of this new site so I can check it out. Thank you.
Lets not forget that about 4 months ago Nvidia deemed 3D Mark2003 a poor representation of real world scenarios, so how could they be "Cheating" if they pointed this out before hand? and what about all the other FX5900 benchmarks where Nvidia had a steady 20 to 30% lead on ATI? This article was posted before the FX cards were released, Nvidia's not trying to "SNEAK" anything by us here. "The primary goal of any benchmark is to arm the consumer with the right information to make the best possible purchase decision. As the gamers' benchmark, 3DMark 03 must emulate as closely as possible the kind of experience that the gaming enthusiast will expect on their machine. It must exercise graphics hardware in the same manner that consumer games will. The graphics features, rendering paths, and effects must all emulate games, or the consumer will be misinformed and their expectations misguided. 3DMark 03 combines custom artwork with a custom rendering engine that creates a set of demo scenes that, while pretty, have very little to do with actual games. It is much better termed a demo than a benchmark. The examples included in this report illustrate that 3DMark 03 does not represent games, can never be used as a stand-in for games, and should not be used as a gamers' benchmark. The ultimate injury to the consumer of such a benchmark is three-fold. First, of course, the consumer is misguided. A purchase decision based on ineffectual data will lead consumers to wrong conclusions. Second, it causes graphics hardware manufacturers to focus attention and engineering resources on optimizing for artificially fabricated cases that are a-typical of games. Such optimizations generally do nothing to improve real game performance, and provide no benefit to the consumer. Finally, the extra engineering effort focused on such benchmarks reduces the effort available for activities beneficial to consumers--improving the actual gaming experience." http://www6.tomshardware.com/column/20030219/index .html
Of course if the article title was, "Everybody cheats on our benchmark!" then that would do more to undermine their benchmark than anything else. Instead they made the focus of the article the fact that NVidia is cheating.
Lasers Controlled Games!
Mirror. Slashdot into oblivion.
It's quite funny that few months after NVIDIA left Futuremarks "developer program" (and thus stopped paying to them), NVIDIA is accused for cheating.
Both companies have optimized their drivers for 3DMark for several years.
According to the article, that's only half the story. I could almost accept it if they were "optimizing" in the sense that, in certain situations, they slightly reduced image quality for a significant gain. That's kind of sketchy, as the card isn't then doing what it's claiming, but you could argue, perhaps, that the tradeoff is worth it. And if this activity were optional, it might be a benefit.
What they're doing here is different, and much worse. They're actually detecting what program is running - whether it is 3D Mark or not. Effectively, what it does is disobey 3DMark, and only 3DMark, when it issues certain commands that would reduce throughput. That has no purpose but to deceive.
So, not only are these not optimizations in that they don't really improve performance, they're not optimizations in that they don't even take effect when you run a program not called 3DMark.
Quite frankly, I think this could be considered false advertising and nVidia should get in deep shit for this. This is the worst kind of cheating, and quite frankly, this could be what puts nVidia down the Voodoo path. I don't know whether I'll ever buy another of their cards.
-Looking for a job as a materials chemist or multivariat
I may be taking this a bit OT, but...
How do you know there have been murders since the beginning of human existance? And don't just quote me a line in a book that's a few thousand years old; that's not proof. I want the proverbial smoking gun. Show me the corpses!
Bad analogy, good sentiment.
What the PDF documents is a smoking gun. The only way to get the displayed results would be to hard-code the clipping plane. While doing so does improve the raw fps while the camera's on the rails, it goes against the very nature of comparitive testing. Its not something you can do in a real environment when the camera isn't on rails.
Page 1 of 7 ,00.asp
Futuremark Corporation
May 23rd, 2003
To the Media, Futuremark's customers and business partners:
Audit Report: Alleged NVIDIA Driver Cheating on 3DMark03
After the launch of 3DMark03 Build 320 Futuremark has received reports from the members of its
BETA Program concerning certain anomalies with 3DMark03 and Nvidia drivers. ExtremeTech
(www.extremetech.com) has published an article1 on suspecting NVIDIA drivers to improperly
boost scores on Futuremark's 3DMark®03. Some of these anomalies have also been reported by
Beyond3D2. Alarmed by all these reports Futuremark has conducted a thorough internal audit
regarding this matter and has verified that certain NVIDIA drivers indeed seem to have detection
mechanisms, which are triggered by components of the 3DMark03 program. We have identified
eight such mechanisms.
In our testing, all identified detection mechanisms stopped working when we altered the
benchmark code just trivially and without changing any of the actual benchmark workload. With
this altered benchmark, NVIDIA's certain products had a performance drop of as much as
24.1% while competition's products performance drop stayed within the margin of error of 3%. To
our knowledge, all drivers with these detection mechanisms were published only after the launch
of 3DMark03. According to industry's terminology, this type of driver design is defined as 'driver
cheats'.
We are publishing this document to report to our customers in detail about our findings. Main
reason behind publishing this document is to answer the criticism presented against synthetic
benchmarks and their reliability when testing hardware performance. The document follows a
question/answer format.
How Were These Driver Cheats Found?
Members of Futuremark's BETA program3 first noticed how parts of the tests in 3DMark03 were
rendered differently on different hardware. When testing NVIDIA hardware on 3DMark03 with socalled
developer's version's free camera enabled, they noticed how some parts of tests were
rendered strangely, and informed Futuremark of their findings. Futuremark investigated further
and our findings show that certain NVIDIA drivers seem to detect when 3DMark03 is running and
then replace the 3DMark03's rendering requests with manually implemented alternative rendering
operations. These alternative rendering operations reduce the amount of rendering work and
thereby increase the obtained benchmark result.
1 Extremetech: http://www.extremetech.com/article2/0,3973,1086025
2 Beyond3D: http://www.beyond3d.com/#news5856
3 Futuremark's BETA program is an open, fee based cooperation program between Futuremark
and the PC industry at large. BETA program members have access to pre-release builds of
upcoming benchmarks and to a so-called developer build. The developer build is exactly the
same as the public version of the benchmark, but with additional functionality. Amongst other
things, the developer build has a 'free camera' mode, where the user can manually move the
camera around while the test is running.
Page 2 of 7
Why Does This Matter - It Is Just a Synthetic Benchmark?
We acknowledge with great pride how big a role 3DMark has in the PC industry. 3DMark score
has become perhaps the most influential metric of PC performance. Enthusiasts, professional
hardware reviewers and OEMs all depend on 3DMark results to a great extent.
We have a tremendous responsibility towards our users, who count on us and on our products
when making important decisions. Thus, it matters a great deal that no one is able to take
advantage of 3DMark - or any other significant benchmark - with unfair means.
Well designed synthetic benchmarks are excellent tools to objectively compare performance and
to reveal different architectures' strengths and weaknesses. Some commentators have argued for
only using benchmarks based exclusively on games; however there are se
I dont think that these benchmark progreams really make much difference in what hardware people buy. I mean heck I'de never buy nVidia. And who really cares if it fudges benchmarks if we get higher FPS in games? i know for shure i dont. Lets stop the flaming and game!
So benchmarking with the leaked ATI-specific demonstration build, "kills nvidia's card"?
Big whoop. Benchmarking and publishing results from that build is utter stupidity.
In other words, 'have convictions - unless they're inconvenient'. Very nice.
...Slashdot was to host a BitTorrent of this and similar files for faster, cooperative downloading?
I've said it before, and I'll say it again: doing this would be a win-win situation. It's a pity that the editorial team are too busy playing with MAME/whatever to actually do something of real benefit to the wider community.
"Accept that some days you are the pigeon, and some days you are the statue." - David Brent, Wernham Hogg
Eat my shit, Zoidberg.
optimizing and cheating is different.. the nvidia drivers detected 3dmark and rendered things differently to improve score, by 24,1 %
.pdf, I really think that futuremark is trying to do the right thing.. and it's not about revenge..
Futuremark pointed also out that ATI-drivers did the same in the last test but much less only 1,9% which is inside the error margin. Try reading the whole
I wonder why this driver cheat was discovered by Extremetech? If you're a video card manufacturer, wouldn't you have your engineers go over every one of the competitions driver releases with a fine-toothed comb, just hoping to find some kind of cheat? You'd think ATI has better testing facilities are resources then ET.
Certainly any negative publicity for NVidia is good for ATI and vice versa.
I am NOT a man!
I am a free number!
Fuck your mom, fag.
eat your watermelon, slant-eye.
When ATI does it everyone screams bloody murder! When Nvidia does it the apologists come out of their holes! Baaaah
This is why gfx drivers should be open source, not to detect backdoors, but to detect performance cheatings! :)
:)
Also, if it was open source, maybe ppl could get all those annoying bugs out of their drivers, since it seems nvidia guys just can't release a 100% stable stuff
What about ati? Are they "stable"? (especially when multiple X servers are running, on normal screen & tv at the same time?)
Greetings,
Me.
What might be relavent here is that 3D Mark might not be culling geometry very agressively, while a normal game would. NVIDIA might use this cheat to keep their cards from getting an unrealistically low score in one benchmark, compared to their performance in other benchmarks.
:)
Does this make their behavior less wrong? Hell no. But it kinda makes it harder to demonize them quite so heartlessly
A deep unwavering belief is a sure sign you're missing something...
Just think about this the next time you do a 5MB driver download. How much of that code is specifically for detecting and defeating benchmarks? How much of the cheats are part of the instability problems in your system?
Here is an interesting quote from the article that seems to have been overlooked so far.
"Our investigations reveal that some drivers from ATI also produce a slightly lower total score on this new build of 3DMark03. The drop in performance on the same test system with a Radeon 9800 Pro using the Catalyst 3.4 drivers is 1.9%. This performance drop is almost entirely due to 8.2% difference in the game test 4 result, which means that the test was also detected and somehow altered by the ATI drivers. We are currently investigating this further.
Gasp, what a shock. Everyone seems to be guilty of having cheated on synthetic benchmarks at some time. This has happened before, it will happen again.
?? I read different results elsewhere.
Check out these numbers on anandtech. Looks like the 5900 Ultra performs very well in Doom3.
~Berj
my 5.7 GB of recently-acquired X-Men comics would disagree with you.
Well, you know, if they were sentient, and all.
0x0D 0x0A
Sorry, the alternate renderers aren't more efficient. They are just optimized for the expected environment.
If you expect a certain situation to exist, you can skip lots of the other stuff you do and get it done faster, easier or with less effort. You could say you are more efficient, but basically you just cut out lots of what you do.
This is why people get in more car accidents close to home, they make many assumptions, ie nobody drives here, or there isn't anyone around that corner. and they don't look. Generally this results in optimized performance, no time wasted being careful.
If it doesn't work right, it isn't efficient, cause you're left with garbage.
Ummm, yes it is. Right now evidence says that Ati-results are withing margin of error. Everything else is just speculation. And there is no concrete proof that Ati has been cheating, but FM is looking in to the matter.
Lesbian Nazi Hookers Abducted by UFOs and Forced Into Weight Loss Programs - -all next week on Town Talk.
Maybe they should have not bought 3dfx's assets . . .too many parallels.
I think it is a curse that has to do with late product cycles.
Also the ridiculousness of the Geforce FX jet engine cooling system is only trunped by the ill fated Voodoo 5 5500's 4 GPU design that was the size of today's ITX motherboards.
Funny, I seem to remember Toms Hardware being rabidly AMD fanboyish about 1.5 years ago when AMD still had the fastest processor. I'm not saying they aren't biased fanboys, what I'm saying is they're fairweather fans.
Isn't that the definition of a good reviewer? Fans of the current top of the line stuff - damn their history?
To keep it on-topic, I also seem to remember ATI doing the exact same thing nVidia is now doing with quake "optimization" for the 8500 cards... Do a google search for "quake quack"
Case in point...
That's not true. They "cheated" by 8.2% on the same benchmark nVdia was "cheating" on, when wieghted into the full benchmark score it came out to a 1.9% difference in total score.
So, ATI results drop from this new patch too? Doesn't this mean that ATI is also cheating? If so, then how do we know that there isn't more cheats ATI is using, as this new patch is only made to exploit the nVidia ones. ATI has access to the developer version of 3D mark, so they could hide their cheats much more efficiently.
So how much did the quality drop during these tests? Was it significant? Maybe the hacks nVidia put in for 3dmark03 specifically could be carried over to other games?
ATI had their own cheating debacle a few years back.
Quake 3 vs Quack 3
I say demonize away. If they believe that customers rely [too much] on the 3DMark benchmarks to make their purchasing decisions and then they proceed to cheat those benchmarks, that's fraud, de facto and de jure.
Allow me to correct you. Ati scores degraded on one benchmark. It's overall score dropped by 1.8% while NV's score fell by 18.2%. Ati's drop was caused by 7.3% drop in Game Test 4, where their performance dropped by 7.3%. On that same test, NV's performance dropped by 49.2%.
Ati's only reduction of performance was on the game-test 4. For NV, their performance fell like this (Ati's results in ()):
Overall score: -18.2% (-1.8%)
Game Test 2: -13.2% (0%)
Game Test 3: -5.6% (0%)
Game Test 4: -49.2% (-7.3%)
Vertex Shader: -37% (0%)
Pixel Shader: -56% (0%
Source: http://www.hardware.fr/html/news/#5797
As you can see, NV cheated _ALOT_ more than Ati did!
Lesbian Nazi Hookers Abducted by UFOs and Forced Into Weight Loss Programs - -all next week on Town Talk.
Where can I get this card that provides 238923213 FPS?
I own 4 nVidia GeForce 3/4Ti cards, and this makes me sick! I guess next time I go shopping for a video card, I should start looking at ATI again. Semms like ATI is innovating again, and nVidia is just hoping that their brand name saves them...?
I love the nVidia Linux drivers, how good are ATI's?
I want my rights back. I was actually using them when our government stole them after 9/11.
Maybe the hacks nVidia put in for 3dmark03 specifically could be carried over to other games?
'Fraid not.
As others have pointed out the hacks were related to the fact that traditionally the camera is "on rails," meaning that you see a pre-determined view.
What nVidia did involved stuff that was outside of this field of view.
Since in a game you generally can't be sure what the field of view will be you can't carry these over.
A proper analysis accounts for this, I should have said disproportionate, not most.
If you take 10 mile drives, and 5 miles is "close". You would expect half the accidents to happen close to home.
If 2/3rd happen close to home this would be disproportionate, and is what I meant.
From the article:
Our investigations reveal that some drivers from ATI also produce a slightly lower total score on this new build of 3DMark03. The drop in performance on the same test system with a Radeon 9800 Pro using the Catalyst 3.4 drivers is 1.9%. This performance drop is almost entirely due to 8.2% difference in the game test 4 result, which means that the test was also detected and somehow altered by the ATI drivers. We are currently investigating this further.
see: http://www.hardware.fr/medias/photos_news/00/06/IM G0006295.gif
If that is a problem for you, I would suggest you get your nose out of your armpit.
I have posted anonymously here in the past for various reasons. I take anonymous posts with ample amounts of salt. What's your problem?
I forget what 8 was for.
Comment removed based on user account deletion
My mirror
Benchmark cheating on video drivers has been going on ever since video driver benchmarking. I worked for a graphics card manufacturer for a few years and a driver development company for a few years and remember a few interesting "optimizations". (This was in the 1991-1997 timeframe.)
We had cheats -- excuse me, optimizations -- that were specific to a particular version of Winbench. Other "cheats" were true optimizations... they were just optimizations you'd probably only see in the benchmark program. A few were known to reduce performance in the "real world" but produced higher Winmarks... so we turned them on only when we detected Winbench was running.
One of the major motivations towards benchmarks that use real applications/games was because of these "optimizations" that were only useful in the benchmarking program. Optimizations for those are still cheats, in the sense that they are intended to improve benchmark results, but at least they are optimizations that have a real-world positive effect.
This is basically a "nothing new under the sun" story. But it's good for these kinds of articles to come out every once in a while, because it helps keep things from getting too out of hand.
-(realname) posting anonymously.
Why is this called a war? Its not like we're seeing the companies flaming each other and dropping prices.
Comment removed based on user account deletion
I can make my ATI Raedon 7000 have an amazing frame rate. The magical 1x1 resolution.
Trident cheats on 3DMark too!
I think it's awesome that Futuremark has come out swinging on this one. NVidia has obviously cheated horribly on these benchmarks. ATI aparently has also taken the low road on these but not as low as NVidia.
NVidia is losing. Their chips and cards are worse than ATI's. What's worse than that, though, is that they are still trying to pretend that it's not the case. They need to seriously sit down and work on their designs but instead they are pissing money away working on cheating on benchmarks. That is a really bad sign for a company. It means managament is diverting money away from becoming successful twords appearing to be successful. A mentality like that is disasterous to the real value of a company.
SELL! SELL NOW! Buy again when they have fixed their mangement and design issues.
Contravertial != Overrated. Reply if you disagree, I'll read it.
set softtabstop=4 shiftwidth=4 expandtab nocp worlddomination
"Who came out with a standard API that ALL manufactures could use without resorting to the arcane obfuscation of OpenGL? That's right, cuntfaces...
It was Microsoft."
Right. All manufacturers... whose hardware works with windows. I'll take cross platform compatability thank you very much.
Before you might argue that nobody uses OpenGL, what about all those licensees of the Quake 3 engine? And what about all those who will license the Doom 3 engine?
read this way back from 8/25/02 on tomshardware then read about the current dealings.
It's amazing how quick the 'good guys' get sleazy. Maybe when NVIDIA bought out 3Dfx they got some bad apples....
-malakai
-Malakai
A Dragon Lives in my Garage
Until AMD joining Bapco, the organization was hijacked by Intel. But with no source code in sight, who knows what other conspiracy is hidden inside the benchmark?
Hardocp got an exclusive Doom 3 demo because of nVidia. That might have given them a little incentive to nudge the results slightly into the right direction.
It has not yet been established that Ati is cheating. The change in the overall-score fits within the margin of error. It has been proven that NV cheats, it has not been proven that Ati cheat. Ati and their drivers are still under investigation
Lesbian Nazi Hookers Abducted by UFOs and Forced Into Weight Loss Programs - -all next week on Town Talk.
"A test system with GeForceFX 5900 Ultra and the 44.03 drivers gets 5806 3DMarks with 3DMark03 build 320. The new build 330 of 3DMark03 in which 44.03 drivers cannot identify 3DMark03 or the tests in that build gets 4679 3DMarks - a 24.1% drop."
5806 * (1 - 0.241) = 4407 != 4679
You could not prove to any court that NVidia is using deceit. NVidia improved their driver so that a certain set of operations runs faster. There is nothing deceitful about this.
Even if they were to state on the box that they have the card that performs best on the 3DMark2003 benchmark, it would still be a truthful statement. Logically, it's a flaw of the benchmark that it is able to be exploited.
If there is any deceit involved, it would be if someone were to claim that the result of this one benchmark conclusively proves that the NVidia card is superior.
They've never done you any harm. And except for recent accusations of revenue massaging, they don't lie.
Well, friend. It's time you learn that nothing is sacred. Yes, Virginia, even Coca-Cola lies and squashes people to keep its bottom line intact. Read the sad and infuriating tale of judicial corruption and corporate fraud of Bob Kolody vs. Coca-Cola. I was outraged for days.
If it's for-profit but free, you're not the customer -- you're the product (e.g., the Slashdot Beta's "audience").
Let's not jump on nVidia too harshly for this. Sure, this spectacle seems to have gained a lot more publicity than ATi's own cheating ( link link link ). At least when nVidia cheated in 3DMark, they publically denounced synthetic benchmarks.
Yeah, that's what makes slashdot the great place it is today. A bunch of morons spouting off before engaging their brains. Assuming brains were standard equipment on their model to begin with.
-Looking for a job as a materials chemist or multivariat
Modded to 5? Who did this guy sleep with?
ATI got a 8.x% change on one test, which (if I recall) ended up being a minor 1.x% overall, within the 3% error margin.
The 24% - TWENTY FOUR PERCENT on the Nvidia was their OVER ALL SCORE. In fact in one place in the article, which you didn't read very damn well, it states that the pixel shader substitution resulted in a TWO FOLD (200%) change in the score of a single test for Nvidia.
ATI released these drivers promising a free %40 improvement in performance. Well, you got it, if all you like to play is 3D Winbench 98.
Tom's Hardware uncovered the discrepancy
But certainly note that Intel's 740 drivers were also blatantly cheating. Everyone cheats. IF you get caught, deal with it, and don't get caught again. Hopefully, ATI got this concept right the 3rd time. I must say I'm shocked at the miniscule amount of easily detectable cheats ATI is using, they're almost playing the game straight-up!
Man is the animal that laughs.
And occasionally whores for Karma.
This is a theme that has been repeated several times here on Slashdot in different forms, whether it refers to games or hardware or something else.
I've written on this subject before, specifically about Real World tests. Real Gear may be another consideration for you when deciding what makes a reliable review. I seem to recall that nVidia supplied most (if not all) of the people who 'reviewed' this card a beta or a reference board. Too bad the average person can't buy a beta or a reference board at their local retailer.
Long-term testing is a critical part of our review philosophy at Geartest.com: Real gear. Real world. Real reviews.
What does that mean? Real gear: We don't write reviews about products in a pre-release stage or based on press releases. Real world: We use the products for an extended period in real conditions. Real reviews: Then we tell readers what we found, with updates as warranted. That results in a fair review. That means that good, bad or mediocre, products will get the reviews that they deserve.
We tend to concentrate on the qualitative aspects in our technology product reviews because our focus is on producing fair, reliable, plain-language reviews with an eye to the user experience and long-term value. When you buy a product you want to be sure that it will serve you well and perform over time.
The problem with most so-called 'reviews' you see in the technology press is that they aren't real reviews at all. Using a gadget for a few hours here and there over a period of a couple of weeks doesn't tell you anything about the product's performance over an extended period of time. Neither does focusing on how pretty something looks. Then there's the regurgitated press release factories.
In this case, the 'reviewers' have to take their share of the blame for nVidia cheating and tweaking the units that they supplied. After all, the 'reviewers' are the ones who agreed to 'review' and benchmark pre-release products.
you have to then assume the driver just happened to, F up, as you say, 8 different times.
So does this mean NV's driver developers are smarter, or more ballsy? :)
Well its certainly not like nVidia was ready for the 9700, I mean it did take ATI a whole 3 years to catch up.
3DMark is also insigificant in the real ream of things, if you actually look at the benchmarks of actual game, the things that people actually play, the 5900 still beats 9800 Pro.
kiss my ass, ass.
Hell, I give anonymous posters +1 automatically, in my prefs. The real trolls still get modded down enough that I don't see 'em (I browse at 2). Works out pretty damned well.
as long as this keeps happening, benchmarks like 3dmark will just become meaningless
then we dont have to listen to the bragging
I can tell you you're doing something wrong as a developer if you're actually *buying* the cards you work on.
I've been in industry for a little over a year (a 3-D programmer) and even before I started working on games to pay my bills, I would regularly get hardware from ATI to work on (for free). nVidia was a little less forthcoming with theirs, although I've heard that they sing a different tune nowadays.
I currently have no clever signature witicism to add here.
is 3dmark03 a synthetic benchmark or a eye-candy?
if i remember correctly some of the people who funded futuremark had something to do with a demo named "second reality". a good old school demo on 2 discs.
if 3dmark was TRULLY a bench it would then resort on code that we find in games!! opts are expected for thoses...even more for stuff...
what if you told carmack that the opt he made for quake and tweaked openGL implementation are just cheats? Sure you remember 3dfx ogl implementation and riva128 drivers...
what if you told ppl from the 'scene that their demo sucks because they don't properly handle Z buffering.
They all rely on tricks.(beter than opts or cheating from a coder point of view), even processors rely on thoses. they're based on user experience, not bogomips or whatever. page-flipping was a inproper behavior at a time when VESA was not VESA but scene called mode-X, eventually it became best practice. Sprites asm hard-coding was the same and most 2d shooters are based on that.
I'm pretty sure ppl at futuremark include some kind of sleazzy code in their bench as coders always do.
the only difference b/w cheating and proper optimization is only PR. if nvidia told us "wow! we made an optimization that runs 3dmark faster" as it would with a game none would complain.
it's just that for a lot of us 3dmark is supposedly an untouchable thing. It's not. it should reflect real world 3d. and in real life you expect those kind of code workaround.
then i ask myself a question... why doesn't futuremark distribute freely a playable bench.
why put us in front of a demo claiming it's a synthetic bench and then why aren't we believing it?
because it'a a lie. either they're real world gaming and tricks are OK, either they're pure demos and tricks are not options.
Comment removed based on user account deletion
Please do not further insert your dick (advertisement) into my ass (posts).
You need to flip me over and try the other side. (signatures)
Mod me down now, trigger-happy-moderators. But first check my post history and drink down the irony of both this comment and this thread.
THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON
Dear nvidia / ATI / etc.,
Please optimize your drivers and hardware for the actual applications and games I run, not the synthetic benchmarks designed to simulate workloads. Benchmarks don't use your products, end-users do.
You have just described an optimization, not a cheat. The point of cheats is that they take advantage of knowledge that's not available to normal processes. If your "cheat" takes no such advantage (e.g. calculating its shortcuts at runtime based only on the actual rendering data) then it's actually an optimization.
There's always some annoying Karma-whore who'll post fulltext. With no CR's.
-Looking for a job as a materials chemist or multivariat
Nothing to see here.
I forget what 8 was for.
Have a look at these benchmarks and tell me if nVidia made a cheat for all of them as well. http://www6.tomshardware.com/graphic/20030512/inde x.html
Tom's Hardware does a really good job of actually testing the hardware with multiple softwares to get an accurate test of the card.
BTW -- Those "enhanced" drivers did nothing for my score.
drink my piss, asshole.
Perhaps I'm misinterpreting your comments, but after reading that article and looking at the results, NVIDIAs card creams the competition with over 50% performence leads in higher resolutions. It would be more accurate to say that doom3 kills ati's card if you make a statement like that.
An open source benchmark suite for all OSes, one that records all inputs and outputs. This would solve the problems of cheating.
You've got two products that are virtually identical. What do you recommend we 'people' base our purchasing decisions on? The top two producs support the same APIs and run within 5% of each other on all comparisons, with or without cheating. Visual quality is indistinguishable if you wear glass or have a dirty monitor. In a year both will be obsolete.
No one is suggesting sacrificing newborns in the name of the winning card, so it's not like there's much for us to get over.
I want to know which card is the best, so I'm going to look for some kind of quantified measurement. If the difference is imperceptable, I still want to make the choice with as much information as possible.
4. Restrictions
You are not allowed to modify or copy the Software, except that You may make one copy solely for backup or archival purposes. It is specifically prohibited to decompile, disassemble, or reverse engineer the Software.
Instead of putting together 7-page PDFs, FutureMark should simply sue NVidia for a DMCA violation, etc. etc.
filmcritic.com - Movie reviews on Internet time
Lest I remind you all of the importance of research.
o rce_fx_5900-10.html#doom_iii_special_preview
... but don't you think it kind of odd that the same thing is happening with this card?
... OF COURSE, this depends upon nvidia releasing the source code as well... oh dear... that'll be a while.
DOOMIII IS an upcoming game. drivers, drivers, drivers eheh.
http://www6.tomshardware.com/graphic/20030512/gef
that's where this article gets interesting. I do believe reviewers were onto something with
"The DOOM III engine offers a selection of rendering modes to choose from. NVIDIA's FX and NV2x cards are explicitly supported with their own codepath and optimizations."
and
"At first, we see the ATi cards lead the pack. The FX 5900 Ultra only overtakes the competition at 1600 x 1200. According to NVIDIA, this may be a driver problem. The NVIDIA-optimized anisotropic filtering may have trouble with the anisotropic levels Doom III uses."
"When 4xFSAA is enabled, not a single card can hold a candle to the FX 5900 Ultra. The FX 5800 Ultra is roughly on a level with the ATI cards."
This particular mode 4xFSAA never did stack up against ATIs quality settings in the same mode
I can't wait for www.omegacorner.com release of the 'omega detonator FX' and the MIRACULOUS graphical improvement it brings.
the 1.9% in 3dmark03 game4 has some debatability but it still falls within the 3% error margin !!!
$0.02 worth
oh, so that's ok then...
ATI are only lying a little bit, so it's ok.
They're BOTH lying...just 'cause nVidia is more blantant about it doesn't mean that ATI has a clear concience....
Advanced users are users too!
Warning... Adult material here...
s age?topicID=676.topic
Evidently, nVidia may have hired porn star Catalina to attend e3. Well. Here are the pics. There sure are lots of ladies at nVidia's party? I wonder if that has anything to do with the bad press.
http://pub30.ezboard.com/fopaage87310frm2.showMes
Does the DMCA make this type of exposure illegal?
I'd go on a Vegan diet but the delivery time from Vega is too long. --brownkitty
The problem here is not Nvidia. It is the dummies who wrote Futuremark.
It would be TRIVIAL to make it so that when run the program chooses a random number seed that modifies the path and the firections in which the camera looks. This would make it impossible to cheat by creating clipping planes that are out of view, and it would still allow them to create reproduceable results. All they would have to do is pick a set of random numbers and run the demo with those seeds on the different cards and compare results.
Other cheats aren't as easy for them to prevent, but the culling thing is pretty simple to stop.
FutureMark hasn't reached any conclusion about ATi's result yet, so it's premature to say that they are cheating.
Holy crap. Trying to find some more info about Mr Kolody. Sounds like the judge was paid off.
"Real World Testing" in general means that they're testing the card on games that are out on the shelf, finished products, right now; i.e. games which were targeted at video cards years old. In other words, one card does 150fps at the highest quality settings, another does 155fps, and when both of them are run on my 80hz refreshing monitor, the results are exactly the same.
Instead, I want testing that approximates the sorts of games that I'll want to buy years from now. Unfortunately those games don't exist yet. In lieu of those games existing, I can look at these eye candy benchmarks to get some idea of what the performance of video cards will be once they're pushed to their limits. How many polygons or how many dynamic lights can programmers squeeze into a scene before the frame rate drops to something unacceptable? How fast can the card whip through those pixel shader programs that everybody is going to be rendering fur and metal and such with in soon? That's what these sorts of benchmarks are supposed to do: tell me how my prospective new purchase will perform on games in the future.
Beyond3d is reporting on the ATI part of this issue.
ATI's official statement:
The 1.9% performance gain comes from optimization of the two DX9 shaders (water and sky) in Game Test 4 . We render the scene exactly as intended by Futuremark, in full-precision floating point. Our shaders are mathematically and functionally identical to Futuremark's and there are no visual artifacts; we simply shuffle instructions to take advantage of our architecture. These are exactly the sort of optimizations that work in games to improve frame rates without reducing image quality and as such, are a realistic approach to a benchmark intended to measure in-game performance. However, we recognize that these can be used by some people to call into question the legitimacy of benchmark results, and so we are removing them from our driver as soon as is physically possible. We expect them to be gone by the next release of CATALYST.
NVidia immidiately put out a rebuttal to these claims, and I'm not sure why they weren't reported along with this article. But, I guess I really can't say that I'm not used to biased or ignorant reporting from slashdot.
From Bluesnews (from an unlinked CNet article):
"Recently, there have been questions and some confusion regarding 3DMark 03 results obtained with certain Nvidia" products, Futuremark said in the statement. "We have now established that Nvidia's Detonator FX drivers contain certain detection mechanisms that cause an artificially high score when using 3DMark 03."
A representative at Nvidia questioned the validity of Futuremark's conclusions. "Since Nvidia is not part of the Futuremark beta program (a program which costs of hundreds of thousands of dollars to participate in), we do not get a chance to work with Futuremark on writing the shaders like we would with a real applications developer," the representative said. "We don't know what they did, but it looks like they have intentionally tried to create a scenario that makes our products look bad."
I know more than you drink.
Eat my cr*p, shithead.
Fondle my finger, fucker.
Not praising Coca-Cola, they basically sell carbonated sugar-water. But I saw nothing in this article that made me offended at what Coca-Cola was doing, though plenty to make me think that Bob Kolody was a strange and irrational man.
There isn't really much difference bewteen a "cheat" and a true optimization. As long as the "cheated" driver produces acceptable results, and produces them faster, I don't see what the problem is.
Some of the cheats potentially reduce image quality, but we are talking about OpenGL and DirectX here - nobody really aims for 100% visual quality, and indeed there is no target to shoot for since neither standard specifies "correct" rendering down to the pixel level.
You might complain that 3DMark is being treated specially, that other software wouldn't receive the same speedups. That is true. But application-specific optimization has a long history. Just look at Windows - the more recent versions detect and flag certain programs that are known to break or run slowly due to compatibility issues. Nobody says Windows is "cheating" because it refuses to install a driver that its internal knowledge base knows will trash your system. In the CAD world, video card makers almost always tweak drivers to support specific CAD and 3D applications. (3DLabs' control panel used to have a box where you could select "optimize for AutoCAD/3D Studio/Maya/etc...")
ATI should be happy that NVIDIA engineers are wasting time fixing specific benchmarks when they could instead be improving performance in general. But I wouldn't read much more than that into this.
Making your buying decision based on a synthetic benchmark, rather than in-context with your intended application, is always going to distort the picture. (Looking at SPEC benchmarks, Itanium blows the competition away - just tell that to the millions of people who are *not* buying IA64 chips!)
If you, the OpenGL developer, end up writing the next wildly-successful game, I'm sure NVIDIA will be happy to tweak their drivers for it.
I find it refreshing, in an unsettling way, that a post that includes the words "fucking" and "cuntfaces" can be modded "4, Insightful."
It's nice to know that Slashdot is still a place where we can express ourselves freely. I'm sure glad that PBS takeover fell through.
Page 4 of 7, middle of the page:
"What is the Performance Difference Due to These Cheats?
A test system with GeForce FX 5900 Ultra and the 44.03 drivers gets 5806 3DMarks with 3DMark03 build 320.
The new build 330 of 3DMark03 in which 44.03 drivers cannot identify 3DMark03 or the tests in that build gets 4679 3DMarks -- a 24.1% drop.
Our investigations reveal that some drivers from ATI also produce a slightly lower total score on this new build of 3DMark03. The drop in performance on the same test system with a Radeon 9800 Pro using the Catalyst 3.4 drivers is 1.9%. This Performance drop is almost entirely due to 8.2% difference in the game 4 result, which means that the test was also detected and somehow altered by the ATI drivers. We are currently investigating this further."
This didn't get mentioned yet, but I thought it deserved it -- Nvidia got caught REALLY red handed today, but ATI isn't 100% innocent either, it appears.
Note for those who just won't read the PDF: This is NOT the same cheat as the Quack Quake 3 thing. ATI got caught cheating with the same "fix" that just caught NVidia.
Every time you act like a stupid troll on a internet forum, you make baby jesus cry.
;.;
Please don't make baby jesus cry.
At the end of the day, nVidia have added optimizations to their drivers, which is what all driver manufacturers do all day long everywhere.
Whenever a special case that might be accelerated can be identified, they add an optimization for it. If that speeds up a benchmark, great, and if it speeds up ONLY a benchmark then that benchmark must be utterly unrepresentative of real apps and hence totally useless.
So what are you saying, that the benchmark was totally useless? If not, then nVidia's optimizations will have sped up some apps as well, which is the whole point of benchmarks and the whole point of optimizations.
Claims of fraud make no sense at all unless the benchmarker acknowledges that his benchmark is worthless so that the optimizations help no real apps.
"The question of whether machines can think is no more interesting than [] whether submarines can swim" - Dijkstra
Hey, don't knock it till you've tried it, Anonymous. (Links are to Kenta Wolf's CG site, which is currently kinda-sorta down. H-mode is his hidden adult page, which, amongst other things, contains fanart of some of the Sailor Scouts.)
:D
Japanese manga-style art can be *really* erotic, on par with anything I've seen come out of a pornography photo studio. I'd post examples of some of my favorite artists, but, I fear they'd be harrassed if I did.
Anyway, We "hentai"-fiends ain't hurtin' no one, so leave us be.
So, now we know a big reason why they don't want to open source their drivers. It's hard to hide cheats in open source code!
:)
With all this discussion (I'm always late to the party), I'm surprised nobody seems to have pointed out that this problem would be much less likely to occur if these companies' drivers were open source. ExtremeTech and other benchmarkers could require vendors to provide the source to their drivers (under an NDA if necessary), or only test boards that have open source drivers. It'd be kinda like blood tests for athletes.
I would not be surprised that the 'proprietary information' or 'trade secrets' that they want to protect with binary drivers is more about this kind of thing than it is about their fancy software or hardware interface secrets.
I'm working on a paper regarding the ethical dilemmas that producers of non-Free (per Stallman) software often face. This is an excellent example. In this case, if the drivers were open source, then both companies would be forced to compete on the merits, and their developers wouldn't be put in a position of participating in a fraud on their users. (Which is, by the way, a violation of the ACM/IEEE Code of Ethics.) In the long run, with open source drivers everybody's product would be improved for real, possibly more so than they are now.
It would still be possible to build cheats into the boards' firmware I suppose, but they would necessarily have to be more generic and probably qualify as 'optimization methods' - there is a grey area. It is, obviously, all an illusion!
It's easier to be a result of the past, but more fun to be a cause of the future! http://www.spacefinancegroup.com/
So basically my taco bell to mcdonalds ratio needs to be lowered. Thank you kind colon expert.
Don't know who to trust (tomshardware sucks), but if it was true it would certainly explain HardOCP's recent behavior....
I'd trust Tomshardware over Tard|OCP any day of the week. Tom may make mistakes, but at least he doesn't act like a thirteen-year-old with a self-esteem problem.
The theory of relativity doesn't work right in Arkansas.
I don't have data, but this statement has been repeated so much, by so many people I thought it was accepted as fact.
Like no two fingerprints are the same. Or no two snowflakes are the same.
There is no proof of such uniqueness, but it is generally accepted as fact.
Comment removed based on user account deletion
See the section marked 'Aren't These Cheats Just Optimizations...' here. These aren't driver optimisations. The drivers are claiming to do work they are not actually doing.
Oceania has always been at war with Eastasia.
I'm not gonna BS, I've owned loads of different brand gfx cards, and if one understands the standpoints of the different companies you will also KNOW what you are getting for your money.
/bin ? might as well put that card in the bin while I'm at it. ;)
Matrox stands for quality analogue image and is usually the only budget choice for people working with art (thus they stay away from selling chips to anyone else)
NVidia stands for cheap 3d graphics, and the larger part of their vendors would be considered "budget".
ATI is somewhere in between, offering good 3d graphics and relatively good quality too.
(and even though they now supply chips to anyone, there's a few quality vendors like Hercules to their merit)
About drivers, I'd go as far as to say that none of the above mfg have had any real trouble with drivers. (only concerning windos)
The "problematic" ATI drivers is a rumour from pre 1995 (mach64 and earlier) it would be better described as "confusing" drivers rather than a real problem (supplying and allowing users to choose the wrong driver)
It was partly MS's fault too, (automatically installing their not-so compatible drivers)
In my opinion both Matrox and ATI drivers have recently been better from a usability point of view. Easy to use, easy to understand with less menus and settings and still getting away with it without causing harm.
I choose ATI of the simple reason that there are open source drivers available from DRI-project, and I prefer their style. (Especially as they are the last major vendor to stay somewhat VESA compliant)
Why don't we thank gamers for buying all that expensive HW and making development of GFX cards go faster?
Or why don't we thank them for budget GFX cards that cost 4x more than better quality cards used to, just because we have to buy that expensive 3d gpu forcibly placed on them, especially as most normal people don't need it?
They all seem to forget.. primitive 3d graphics aren't the most important technologies of desktop computer graphics, we will soon toss all of this nonsense for voxels and volume graphics which might even have scientific purposes. (just like we did vector (2d primitive) graphics for bitmaps)
Also, for some people a more important aspect is the visual quality of the TV-out (ATI wins, no question) or the quality and performance of video input, (Both Matrox and ATI offer good budget options for this)
No points to nvidia from me since my scale does not place any value for 4500fps in quake3 nor does it place it on having 250 options for setting different aspects of 3d graphic options and obscuring the (for me more important) TV-out settings.
Last but not least... binaries.. wtf? can I choose? no? don't those go into
my 24 cents