Slashdot Mirror
NVidia Accused of Inflating Benchmarks
Junky191 writes "With the NVidia GeForce FX 5900 recently released, this new high-end card seems to beat out ATI's 9800 pro, yet things are not as they appear. NVidia seems to be cheating on their drivers, inflating benchmark scores by cutting corners and causing scenes to be rendered improperly. Check out the ExtremeTech test results (especially their screenshots of garbled frames)."
To bad Nvidia has to resort to these things to keep selling there cards.. The used to be great.. but now i have my doubts..
Isn't this SOP for the entire video card industry? Every few years someone gets caught targeting some aspect of performance to the prevailing benchmarks. I guess that's what happens when people wax on about "my video card does 45300 fps in quake and yours only does 45292, your card sucks, my experience is soooo much better". For a while now it's been the ultimate hype driven market wrt hardware.
Well they got caught...they obviously arnt to good at it, after all they did get caught
I dont know why anyone ever cheats on benchmarks...how could you ever get away with it? do you really think no one is going to do their own benchmark? Come on. This is probably one of those most retarded things I have ever seen a company do.
Oh well, Nvidia is getting to the point were they are going to have beat out ATI at some point if they want to survive
I looked at the photos, and it seems to me to be just a driver fuckup on the 3dmark benchmarks.
Since when did rendering errors caused by driver problems become "proof" of a vendor inflating benchmarks?
And this story was composed by someone with the qualifications of "Website content creator, who likes video games alot" not a driver writer, not anyone technically inclined beyond the typical geek who plays alot of video games and writes for a website called "EXTREME tech" because you know, their name makes them extreme!
note: I'm not an Nvidia fanboy, i just bought an ATI Radeon 9500, so I am just a skeptic of incredulous, idiotic derivations of fact, when all he has are some screenshots of a driver screwing up the render of a scene.
Read this article NVIDIA's Back with NV35 - GeForceFX 5900 Ultra
3Dmark03 may be inflated but what counts is real world game benching. And FX 5900 wins over ATI in all but Comanche 4.
Interesting ehh?
"Engineers do the work of man, Physicists do the work of God"
They just hired some ATI engineers.
nVidia has been one of the more customer friendly video card makers...ever. They have full support for all platforms from Windows to Macs to Linux, this makes them, to me, one of the best companies around.
So now they are falling into the power trap of "we need to be better and faster then the others" which is only going to have them end up like 3DFX in the end. Cutting corners is NOT the way to gain consumer support.
As I look at it, it doesn't matter if your the fastest or not...it's the wide variety of platform support that has made them the best. ATi does make better hardware but their software (drivers) are terrible and not very well supported. If ATi would get the support that nVidia has been giving for the last few years, I would start using ATi hands down...It's the platform support that I require, not speed.
"Some things have to be believed to be seen." - Ralph Hodgson
"Because nVidia is not currently a member of FutureMark's beta program, it does not have access to the developer version of 3DMark2003 that we used to uncover these issues."
Wow, some prelease software is having issues with the new brand-new drivers? Who would have thought... Why not wait for official release of the software and the drivers before making hasty conclusions?
In addition, who really cares about 3DMark? Why not use time which is wasted on 3DMark benchmark for benchmarking real games? After all 60fps tells a lot more about performance than 5784 3DMarks.
You make an excellent point. I am tired of spending way too much money trying to reach that holy grail of gaming. The slight improvement in hardware isn't going to change the fact that I'm only a mediocre gamer. The best gamers are going to kick my ass regardless of what hardware they use. I don't need to spend $400 every six months to be reminded of that.
I recall about 10 years ago that one of the video adaptor manufacturers optimised their Windows 3.1 acclerated video drivers to give the best performance possible with the benchmark program Ziff-Davis used for their reviews.
One test involved writing a text string in a particular font continuously to the screen in. This text string was encoded directly in the driver for speed. Similarly one of the polygon drawing routines was optimised for the particular polygons used in this benchmark.
Back in the day, Voodoo cards were the fastest (non-pro) cards around when they first came out. A significant subset of users became Voodoo fanboys, which was ok, since Voodoo was the best.
;^)
Voodoo was beaten squarely by other, better video cards in short order. The fanboys kept buying Voodoo cards, and we all know what happened to them
GeForce cards appeared. They were the best. They have their fanboys. Radeon cards are slowly becoming the "other, better" cards now.
Interesting....
(I'm not sure what point I was trying to make. I'm not saying that nVidia will suck, or that Radeon cards are the best-o. The moral of this story is: fanboys suck, no matter their orientation.)
GeekNights!
Late Night Radio for Geeks!
The problem is that people are buying cards based on these silly synthetic benchmarks. When performance in one arbitrary set of tests is so important to sales, naturally you're going to see drivers tailored to improving performance in those tests.
Of course, if Nvidia's drivers were released under the GPL, none of the mud from this would stick as they could just point to the source code and say "look, no tricks". As it is, we just get a nasty combination of the murky world of benchmarks and the murky world of modern 3D graphics.
Well, to tell you the truth...I LIKE application specific optimization as long as it is general purpose enough to be applied across the board to that application. However, in this case, the corners are cut in a benchmark and are targetted SPECIFICALLY at the scene as rendered in the benchmark. If ATI had done the same thing in Quake, the pre-recorded timedemos would be faster, but not actual gameplay...that wasn't the case, the game itself was rendered faster. The only poor choice they made was how they recognized that Quake was what was being ran, optimizing a specific rendering path would have been more general purpose and have seemed a lot less like cheating.
This on the other hand, if true, could be construed as NOTHING BUT cheating. Especially when coming from a company that said they didn't support 3Dmark 2003 because it was possible for companies to optimize their drivers specifically FOR such benchmarks...well, they proved their point.
One has to take all benchmarks with a grain of salt if they come from a party with financial interestes in the product. Win 2K server outperforms Linux, a Mac is 2x the speed of the fastest Wintel box, my daddy can beat up your daddy..
It's not suprising but it is somewhat disappointing.
Trolling is a art,
I bought the first 64MB DDR Radeon right after it came out. I held on to the card for months waiting for ATI to release a driver, didn't happen. I heard of people having sucess getting 3D acceleration to work, but I could never duplicate that success.
Finally after months of waiting I traded my Radeon to my roommate and got a GeForce 2 Pro with 64MB of DDR. Runs beutifully on Linux, I even play UT2K3 with it on an Athlon 850. Finally after having the GeForce2 for about four months I happen across a site that tells me how to make 3D acceleration work for the Radeon. To late now, I'm happy with my GeForce, and UT2K3 seems to only really want to work with nVidia anyways.
I don't think drivers are the best way to defend ATI considering they tend to shrug off other OS's and nVidia has committed themselves to supporting Alternate OS's.
The preceding post was not a Slashvertisement.
Since upon reading the article it even states that nVidia don't have access to the version of 3dmark2003 (not on the beta team) so they can have errors between the drivers and the code for 3dmark and not know. This is the kind of thing that can happen, and will take a driver update to fix, but does not necessarily mean they are doing anything wrong.
As someone who has always been impressed by nVidia's driver updates and the benefits they can give each time, I am going to wait to see if it really is something bad they are doing deliberately before changing my opinion of them.
There is, at the moment, no real evidence in anyones favour.
Targetting performance for benchmarks is one thing.
These drivers were written with specific limits built in that make the drivers COMPLETELY irrelevant to ordinary gaming, as ET demonstrates by moving the camera just a bit from the designated path.
This would be like chopping the top off of a car to make it lighter, to reduce the distance it takes for it decellerate in a brake test. Or compensating for a crappy time off the starting line by removing the back half of the car and bolting a couple of RATO rockets where the back seats used to be. Or loading the car up with nitro, or something. You think Car and Driver Magazine wouldn't say something?
These drivers make the card completely unsuitable for ordinary gaming. They aren't 'more powerful' -- they are a completely altered version of the drivers that are ONLY good at improving one particular set of benchmarks.
Please try reading the article in more detail.
The developer version is not a pre-release, it's the same version with some extra features that let you debug things, change scenes, etc.
As soon as you move the camera away from it's usual benchmark path, you can see that nVidia hard-coded clipping of the benchmark scenes to make it do less work than it would need to in a real game, where you don't know where the camera will be in advance.
As I mentioned in another post, it's a step in the direction of recording an mpeg of the benchmark and playing it at a high fps rate.
Never underestimate the bandwidth of a 747 filled with CD-ROMs.
Overclockers.com has a very well thought out Editorial on this issue titled ""Trust is Earned" It is well worth the read.
Why is it that people are assessing the performance of cards based on running the same narrow set of benchmarks each time? Of _course_ if you do that then performance optimization will be narrowly focused towards those benchmarks. Not just on the level of blatant cheating (recording a particular hardcoded text string or clipping plane) but more subtle things like only optimizing one particular code path because that's the only one the benchmark exercises.
More importantly why is any benchmark rendering the exact same scene each time? Nobody would test an FPU based on how many times per second it could take the square root of seven. You need to generate thousands, millions of different scenes and render them all. Optionally, the benchmark could generate the scenes at random, saving the random seed so the results are reproducible and results can be compared.
-- Ed Avis ed@membled.com
Show me the Quack 3 Arena benchmarks! Then we'll decide which card is the best!
Damn, a few years ago ATI did a similar thing to the drivers with the Xpert@play cards. The cards got good benchmarks that never held up once people actually played the games. They got beat up pretty bad for it at the time. Now it looks like nVidia's turn.
Do really dense people warp space more than others?
I would suspect something like this too... I'm not a 3D card expert, but from what I understood the way the "cheating" was found was by stopping the whole scene, freezing everything going on (including all processing of culling information). When you then start rotating the camera around, you are supposed to get rendering anomalities, since the scene is optimised to be viewed from a different angle. Why this happens with the geforce only I dont know, but I would guess that its because nvidia and ati drivers and cards work very differently since they are designed by very different people. Though of course, it is possible that nvidia would be "cheating" in driver level, but before doing that kind of accusations they should get solid proof, and especially let nvidia give their own explanation first. Then again, if this is a feature, happening because of some advanced optimization in the card/drivers nvidia propably doesnt want to give an accurate explanation since that would reveal the method for its competitors to use.
The article talks about possible solutions to the problem of "repeatability" while still avoiding the problem of cheating in the way alleged here. I don't remember it mentioning this possible solution though: How about if the camera was controlled by a mathematical function of a seed given by hand. Like you'd seed a PRNG.
This way you could repeat the benchmarks by giving the same seed. Generate a 'default one' at each new install (this to ensure clueless reviewers get a new seed). Make it easy to enter a new one or generate a random one.
The explosion of possible views (if implemented correctly) would make it all but impossible to cheat in the way alleged, no?
Belief is the currency of delusion.
Yeah, but they all do it, and it isn't strictly video board manufacturers either. That '80 GB' hard drive you just bought isn't 80 GB, it's (depending on the manufacturer) either a 80,000,000,000 byte hard drive or a 80,000 MB hard drive...either way it isn't by any stretch of imagination 80 GB. That Ultra DMA 133 hard drive, BTW, can't really do a sustained 133 MB/s transfer rate either, that's the burst speed and you'll probably NEVER actually achieve that transfer rate in actual use. That 20" CRT you just bought isn't 20", it's 19.2" inches of viewable area. A 333 MHZ FSB isn't 333 MHZ, it's 332-point-something mhz, and even then it isn't really 333 MHZ because it's really like 166 mhz and doubled because DDR memory allows you to read and write on the high and low side of the clock. That 2400 DPI scanner you just bought is only 2400 DPI with software interpolation. Your 56K modem can really only do 53K due the FCC regulations requiring them to disable the 56K transfer rate. The list goes on.
My journal has hot
20zillion transactions per second provided you have a massive parallel Alpha with 1024 processors and 256 TB of physical memory for just 23.99$ per transaction assuming that you found your massive parallel Alpha on a heap of scrap metal.
ich bin der musikant
mit taschenrechner in der hand
kraftwerk
The 3dmark03 benchmark is cheating in the first place, implementing stencil shadows in two of the game tests in such a braindead manner which no sane programmer would put in an actual game.
It also uses ATI-only pixel shaders 1.4, and reverts to dual-pass on other cards.
Why all this?
NVIDIA isn't on the 3dmark03 beta program (read: didn't pay FutureMark a hefty lump of greenbacks).
But then the benchmark would be useless, unless you repeated it a few dozen times and averaged the results.
By sheer luck, card A could get a 'rail' that drags it along a plain brick wall with nothing fancy to render, and card B could go through the heart of some mega explosion with fragments and fire and smoke and all that. Card A would get 4000000 fps, card B gets 20.
It would be fine to take them off the rails to "keep em honest", but you need to run both cards in the exact same situation for your test to have any sort of merit at all.
I don't need no instructions to know how to rock!!!!
So, because he isn't interested in this boring, repetative, inane and stupid ego-massaging 'my computer is more 1337 then yours' willy waving competition his opinion is invalid?
The trouble with free speech is that everyone has it.
http://twitter.com/onion2k
My personal favorite from this article:
:-)
nVidia believes that the GeForceFX 5900 Ultra is trying to do intelligent culling and clipping to reduce its rendering workload
It's alive !
Hello! I'm a disaster waiting to happen!
It's rude, but also true.
Benchmarks, even so-called 'real-world' benchmarks, are a poor indicator of system performance. Sites like Tom's Hardware and Anandtech exist as a kind of group therapy for hardcore gamers and 'performance enthuiasists'. You know if you read their "technical" articles that they understand as much about the inner workings of a computer as the rice rocket driver with the huge spoiler and chrome wheel covers understands about his car's engine.
These sites always have an incestuous relationship with their advertisers, they don't know anything about statistics, the scientific method, or how valid data is gleaned and collected.
Even ArsTechnica has tons of articles that pass off conjecture as fact (case in point: the latest PPC970 article). While their writers seem more technically knowledgeable, it's still deceipt.
Benchmark and "performance enthusiast" sites are a con job, plain and simple. They should be treated as what they are, the "EZ WEIGHT LOSS PLAN!!!!" scams of the geek community.
(-1, Raw and Uncut is the only way to read)
hardocp.com on the front page has a great writeup on this.
But basically, extremetek is just a little bit mad because they were excluded from the doom3 benchmarks. Since nvidia refused to pay the 10s of thousands of dollars to be a member of the 3dmark03 board, they have absolutely no access to the software used to create this bug.
Here is the full exept from hardocp.com:
3DMark Invalid?
Two days after Extremetech was not given the opportunity to benchmark DOOM3, they come out swinging heavy charges of NVIDIA intentionally inflating benchmark scores in 3DMark03. What is interesting here is that Extremetech uses tools not at NVIDIA's disposal to uncover the reason behind the score inflations. These tools are not "given" to NVIDIA anymore as the will not pay the tens of thousands of dollars required to be on the "beta program" for 3DMark "membership".
nVidia believes that the GeForceFX 5900 Ultra is trying to do intelligent culling and clipping to reduce its rendering workload, but that the code may be performing some incorrect operations. Because nVidia is not currently a member of FutureMark's beta program, it does not have access to the developer version of 3DMark2003 that we used to uncover these issues.
I am pretty sure you will see many uninformed sites jumping on the news reporting bandwagon today with "NVIDIA Cheating" headlines. Give me a moment to hit this from a different angle.
First off it is heavily rumored that Extremetech is very upset with NVIDIA at the moment as they were excluded from the DOOM3 benchmarks on Monday and that a bit of angst might have precipitated the article at ET, as I was told about their research a while ago. They have made this statement:
We believe nVidia may be unfairly reducing the benchmark workload to increase its score on 3DMark2003. nVidia, as we've stated above, is attributing what we found to a bug in their driver.
Finding a driver bug is one thing, but concluding motive is another.
Conversely, our own Brent Justice found a NVIDIA driver bug last week using our UT2K3 benchmark that slanted the scores heavily towards ATI. Are we to conclude that NVIDIA was unfairly increasing the workload to decrease its UT2K3 score? I have a feeling that Et has some motives of their own that might make a good story.
Please don't misunderstand me. Et has done some good work here. I am not in a position to conclude motive in their actions, but one thing is for sure.
3DMark03 scores generated by the game demos are far from valid in our opinion. Our reviewers have now been instructed to not use any of the 3DMark03 game demos in card evaluations, as those are the section of the test that would be focused on for optimizations. I think this just goes a bit further showing how worthless the 3DMark bulk score really is.
The first thing that came to mind when I heard about this, was to wonder if NVIDIA was not doing it on purpose to invalidate the 3DMark03 scores by showing how the it could be easily manipulated.
Thanks for reading our thoughts; I wanted to share with you a bit different angle than all those guys that will be sharing with you their in-depth "NVIDIA CHEATING" posts. While our thoughts on this will surely upset some of you, especially the fanATIics, I hope that it will at least let you possibly look at a clouded issue through from a different perspective.
Further on the topics of benchmarks, we addressed them earlier this year, which you might find to be an interesting read.
We have also shared the following documentation with ATI and NVIDIA while working with both of them to hopefully start getting better and more in-game benchmarking tools. Please feel free to take the documentation below and use it as you see fit. If you need a Word document, please drop me a mail and let me know what you are trying to do please.
Benchmarking Benefiting Gamers
Objective: To gain reliable benchmarking and image quality tools
So who cares? It matters little to me how fast something is in a synthetic benchmark if there is no correlation to real world applications, and I am sure Nvidia isnt doing this in games cause who would buy a card that didnt properly render most scenes.
I dunno, but synthetic benchmarks seem a bit irrelevant as does what Nvidia does in them. Show me how many FPS it gets in Q3A, that I care about.
Companies always tweak their code, insist on tests optimized for their hardware, etc. in order to get an edge up on benchmarks. This is probably especially true in cases where the competition is so neck-and-neck, as it seems to be with the video card industry. It seems that these companies will do anything to show they can get even two or three more FPS than the competition. It is hard to treat any benchmark seriously because of this.
At the same time, I'm debating what my next video card should be. Even though ATI's hardware might be slightly better this round, the differences will probably be negligable to all but the most extreme gamers. At the same time NVidia has proven to me that they have a history of writing good drivers, and they still provide significantly better support to the Linux community than ATI does.
For this reason I'm still siding with the GeForce family of video cards.
"You spoony bard!" -Tellah
Did you see what they had to do to "prove" the cheat? Read the article. In other game tests the card beats the ATI 9800PRO so obviously it is faster. (see anandtech, hardocp, tom's hardware, etc if you really care).
The things that they're being accused of reduce work to the graphics engine - and doesn't affect image quality - it's called OPTIMIZATION. The fastest frame rate with the best image quality.
Man someone must have spent hours in front of their computer coming up with a way to get a sensational story like this. ATI has done it, and so does everyone else but what sucks is that this "news" is being flogged everywhere like it's the most incredible piece of news ever.
In this case it's not ANYWHERE NEAR as bad as changing the card's performance based on the name of the program that's being run - I think most people remember that one.
In this case it's a non-story. And yes, we all pay too much attention to benchmarks. I am now one to two generations behind leading edge and plan to stay there. It's far less expensive than driving a new car of the lot every four months.
This sig contains a manual self-destruct. Kindly please put your foot through your monitor in 8 seconds.
On the whole scene being rendered correctly:
It is perfectly possable ot read the graphics data from the card and write it to a file, like a tiff. In fact, I've seen some benchmarking programs that do. Then what you can do, for DirectX at any rate, is compare against a reference renderer. The development version of DX has a full software renderer built in that can do everything. It is slow as hell, being a pure software implementation, but also 100% 'correct' being that it is how DirectX intends for stuff to be rendered.
Well, if you have a benchmark that includes images from the reference renderer, you can then compare those to the current renderer. Aside from just looking at them, you can do mathematical calculations of the images to see where and how they differ. A simple one would just be a straight XOR on all the pixels. If the current renderer got the same result as the reference renderer, you'll get black as a result (since anything XORed with itself is 0). Any time there is a difference, it will show up as a soloured pixel, and the more colour, the more it was different. I've seen a benchmark do this but I don't remember which one.
Not saying that this is the perfect, end-all solution for graphics cards, but there ARE ways that they can be tested versus some kind of reference.
Video performance from my Radeon 7500 under Linux (using the ATI optimized drivers for XFree86 4.3) is not nearly as good as the ATI-provided drivers under Windows 2000. I think ATI gives the type of ingredients to the Linux driver developers, but the quantity of those ingredients it keeps to themselves.
nVidia could really follow along this same philosophy, instead of hearing the massive complaints from their oft-buggy video driver.
Ayup
Companies have long adopted the "open-source" fundamental philosophy even before Linux and what I call the modern open source movement caught on. Often, a company would have a nice product - license the code to a sub-company (who would modify/repackage/etc the original product). The license agreement stipulated that all modifications would 1) have to be reviewed by the company without restriction from the sub-company 2) the modifications would have to be approved by the company.
Take for instance the relationship between Microsoft and IBM during the OS/2 era. The two companies working on the same code base produced OS/2 and, eventually, the NT kernel.
Or, more recently - the brilliant strategy of Netscape Communications Corporation - the birth of the Mozilla project. To the open source community - take our browser, modify it like hell, make it a better project. You have, of course, Mozilla as the browser - but Netscape (Navigator) still exists (as a repackaged, "enhanced" Mozilla).
nVidia's source code release would have two major impacts as far as their performance goes.
1) ATI (et al.) would find the actual software-based enhancements they could also incorporate into their own driver to improve their product.
2) nVidia could capture the many brilliant software developers that happen to be a part of the whole nVidia "cult" - this could lead to significant advancements to their driver quality (and overall product quality).
My guess is that the lid is kept so tightly shut on nVidia's drivers because they can keep their chips relatively simple through their complex software driver. ATI, perhaps, has the technical edge in the hardware arena, but does not have the finesse for software enhancing drivers like nVidia does.
Ayup
The point I was making is simply this - if they cheated or did not cheat on the benchmarks, does it really make a difference? For some, sure. But for me and probably a good chunk of people out there, the slight extra edge that NVIDIA may or may not have given themselves in this benchmark isn't going to be enough to make me run out and purchase the new geforce over the radeon unless I wanted to particpate in the "I have the fastest graphics card available as of 3:00 this afternoon" pissing contest. The few extra FPS nvidia can boast by rigging this benchmark will not help me become a better gamer, nor will it help most people become better gamers. So what's the point of becoming enraged over something like this? Even if you are one of the lucky few who can tell the difference between a great card and a slightly less great card, has this really altered your opinion so much of your choice of video cards?
Reading the posts, I dont think everyone is understanding the point of the rail test.
Using the rail test, Nvidia excluded almost all non-visible data. This shows nvidia tweaked its drivers to only render data seen on the rail test, which would only happen if you tweak your drivers for the benchmarks. (aka the cheat)
I like it better if benchmarks uses average FPS on a game, and you go PLAY the game, and watch for yourself.
Try 1024x768/1280x1240/1600x1200 with all AA/AF modes. Also stop using 3ghz P4's for the benchmarks, use a mix of 1ghz/2ghz/3ghz AMD/Intel boxes so we can know if the hardware is worth the upgrade.
I believe my 19.2" viewable-area monitor is a twenty-ONE inch monitor, thank-you-very-much!
With the Riva128, back when I had a 3Dfx Voodoo (or Voodoo2).
They garbled texture maps to achieve a higher transfer rate and frame rate. Then they went legit for the TNT line.
I guess the belief "if you can't win, cheat" is still there at nvidia.
I wonder if ATi makes a good Linux driver...
Some compilers miscomplied the modified benchmark, because they recognized the code as the standard benchmark even though it wasn't exactly the same.
(Anybody have a reference for this? I heard the author give a talk at Stanford years ago.)
Nvidia's current problems sound familiar don't they? 3DFX started floundering once they made it to the top, and started worrying more about profit margin and market share than putting out the best video cards. If they keep this behavior up, I give it two years before ATI starts looking at buying them out.
giga = 10^9, and an 80 GB hard drive has 80 x 10^9 (10 billion) bytes. This is standard notation that has been in use for at least a hundred years. Perhaps what you're looking for is 80 GiB, which the hard drives are not advertised as.
This is standard even in most other parts of computing (anything engineering-oriented especially). For example, that 128kbps mp3 you downloaded is 128000 bits/second, not 128*1024 bits/second.
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
NVida has always stood silent in the race to win benchmarks. Fact: Every video card manufacturer tweeks drivers specificly for benchmarks. ATI scammed people into a 50% performance increase years ago with a new set of drivers. This of course was completely false.
Fact: NVidia is probably the last company to join in this race. when they denouced the use of Futuremarks programs after 3DMark2k3 showed undeserved favorability towards ATI's driver set they were ostracized for not being a big player. It seems to me that they finally said "fuck it, the public wants bullshit drivers that inflate thier benchmarks, then we will give it to them!"
Good for NVidia. They always have been, and for the forseeable future always will be the no compromise 3d gaming solution.
The funniest part of this all is this, in unreal2k3 I personally have seen a 160/80 flyby/botmatch score jump up to 220/103 on a 5800FX based AMD1700+ system. So the drivers are not complete bullshit. Unlike ATI who was chastised in the past for having lower game scores after the fact.
First, faster video cards are not designed to make you a better gamer, they are designed to make your gaming experience better. If they are not doing that for you, then you're not playing the games that need the improvement, and you don't need the card. Which, I'm sure, is true for a lot of people out there.
On the other hand, ATI sold over 1 million Radeon 9700s in first few months of it being out, so there are definitely a lot of people out there who do need and want the best card the money can buy.
So, that gets us to your question of whether nvdia cheating really makes a difference. Obviously, it doesn't make a difference to you, because you don't want the buy any of the high-end cards in the first place. It should be obvious in the same way, though, that it does make a big difference to somebody who will buy a high end card.
If 9800 and FX5900 have the same price, and speed is what you're after (and it should be, since you're buying these cards), then you want to buy the faster one. The only way to figure out which one is faster is to check the benchmark results (unless you buy both and try them tyourself). If one of the companies cheated in a benchmark, they have tricked you into thinking that you're buying a faster card, while you're really buying a slower one.
Imagine you're picking between two equally expensive cars, and you want to buy the faster of the two. One claims to do 0-60 in 5s, and the other claims to do it in 3s. You'll go ahead and buy the latter one, only to learn later that they were testing the car going downhill while the other was accelerating on level ground! I think enraged would only begin to describe your reaction to that.
Never underestimate the bandwidth of a 747 filled with CD-ROMs.