Slashdot Mirror
NVidia Accused of Inflating Benchmarks
Junky191 writes "With the NVidia GeForce FX 5900 recently released, this new high-end card seems to beat out ATI's 9800 pro, yet things are not as they appear. NVidia seems to be cheating on their drivers, inflating benchmark scores by cutting corners and causing scenes to be rendered improperly. Check out the ExtremeTech test results (especially their screenshots of garbled frames)."
To bad Nvidia has to resort to these things to keep selling there cards.. The used to be great.. but now i have my doubts..
Isn't this SOP for the entire video card industry? Every few years someone gets caught targeting some aspect of performance to the prevailing benchmarks. I guess that's what happens when people wax on about "my video card does 45300 fps in quake and yours only does 45292, your card sucks, my experience is soooo much better". For a while now it's been the ultimate hype driven market wrt hardware.
Well they got caught...they obviously arnt to good at it, after all they did get caught
I dont know why anyone ever cheats on benchmarks...how could you ever get away with it? do you really think no one is going to do their own benchmark? Come on. This is probably one of those most retarded things I have ever seen a company do.
Oh well, Nvidia is getting to the point were they are going to have beat out ATI at some point if they want to survive
I looked at the photos, and it seems to me to be just a driver fuckup on the 3dmark benchmarks.
Since when did rendering errors caused by driver problems become "proof" of a vendor inflating benchmarks?
And this story was composed by someone with the qualifications of "Website content creator, who likes video games alot" not a driver writer, not anyone technically inclined beyond the typical geek who plays alot of video games and writes for a website called "EXTREME tech" because you know, their name makes them extreme!
note: I'm not an Nvidia fanboy, i just bought an ATI Radeon 9500, so I am just a skeptic of incredulous, idiotic derivations of fact, when all he has are some screenshots of a driver screwing up the render of a scene.
anyone tryed to get a comment from NVidia ? ... from all the guys on earth doing reviews of the latest FXs only these ones found this ... seems spooky to me ... Let 's hope that at least ATI is not involved ... FUD you know : also happens with Linux ...
Read this article NVIDIA's Back with NV35 - GeForceFX 5900 Ultra
3Dmark03 may be inflated but what counts is real world game benching. And FX 5900 wins over ATI in all but Comanche 4.
Interesting ehh?
"Engineers do the work of man, Physicists do the work of God"
They just hired some ATI engineers.
Surprisingly, most people didn't flinch when ATI did it. (Remember the Quake.exe vs Quack.exe story?)
nVidia has been one of the more customer friendly video card makers...ever. They have full support for all platforms from Windows to Macs to Linux, this makes them, to me, one of the best companies around.
So now they are falling into the power trap of "we need to be better and faster then the others" which is only going to have them end up like 3DFX in the end. Cutting corners is NOT the way to gain consumer support.
As I look at it, it doesn't matter if your the fastest or not...it's the wide variety of platform support that has made them the best. ATi does make better hardware but their software (drivers) are terrible and not very well supported. If ATi would get the support that nVidia has been giving for the last few years, I would start using ATi hands down...It's the platform support that I require, not speed.
"Some things have to be believed to be seen." - Ralph Hodgson
they should give me a new video card... how the FUCK am i going to play half life 2, doom 3, etc etc.
If I recall right my ati with 4MB(or 2MB) of memory beated nVidia TNT in speedtests.. though most of the output was black screens :)
telax - Just another vim and c hacker.
Who doesn't???
"Because nVidia is not currently a member of FutureMark's beta program, it does not have access to the developer version of 3DMark2003 that we used to uncover these issues."
Wow, some prelease software is having issues with the new brand-new drivers? Who would have thought... Why not wait for official release of the software and the drivers before making hasty conclusions?
In addition, who really cares about 3DMark? Why not use time which is wasted on 3DMark benchmark for benchmarking real games? After all 60fps tells a lot more about performance than 5784 3DMarks.
Anyone remember when Intel did this a few years ago with motherboard chipsets? Programs like HDTach got insane benchamrks with their chips.
You make an excellent point. I am tired of spending way too much money trying to reach that holy grail of gaming. The slight improvement in hardware isn't going to change the fact that I'm only a mediocre gamer. The best gamers are going to kick my ass regardless of what hardware they use. I don't need to spend $400 every six months to be reminded of that.
I recall about 10 years ago that one of the video adaptor manufacturers optimised their Windows 3.1 acclerated video drivers to give the best performance possible with the benchmark program Ziff-Davis used for their reviews.
One test involved writing a text string in a particular font continuously to the screen in. This text string was encoded directly in the driver for speed. Similarly one of the polygon drawing routines was optimised for the particular polygons used in this benchmark.
Back in the day, Voodoo cards were the fastest (non-pro) cards around when they first came out. A significant subset of users became Voodoo fanboys, which was ok, since Voodoo was the best.
;^)
Voodoo was beaten squarely by other, better video cards in short order. The fanboys kept buying Voodoo cards, and we all know what happened to them
GeForce cards appeared. They were the best. They have their fanboys. Radeon cards are slowly becoming the "other, better" cards now.
Interesting....
(I'm not sure what point I was trying to make. I'm not saying that nVidia will suck, or that Radeon cards are the best-o. The moral of this story is: fanboys suck, no matter their orientation.)
GeekNights!
Late Night Radio for Geeks!
Investigate on what? On how to make up excuses "this is an unexpected irregularity of the driver"? This is ridiculous.
It's clearly a deliberate attempt. But it looks like NV's going to deny responsibility on this one.
Shame on them...
The problem is that people are buying cards based on these silly synthetic benchmarks. When performance in one arbitrary set of tests is so important to sales, naturally you're going to see drivers tailored to improving performance in those tests.
Of course, if Nvidia's drivers were released under the GPL, none of the mud from this would stick as they could just point to the source code and say "look, no tricks". As it is, we just get a nasty combination of the murky world of benchmarks and the murky world of modern 3D graphics.
My suspicion is the benchmark is giving (incorrect) culling info to the driver. ATI's driver ignores it, and Nvidia honors it.
Reviews should try to uncover it and find out who does it right now which is the only thing that really matters when getting a product.
The whole Quake / Quack fiasco for ATI was enlightening, but does anyone know if ATI does this currently?
Frame rates are overrated anyway, since people buying these cards are buying new ones before their current ones go down to noticable frame rates. Features, picture quality and noise is what matters.
ATI seems to still have the upper hand, and at least for ATI cards there is some free drivers for Linux that can handle 3d-acceleration.
Benchmarks are nothing else than statistics: In order to get to a (more or less) meaningful benchmark, you repeat the same process over and over, possibly in different environments. Then you analyze the results, resulting in a statistic of whatever you've benchmarked.
Therefore, the old Disraeli saying applies: "There are lies, damn lies, and statistics."
Or, to essentially say the same thing without expletives: Never trust a statistic you haven't faked yourself.
They are NFlating benchmarks? ;)
But wtf am I thinking. This is /. Hundreds of idiots post without reading the article and many get modded up.
Liberty.
One has to take all benchmarks with a grain of salt if they come from a party with financial interestes in the product. Win 2K server outperforms Linux, a Mac is 2x the speed of the fastest Wintel box, my daddy can beat up your daddy..
It's not suprising but it is somewhat disappointing.
Trolling is a art,
Why do you feel obligated to post the "I don't care about the zillion fps in quake"? Do you post a similar message to every story that you don't care about?
This is a big deal to people who care -- it insults the reviewers who spent hours benchmarking their card, and it insults the users who bought/will buy their card. There are people who care, and people who do want the fastest card for a reason, and they are interested to hear from other people who care, and not the people who don't!
Never underestimate the bandwidth of a 747 filled with CD-ROMs.
Lies, Damn Lies, and Marketing
"Not knowing when the dawn will come, I open every door." - Emily Dickinson
They are a member of the beta program and therefore have access to the developer version of the benchmark. They are not using a beta of the benchmark (as I understand it), but a developer version. There's a difference. See: Beta. Developer. They're even spelled differently. Your second comment is addressed in the article, which I'm assuming that you read, right?
Ironic. This is the same thing ATI did about a year or two ago. ATI used those special quake3 drivers that increased preformance *only* in q3.
It's a shame that companies resort to lying to consumers.
ATI was caught optimizing Quake3. In theory, this is a *good* thing. Quake3 is used by a lot of people, and was/is the engine for many of the games that people buy top end video cards for.
I'm sure nVidia does the same thing: new Detonator driver releases have been known to get amazing improvements for specific games.
ATI screwed up by affecting the visual quality. Well, screwing up visual quality would be acceptable if there was a documented setting to turn that particular optimization off, but there wasn't, so public chastisement followed.
In other words, it was an implementation problem. It sucks, but I write software for a living, and I can guarantee that every piece of software I have in the wild has at least one bug.
nVidia was caught optimizing benchmarks. No excuse. A public flaying is in order.
Bryan
Since upon reading the article it even states that nVidia don't have access to the version of 3dmark2003 (not on the beta team) so they can have errors between the drivers and the code for 3dmark and not know. This is the kind of thing that can happen, and will take a driver update to fix, but does not necessarily mean they are doing anything wrong.
As someone who has always been impressed by nVidia's driver updates and the benefits they can give each time, I am going to wait to see if it really is something bad they are doing deliberately before changing my opinion of them.
There is, at the moment, no real evidence in anyones favour.
Targetting performance for benchmarks is one thing.
These drivers were written with specific limits built in that make the drivers COMPLETELY irrelevant to ordinary gaming, as ET demonstrates by moving the camera just a bit from the designated path.
This would be like chopping the top off of a car to make it lighter, to reduce the distance it takes for it decellerate in a brake test. Or compensating for a crappy time off the starting line by removing the back half of the car and bolting a couple of RATO rockets where the back seats used to be. Or loading the car up with nitro, or something. You think Car and Driver Magazine wouldn't say something?
These drivers make the card completely unsuitable for ordinary gaming. They aren't 'more powerful' -- they are a completely altered version of the drivers that are ONLY good at improving one particular set of benchmarks.
An interesting variation on the standard graphics-card article troll.Personally I can see at 34.152 fps on a good day, but can only manage 28.693 when I'm tired.
Please try reading the article in more detail.
The developer version is not a pre-release, it's the same version with some extra features that let you debug things, change scenes, etc.
As soon as you move the camera away from it's usual benchmark path, you can see that nVidia hard-coded clipping of the benchmark scenes to make it do less work than it would need to in a real game, where you don't know where the camera will be in advance.
As I mentioned in another post, it's a step in the direction of recording an mpeg of the benchmark and playing it at a high fps rate.
Never underestimate the bandwidth of a 747 filled with CD-ROMs.
How difficult would it be to have a "random rail" generator? This would be fair for review purposes, just generate a "random rail" path for the specific review and run the benchmark with each card. This is essentially what they did to discover the "driver bug" anyway, so why not make that a 3D benchmark feature?
+1, Quite Funny
That was classic intercourse!
Remember when the Radeon 8500 drivers fucked with mipmapping to give better benchmarks?
http://www.hardocp.com/article.html?art=MTEx
A geek can use a piece of hardware/software and tell you it's strengths/weaknesses pretty easily. Outside of geeks, marketoids and management types don't have that "magic touch," so they demand numbers that they can then put into charts and graphs.
They don't want to hear "Card A is good at foo, but it everheats, and card B is good at bar, but slow at foo..." They want to hear "Card A is 125 foobars better than Card B."
GeekNights!
Late Night Radio for Geeks!
It would be real funny, if true, as nVidia was slamming 3DMark for not being a real-world indicator of performance of their NV30.
Remember, those great Doom III numbers were obtained on machines that nVidia supplied to reviewers. These numbers should also be suspect. If this is true, they had to know it would not look good. If nVidia did cheat like this, it can only mean the 5900 DOES NOT BEAT the ATI card. Desperate times indeed at nVidia.
Overclockers.com has a very well thought out Editorial on this issue titled ""Trust is Earned" It is well worth the read.
so what all the fuss is about, I have no clue.
The fuss is about the honesty of nvidia's business practices. I dont know about you, but I do not excuse dishonesty from business people -- they should be held to a very high standard.
If what extremetech is saying (that nvidia purposefully wrote their driver identify a specific benchmark suite, and then ONLY to inflate the results) it would be increadiby significant. if so, I would *NEVER* buy another nvidia product again -- and I would make clear to the (many unfortunately) people with whom I speak regularily about computer-purchase decisions...
Although does it really matter? Even with errors in a driver they beat the living hell out of ATI (in my opinion).
Apparently the first run of drivers had major bugs that would screw the images up, deleting objects and textures. I surmise its a problem with the new z-buffer whizbang.
It's funny that the site I read this 'review' on was very forgiving about these problems, yet its forums were full of rants about 'terrible ati drivers'. Of course, 99% of those rants has to be user error, as I've never had a serious driver issue with any of the ati cards I've used. They probably dont follow the installation instructions right.
Anyways, yeah. nVidia invents 'benchmarks' to make themselves look faster. So does AMD (the 2800+ ratings are based on an old t-bird core running at 2800+, not a comparison to intel). It's not so much fraudulent, since benchmarks really don't mean shit in the first place.
But fanboys swallow it up as gospel.
Anyhow, anyone have any firsthand experience with the 5200 and/or 5200 ultra? It seems like a worthy upgrade for $100 but so far I havent seen any really objective reviews of the card.
I don't need no instructions to know how to rock!!!!
Why is it that people are assessing the performance of cards based on running the same narrow set of benchmarks each time? Of _course_ if you do that then performance optimization will be narrowly focused towards those benchmarks. Not just on the level of blatant cheating (recording a particular hardcoded text string or clipping plane) but more subtle things like only optimizing one particular code path because that's the only one the benchmark exercises.
More importantly why is any benchmark rendering the exact same scene each time? Nobody would test an FPU based on how many times per second it could take the square root of seven. You need to generate thousands, millions of different scenes and render them all. Optionally, the benchmark could generate the scenes at random, saving the random seed so the results are reproducible and results can be compared.
-- Ed Avis ed@membled.com
Show me the Quack 3 Arena benchmarks! Then we'll decide which card is the best!
Not enough that NVidia only distributes binary only drivers. And refuse to release hardware API.
With ATI you get the source which means that the you won't get stuck because you have no way of getting drivers for the new kernel which has it's binary kernel module api changed.
With NVidia your videocard upgrade future is hostage..
I know man, every time I think I've found the holy grail of gaming, it just turns out to be a beacon!
"I only speak the truth"
Karma: null(Mostly affected by an unassigned variable)
Damn, a few years ago ATI did a similar thing to the drivers with the Xpert@play cards. The cards got good benchmarks that never held up once people actually played the games. They got beat up pretty bad for it at the time. Now it looks like nVidia's turn.
Do really dense people warp space more than others?
I think you're missing the point.
Better gaming hardware isn't supposed to make you better at gaming, that's a skill you either develop or don't.
What this hard IS for is to make games look better and provide for more sophisticated engines. If you don't care whether you play pacman or Unreal Tournament 2k3 then you're commenting out of your element.
I want hardware that can render pixar quality 3d environments in realtime on a 16:9 display in high definition so I can totally suspend my disbelief and reach new levels of game immersion. This is what I feel is the common drive for better hardware, and so far, IMHO it's working. I love playing games to the point that the line between the game and reality blur and I for one look forward to the traumatic experience of losing track of that line.
If you're hearing rhetoric about Linux, open source, or Mac and everyone's bashing Microsoft, you've found Slashdot.
I miss 3dfx... Anyway, who cares if they inflate their scores? Companies have been doing this for years! Intel's done it numerous times (Reverse engineering benchmark programs to get the best score for FPO's...). I would choose Nvidia over Ati for one simple reason, have you ever tried to get a driver for an ATI card? AUGH! Nvidia, one driver set, one download. Ati, There are drivers where the Ati Mach 64 pro is a DIFFERENT driver then the Ati Mach Pro 64. Plus, half the time when you go to install the driver you have to uninstall the driver, boot to VGA mode, then install it. Until ATI gets their driver act together, they'll always be second class. Too bad Dell feels like slapping the cheapest ATI's into computers they can get.
The article talks about possible solutions to the problem of "repeatability" while still avoiding the problem of cheating in the way alleged here. I don't remember it mentioning this possible solution though: How about if the camera was controlled by a mathematical function of a seed given by hand. Like you'd seed a PRNG.
This way you could repeat the benchmarks by giving the same seed. Generate a 'default one' at each new install (this to ensure clueless reviewers get a new seed). Make it easy to enter a new one or generate a random one.
The explosion of possible views (if implemented correctly) would make it all but impossible to cheat in the way alleged, no?
Belief is the currency of delusion.
As a programmer, I develop a test plan before I even start writing code. This is similar to someone giving me a requirement, and then changing the requirement after I've built a test plan and developed code toward that test. . . it's not really fair to the driver developers.
I'm going to side with nVidia, that this is a bug in the driver. Benchmarks make good testing software, but the best way to ensure good drivers is to make the benchmarks as comprehensive as possible. ExtremeTech is attributing to malice what is very likely an oversight on nVidia's part - seems a bit yellow on their part, IMO, especially since they choose to call nVidia's work on driver performance enhancement "corner-cutting."
This is a good lesson to nVidia, though, to give its developers better requirements and probably to test its drivers more comprehensively. If you tell me to maximize the frame-rate, you need to give guidance on how much of a hit you're willing to take on picture quality. And then have humans sit down and test the picture quality, rather than simply running it through a benchmark and analyzing the numbers.
!#@%*)anks for hanging up the phone, dear.
20zillion transactions per second provided you have a massive parallel Alpha with 1024 processors and 256 TB of physical memory for just 23.99$ per transaction assuming that you found your massive parallel Alpha on a heap of scrap metal.
ich bin der musikant
mit taschenrechner in der hand
kraftwerk
10 spelling and gramar errors in a post barely over one line. Good work! :)
I worked for a pretty crappy graphics card company (with hardware that was either architected poorly for Direct3D or simply buggy), and it was standard operating procedure to cheat any way we could.
One thing we'd do was do the fastest but lowest quality path UNLESS we found that the EXE was one of a list of games where quality was important. We used to test for the benchmarks instead but got caught at least once when one of the benchmark makers figured out they could randomly change their EXE name right before launch and catch people that way (not just us, by the way).
Even the good companies like 3dfx and nvidia cheated - they just did it less because their hardware was generally better.
People wonder why drivers are buggy; it's because of crappy hardware design first (NVidia used to just design to the reference rasterizer; which helped BIG-TIME) and then second the fact that 90% of the driver-writers' time was spent either cheating or eking out another 1% of performance.
Yes and no. The eye doesn't poll in the same way a camera does (I'm still trying to figure it out exactly through some not so scientific tests.) When you're playing a game, and it drops from 60fps to 45fps, you can notice the difference. It's somehow, less smooth. It seems 60 is the upper limit, because if the framerate drops from 90fps to 60fps it doesn't like a smidgen different.
...the fix consists of another vacuum cleaner to be attached to the card.
The 3dmark03 benchmark is cheating in the first place, implementing stencil shadows in two of the game tests in such a braindead manner which no sane programmer would put in an actual game.
It also uses ATI-only pixel shaders 1.4, and reverts to dual-pass on other cards.
Why all this?
NVIDIA isn't on the 3dmark03 beta program (read: didn't pay FutureMark a hefty lump of greenbacks).
The 3D graphics world is filled with contradictions. Nvidia has not always released WHQL certified drivers in recent months, but their quality has been the best (my ATI 9700 still hangs the machine with current ATI drivers). The 4403 drivers are not beta drivers, but were released yesterday as the "Detonator FX" (yawn). They claim to be WHQL-certified, but the asterisk points out that this is only for FX-class graphics boards. My 4600 doesn't qualify. All that being said, they seem fast and reliable in my anecdotal benchmarks/testing. Regarding illegal tunes: I ran strings on the driver and the only weird thing I saw was the nvcpl.dll knows where 3DSMAX.EXE is installed. Since this is the display properties control panel, its probably inocuous. This doesn't prove that they don't tune for 3DMark2003, though. Reading the report, it does seem suspicious, but I believe the tune would have to be specific to that particular benchmark. Finally, isn't anyone bothered that all the real world benchmarks show little difference between the cards? No one runs 3DMark2003 for fun. Are we all buying cards in the hopes that Doom3 will save us? Do 3D graphics hardware improvements matter when the software is so far behind?
Those of us who don't really care about comparing video card FPS, well, don't care about this.
My beliefs do not require that you agree with them.
So, because he isn't interested in this boring, repetative, inane and stupid ego-massaging 'my computer is more 1337 then yours' willy waving competition his opinion is invalid?
The trouble with free speech is that everyone has it.
http://twitter.com/onion2k
Take a step back for a second- Does it really matter if nVidia is cheating on a benchmark? If you think about all the amazing things a card will do for you, we are really only arguing about a few frames a second here, or maybe a slight image quality improvement. It's not like their latest card only gives you 2 FPS and only in grayscale and they say it does 500 other things...
The points between ATI and nVidia really boiled down the arguing between two rack bands at this point. People are going to be loyal to one or the other, but really, is there THAT much of a difference?
Why do you feel obligated to post the "I don't care about the zillion fps in quake"? Do you post a similar message to every story that you don't care about?
;) The scenery is a little prettier when I am looking at it sideways in TFC after my ass gets sniped, however.
I think the point he raises is that many of us, the MAJORITY of us, are not as concerned with squeeking out an extra couple of frames per second. Some of us would prefer 'better than average' 3d performance, 'much better' 2d performance, and more importantly, better STABILITY. I know that the people that first buy the cards, at the inflated prices, are what drive the market, but this is not currently benefiting the majority of buyers. I can look at my upgrade from a Geforce2 GTS w/32mb to a Geforce4 4200 w/64mb and although theoretically it was supposed to be a HUGE jump in performance, for the games I play, it was only a nominal rise. And it didn't make ME a better gamer either
There really should be a better way to benchmark these cards. Until they come out with a more 'random' way to benchmark, card makers will continue to pander to the benchmarks, instead of the real world. I don't have to have the solution in order to see the problem.
Tequila: It's not just for breakfast anymore!
I guess they're not worried about "the way it's meant to be played"!
It's all about the motion blur.
Films go at 24 FPS, but film also has perfect motion blur and the eye can harldy tell.
Strobing is a big problem with games as good motion blur is next to impossible to make without just rendering out all of those in between frames.
-Derick
Here is the real problem. To give you people an idea of what happens when a company ignores the marketing hype and produces a rpoduct that JUST plain performs we dont have to look any furrther than the Intel/AMD rivalry. Intels chips are not 'superior' in ANY sense of the word but still enjoys the majority market share because most people are idiots who only get their information from the marketers and the marketeres know this.
If AMD managed to get 2 years worth of benchmarking successes over intel then I bet the house that the majority share that Intel now possesses will be in jeopardy.
In addition, who really cares about 3DMark?
... sorry, I got on my soapbox for a minute there. I hope my cynicism of the online hardware review industry wasn't too obvious ;)
It's interesting that you bring that up, because the simple fact is that all the fanboys and hardware junkies care about the 3DMark scores. I mean, let's face it - if nobody cared about 5784 3DMarks, then this wouldn't be news.
But it is news because all those hardware sites out there that have to publish every single number under the sun and then present them on graphs with cut axes so they can have the longest review and be the biggest and best hardware site and get early beta products that they can favourably review so they get more early products
Why not use time which is wasted on 3DMark benchmark for benchmarking real games?
I don't think FutureMARK would be very happy to hear you say that, but would it solve the fact that the drivers could be written to take shortcuts when the games are used for benchmarking? The number of games commonly used for benchmarking is a reasonably small subset. I'm not a programmer, but it would surely be no more difficult than what was done with the 3DMark benchmark?
"Because it's there." - George Mallory, when asked why he wanted to climb Mt Everest, March 18, 1923 (New York Times)
would you really rather buy a car that REALLY does 0-100kmh in 3.6 secs or one that does that only in free fall assisted by rockets?
that's whats this is about.
and the thing that you would really think them to stop doing this since they tend to get caught and it's bad PR for them. cheating like this is ridiculous.
it clearly isn't a case of driver bug, but definitive optimizations for the 'normal' test run. how would you like a doomIII experience where you could only walk on a certain path, and have your drivers be 200mb for that..
world was created 5 seconds before this post as it is.
why did it only occur with binaries named QUAKE.EXE? If the exact same binary was named QUACK.EXE and the EXACT same options were selected in the game, the "Bug" did not manifest.
It was bold-faced cheating and they were caught at it.
This on the otherhand, looks like a bug. I've seen that sort of thing happen in several games with various video cards. Doesn't mean they were all trying to cheat some benchmark in an obscure game, it means that a bug in the driver (or game) was uncovered.
"This doesn't prove that they don't tune for 3DMark2003, though. Reading the report, it does seem suspicious, but I believe the tune would have to be specific to that particular benchmark."
In the past, to tune for a specific benchmark you generally had to detect the benchmark application running and hence your strings check might work (though any competent programmer can think of an infinite number of ways to defeat it).
3DMark03 used a timed demo playback so depending on your frame rate your driver could get different calls every time the benchmark ran. However, 3DMark03 uses frame-based playback and makes _exactly_ the same calls to the driver, down to the very last bit, every single time it runs: it is, therefore, trivial for someone to add code to say 'are we being asked to render with pixel shader xyz at camera position (0.512, 23.76542, -89.67325)? If so, we're running 3DMark03. For frame 1, we can ignore the first twelve rendering calls as they're offscreen. For frame 2 we can ignore the last eight rendering calls as they're offscreen. For frame 3 we can artifically clip anything outside this area as they're offscreen, etc, etc' That would give you a big, big boost in the benchmark, but the rendering would go wrong the instant the camera was turned away from the default position for that frame, and would give you a high value that bore no relation whatsoever to the performance that card would give in a real game.
And this is why frame-playback benchmarks are a bad idea: they're trivial for programmers to develop hacky optimizations for that artificially inflate their numbers. There's no definite proof that nvidia have deliberately done this, but the pictures in the web articles look very suspicious.
Labelling a 1800 MHz processor as a 2200+ one, or a pair of 3 Watt RMS speakers as 120 W (pmpo) capable?
Marketing stinks, marketing without enforced rules stinks more. Those who use deceiving tactics to sell a product should be punished hard.
My personal favorite from this article:
:-)
nVidia believes that the GeForceFX 5900 Ultra is trying to do intelligent culling and clipping to reduce its rendering workload
It's alive !
Hello! I'm a disaster waiting to happen!
"Films go at 24 FPS, but film also has perfect motion blur and the eye can harldy tell."
What's "perfect" motion blur?
The reality is that you _can_ see stuttering in movies if you look for it and the director doesn't know how to avoid it (which is, for example, one reason why you rarely see a fast pan in IMAX movies). I remember some of the pans in 'Sixth Sense' being particularly bad.
> The slight improvement in hardware isn't going to change the fact that I'm only a mediocre gamer.
You're telling me. My on-screen persona in UT usually commits suicide right from the start in order to avoid being tortured to death by the other players. No amount of money spent for the newest and improvest graphics card is going to change that.
"There are already a million monkeys on a million typewriters, and Usenet is NOTHING like Shakespeare." - Blair Houghton
It's rude, but also true.
Benchmarks, even so-called 'real-world' benchmarks, are a poor indicator of system performance. Sites like Tom's Hardware and Anandtech exist as a kind of group therapy for hardcore gamers and 'performance enthuiasists'. You know if you read their "technical" articles that they understand as much about the inner workings of a computer as the rice rocket driver with the huge spoiler and chrome wheel covers understands about his car's engine.
These sites always have an incestuous relationship with their advertisers, they don't know anything about statistics, the scientific method, or how valid data is gleaned and collected.
Even ArsTechnica has tons of articles that pass off conjecture as fact (case in point: the latest PPC970 article). While their writers seem more technically knowledgeable, it's still deceipt.
Benchmark and "performance enthusiast" sites are a con job, plain and simple. They should be treated as what they are, the "EZ WEIGHT LOSS PLAN!!!!" scams of the geek community.
(-1, Raw and Uncut is the only way to read)
Coming from a 3D Graphics programmer, the NVidia is ALOT better than ATI.
My 3D terrain visualisation tool runs on OpenGL. I do think that ATI might be better at doing Microsofts Direct3D crap, they are certainly lacking with OpenGL optimisations.
I've done speed tests on higher end ATI and the fps was quite low.. somewhere around 15-20 fps. The NVidia (GeForce2 GO chipset) ranged from 35-45 fps.
Also, we all know that ATI and NVidia optimise their drivers for benchmarks, so wouldn't that make the benchmarks meaningless?
if thats the main reason for not releasing the code of the Linux and FreeBSD drivers. Are there "commercial" test benchmarks aviable for free software operating systems?
They'll be detecting that they're running under 3dmark and changing how they render. The most interesting way to show that this was a benchmark cheat not a coding fuckup would be to
'grep -i 3dmark <nvdriverfiles>'
However they will have learned from ATI's q3 mistake and obfuscated any dodgy code if its there - oh well.
The point is that unless you run specific programs, the "optimisations" don't show up at all. If, in fact, they are benchmark cheats not driver bugs.
The right answer to this problem is to use the same benchmarking techniques that professional software and hardware vendors use in verification. Simply use a random but recorded factor in each test. Then you can compare two seperate sets of hardware against each other for exactly the same deterministic test, but the test itself is random. That way, you can avoid being susceptable to this kind of cheat while still maintaining a fair comparison between vendors.
-=Eric
www.geoskd.com
I wish I had a good sig, but all the good ones are copyrighted
hardocp.com on the front page has a great writeup on this.
But basically, extremetek is just a little bit mad because they were excluded from the doom3 benchmarks. Since nvidia refused to pay the 10s of thousands of dollars to be a member of the 3dmark03 board, they have absolutely no access to the software used to create this bug.
Here is the full exept from hardocp.com:
3DMark Invalid?
Two days after Extremetech was not given the opportunity to benchmark DOOM3, they come out swinging heavy charges of NVIDIA intentionally inflating benchmark scores in 3DMark03. What is interesting here is that Extremetech uses tools not at NVIDIA's disposal to uncover the reason behind the score inflations. These tools are not "given" to NVIDIA anymore as the will not pay the tens of thousands of dollars required to be on the "beta program" for 3DMark "membership".
nVidia believes that the GeForceFX 5900 Ultra is trying to do intelligent culling and clipping to reduce its rendering workload, but that the code may be performing some incorrect operations. Because nVidia is not currently a member of FutureMark's beta program, it does not have access to the developer version of 3DMark2003 that we used to uncover these issues.
I am pretty sure you will see many uninformed sites jumping on the news reporting bandwagon today with "NVIDIA Cheating" headlines. Give me a moment to hit this from a different angle.
First off it is heavily rumored that Extremetech is very upset with NVIDIA at the moment as they were excluded from the DOOM3 benchmarks on Monday and that a bit of angst might have precipitated the article at ET, as I was told about their research a while ago. They have made this statement:
We believe nVidia may be unfairly reducing the benchmark workload to increase its score on 3DMark2003. nVidia, as we've stated above, is attributing what we found to a bug in their driver.
Finding a driver bug is one thing, but concluding motive is another.
Conversely, our own Brent Justice found a NVIDIA driver bug last week using our UT2K3 benchmark that slanted the scores heavily towards ATI. Are we to conclude that NVIDIA was unfairly increasing the workload to decrease its UT2K3 score? I have a feeling that Et has some motives of their own that might make a good story.
Please don't misunderstand me. Et has done some good work here. I am not in a position to conclude motive in their actions, but one thing is for sure.
3DMark03 scores generated by the game demos are far from valid in our opinion. Our reviewers have now been instructed to not use any of the 3DMark03 game demos in card evaluations, as those are the section of the test that would be focused on for optimizations. I think this just goes a bit further showing how worthless the 3DMark bulk score really is.
The first thing that came to mind when I heard about this, was to wonder if NVIDIA was not doing it on purpose to invalidate the 3DMark03 scores by showing how the it could be easily manipulated.
Thanks for reading our thoughts; I wanted to share with you a bit different angle than all those guys that will be sharing with you their in-depth "NVIDIA CHEATING" posts. While our thoughts on this will surely upset some of you, especially the fanATIics, I hope that it will at least let you possibly look at a clouded issue through from a different perspective.
Further on the topics of benchmarks, we addressed them earlier this year, which you might find to be an interesting read.
We have also shared the following documentation with ATI and NVIDIA while working with both of them to hopefully start getting better and more in-game benchmarking tools. Please feel free to take the documentation below and use it as you see fit. If you need a Word document, please drop me a mail and let me know what you are trying to do please.
Benchmarking Benefiting Gamers
Objective: To gain reliable benchmarking and image quality tools
This is not hard. Just randomly create some test case, record it down to a file, and use it for all test. Testers can generate as many of these as they wish. They they can pick one to use for all. You can argue that the tester can be bias (well, you either trust them or you don't). You can argue that the random test case is not well representation, testers can run many of them, and take average. You can keep argue, but hey, for benchmark, it's up to you trust the result or not. If not, develop one yourself, or just run it yourself, or just ignore the whole thing (or go and get a beer).
"The best gamers are going to kick my ass regardless of what hardware they use"
You know, every time I want to play an online game I remind myself of how many times I've been killed by better players. So much so that I detest even the idea of an online game now. Screw those other people, I want to match my wits against this heap of iron in front of me.
My ego can only take so much.
"If you could only see what I've seen with your eyes..." - Roy Batty
So who cares? It matters little to me how fast something is in a synthetic benchmark if there is no correlation to real world applications, and I am sure Nvidia isnt doing this in games cause who would buy a card that didnt properly render most scenes.
I dunno, but synthetic benchmarks seem a bit irrelevant as does what Nvidia does in them. Show me how many FPS it gets in Q3A, that I care about.
"Also, we all know that ATI and NVidia optimise their drivers for benchmarks"
This is why all benchmarks are completley meaningless. As soon as a benchmark is published, everyone writes their code/designs their hardware to be good at the benchmark and not necessarily good at anything else.
This is why I have never and will never pay one iota of attention to CPU/Video/Ram/$RANDOM_WIDGET/Software "benchmarks."
Besides, I'm much more concerned about how accurately my AthlonMP calculates 2.999999+0.000001=3.000000 than how fast my Intel can calculate 2.999999+0.000001=3.000084...
Companies always tweak their code, insist on tests optimized for their hardware, etc. in order to get an edge up on benchmarks. This is probably especially true in cases where the competition is so neck-and-neck, as it seems to be with the video card industry. It seems that these companies will do anything to show they can get even two or three more FPS than the competition. It is hard to treat any benchmark seriously because of this.
At the same time, I'm debating what my next video card should be. Even though ATI's hardware might be slightly better this round, the differences will probably be negligable to all but the most extreme gamers. At the same time NVidia has proven to me that they have a history of writing good drivers, and they still provide significantly better support to the Linux community than ATI does.
For this reason I'm still siding with the GeForce family of video cards.
"You spoony bard!" -Tellah
I get this EXACT same problem with an ATI-7xxx in Eve Online. Does that mean that ATI is optimizing their drivers for Eve Online with a hardware clip-plane to speed up the Eve Online benchmarks? Or does it mean there's a bug in their driver?
I know which one I'd choose.
No, my problem is that he didn't have an opinion on the subject at all. The topic, in general, is "is everybody cheating in 3d benchmarks", or in particular, "is nvidia cheating in 3d mark 2003".
His post was "I don't want to spend $400 and still lose against better gamers".
I am honestly interested in why people who do not care about the topic feel compelled to post a message stating that fact.
Never underestimate the bandwidth of a 747 filled with CD-ROMs.
Honestly. ATi caters to the MAJORITY OF THEIR CUSTOMERS FIRST.
Of course they don't put out Linux drivers at hyperspeed. They devote the resources to their Windows unit, which if you hadn't noticed have gone from a "wait forever for a driver" status to being, as far as I have seen, on par with NVidia (perhaps more so -- all the "unofficial driver leaks" I've ever seen for NVidia cards tended to be unstable in one way or another).
On the other hand, my 9700 Pro is gliding along smooth as silk on ATi's last release set.
Did you see what they had to do to "prove" the cheat? Read the article. In other game tests the card beats the ATI 9800PRO so obviously it is faster. (see anandtech, hardocp, tom's hardware, etc if you really care).
The things that they're being accused of reduce work to the graphics engine - and doesn't affect image quality - it's called OPTIMIZATION. The fastest frame rate with the best image quality.
Man someone must have spent hours in front of their computer coming up with a way to get a sensational story like this. ATI has done it, and so does everyone else but what sucks is that this "news" is being flogged everywhere like it's the most incredible piece of news ever.
In this case it's not ANYWHERE NEAR as bad as changing the card's performance based on the name of the program that's being run - I think most people remember that one.
In this case it's a non-story. And yes, we all pay too much attention to benchmarks. I am now one to two generations behind leading edge and plan to stay there. It's far less expensive than driving a new car of the lot every four months.
This sig contains a manual self-destruct. Kindly please put your foot through your monitor in 8 seconds.
Actually there's a good point of 3DMark, much as I hate to say it -- since we don't have any DX9 games yet (instruction sets and cards having gotten years beyond the curve of game production), without 3DMark's scores, we have no idea how the card will perform on the games that will come out this fall.
Of course by then the drivers will have been tweaked again and again and there'll be another round of cards out anyways...
On the whole scene being rendered correctly:
It is perfectly possable ot read the graphics data from the card and write it to a file, like a tiff. In fact, I've seen some benchmarking programs that do. Then what you can do, for DirectX at any rate, is compare against a reference renderer. The development version of DX has a full software renderer built in that can do everything. It is slow as hell, being a pure software implementation, but also 100% 'correct' being that it is how DirectX intends for stuff to be rendered.
Well, if you have a benchmark that includes images from the reference renderer, you can then compare those to the current renderer. Aside from just looking at them, you can do mathematical calculations of the images to see where and how they differ. A simple one would just be a straight XOR on all the pixels. If the current renderer got the same result as the reference renderer, you'll get black as a result (since anything XORed with itself is 0). Any time there is a difference, it will show up as a soloured pixel, and the more colour, the more it was different. I've seen a benchmark do this but I don't remember which one.
Not saying that this is the perfect, end-all solution for graphics cards, but there ARE ways that they can be tested versus some kind of reference.
The Nvidia drivers seem to be getting bigger and bigger and bigger.
What next? They going to store prerendered movies in the driver for all the popular benchmarks?
Video performance from my Radeon 7500 under Linux (using the ATI optimized drivers for XFree86 4.3) is not nearly as good as the ATI-provided drivers under Windows 2000. I think ATI gives the type of ingredients to the Linux driver developers, but the quantity of those ingredients it keeps to themselves.
nVidia could really follow along this same philosophy, instead of hearing the massive complaints from their oft-buggy video driver.
Ayup
Well, nethack isn't exactly 3D-intensive.
Companies have long adopted the "open-source" fundamental philosophy even before Linux and what I call the modern open source movement caught on. Often, a company would have a nice product - license the code to a sub-company (who would modify/repackage/etc the original product). The license agreement stipulated that all modifications would 1) have to be reviewed by the company without restriction from the sub-company 2) the modifications would have to be approved by the company.
Take for instance the relationship between Microsoft and IBM during the OS/2 era. The two companies working on the same code base produced OS/2 and, eventually, the NT kernel.
Or, more recently - the brilliant strategy of Netscape Communications Corporation - the birth of the Mozilla project. To the open source community - take our browser, modify it like hell, make it a better project. You have, of course, Mozilla as the browser - but Netscape (Navigator) still exists (as a repackaged, "enhanced" Mozilla).
nVidia's source code release would have two major impacts as far as their performance goes.
1) ATI (et al.) would find the actual software-based enhancements they could also incorporate into their own driver to improve their product.
2) nVidia could capture the many brilliant software developers that happen to be a part of the whole nVidia "cult" - this could lead to significant advancements to their driver quality (and overall product quality).
My guess is that the lid is kept so tightly shut on nVidia's drivers because they can keep their chips relatively simple through their complex software driver. ATI, perhaps, has the technical edge in the hardware arena, but does not have the finesse for software enhancing drivers like nVidia does.
Ayup
All i got to say is my voodoo 3 owns you at any game. Its got 16 megs of mem and does about 15fps in dos and if i leave my computer on to long it starts blinking(the screeen)
The point I was making is simply this - if they cheated or did not cheat on the benchmarks, does it really make a difference? For some, sure. But for me and probably a good chunk of people out there, the slight extra edge that NVIDIA may or may not have given themselves in this benchmark isn't going to be enough to make me run out and purchase the new geforce over the radeon unless I wanted to particpate in the "I have the fastest graphics card available as of 3:00 this afternoon" pissing contest. The few extra FPS nvidia can boast by rigging this benchmark will not help me become a better gamer, nor will it help most people become better gamers. So what's the point of becoming enraged over something like this? Even if you are one of the lucky few who can tell the difference between a great card and a slightly less great card, has this really altered your opinion so much of your choice of video cards?
Reading the posts, I dont think everyone is understanding the point of the rail test.
Using the rail test, Nvidia excluded almost all non-visible data. This shows nvidia tweaked its drivers to only render data seen on the rail test, which would only happen if you tweak your drivers for the benchmarks. (aka the cheat)
I like it better if benchmarks uses average FPS on a game, and you go PLAY the game, and watch for yourself.
Try 1024x768/1280x1240/1600x1200 with all AA/AF modes. Also stop using 3ghz P4's for the benchmarks, use a mix of 1ghz/2ghz/3ghz AMD/Intel boxes so we can know if the hardware is worth the upgrade.
Agreed. Culling / hidden surface removal is based on the camera's current POV. That said, if the camera angle changes, the game engine or the video card's own culling engine should supply the card with new data based on the new camera POV.
What this test attempts to show is that Nvidia's card selectively clips based on a SINGLE, DEFINED POV.
THIS IS NOT A BUG. Your camera changes, your visible surfaces should change. If it doesnt, this is a sound indicator that you are anticipating the clipping plane.
What astounds me is that people like Kyle Bennett & Co., who have no concept of how 3D rendeing works, would pass this off so readily as a retaliatory strike.
Man is the animal that laughs.
And occasionally whores for Karma.
wait, i understand 24fps is the "magic number"
I chose to turn up that detial and play in the high teens range and manage OK (or so I think i do.)
and its smooth to me.
Is it just me? wheras my stepbrother buys a new card every six months so he can see 200 fps?
Also, if film is 24, why is tv like 30?
-Grump
Is it true that more people vote for the winner of American Idol, than vote for the president? -Ali G.
With the Riva128, back when I had a 3Dfx Voodoo (or Voodoo2).
They garbled texture maps to achieve a higher transfer rate and frame rate. Then they went legit for the TNT line.
I guess the belief "if you can't win, cheat" is still there at nvidia.
I wonder if ATi makes a good Linux driver...
Nvidia supplies binary drivers, but a lot of people (including myself) have had issues with stability and compatibility of these drivers. Supposedly they are in pretty good shape right now, but all my current machines use ATI cards or ancient S3 and Trident cards.
ATI has instead submitted accelerated drivers to the XFree86 project, and produced binary Linux drivers for only their workstation cards. This hasn't worked so well for them, because of the (publicized on Slashdot) problems with getting stuff into XFree86 at the moment, but independent vendors such as Red Hat have picked up newer ATI drivers.
I think that ATI is actually contributing more to the community by providing modern drivers with source code, even if that means they get slightly lower performance because they can't reveal all of the IP in their proprietary drivers.
Of course, ATI used to have a terrible record with drivers. The existing Rage??? drivers all suck, and only the very latest card based on that (coincidentally the only one which is still sold) has current drivers. The record for Radeon cards with unified drivers has been very good, though. In my opinion better than Nvidia, who have a habit of breaking support for old hardware in the unified drivers and not fixing it for a dozen revisions.
The recently released card was the GeFarceFX 5900!
There are no karma whores, only moderation johns
Insults the reviewers?! I'm more insulted that the reviewers try to spew 3D Marks everywhere like they are a meaningful unit of measure.
_exact_ problem? that the clipping planes go to hell when you play eve online and walk away from some pre-calculated path?
i don't know eve onlines benchmark runs.. is it possible to stop it in the middle and walk away and then the clips go to hell?
world was created 5 seconds before this post as it is.
Millions of young males are playing games on their new video cards instead of creating teen pregnancies like they should be, authorities have stated!!!
In other news, skin cancer rates are inexplicably dropping, pizza companies posted an 80% increase in business and deodorant sales have dropped.
This sig contains a manual self-destruct. Kindly please put your foot through your monitor in 8 seconds.
The normal rage is bad enough, I can't even play the original UnrealTournament on the mobile rage with Linux, though the Windows version works just fine.
The preceding post was not a Slashvertisement.
Some compilers miscomplied the modified benchmark, because they recognized the code as the standard benchmark even though it wasn't exactly the same.
(Anybody have a reference for this? I heard the author give a talk at Stanford years ago.)
I agree. While I really like my gaming to be very pretty (look ma, see that pimple on that Skarj's ass?), I also don't really find it necissary to buy a new card for $200, every 6 months. Mostly I upgrade to keep up with development, and not to "weewee-wave". (okay the 1gig of DDR wasn't STRICTLY necissary)
I just did a full upgrade of my box, because I didn't meet the min requirments for a couple games. Mind you I was still a proud owner of a Voodoo2 16meg, which allowed me to play Warcraft3 with full textures, and get a decent FPS from UT. But Morrowind and Unreal2 told me to upgrade.
Though, if I had a larger disposable income, I would probably be buying a 256 ATI. Just so I don't need another card for another 2 years.
A patriot must always be ready to defend his country against his government. -edward abbey
Nvidia's current problems sound familiar don't they? 3DFX started floundering once they made it to the top, and started worrying more about profit margin and market share than putting out the best video cards. If they keep this behavior up, I give it two years before ATI starts looking at buying them out.
As a Canadian, I had always tried to stick with ATI video cards. I knew they weren't the best, but their pricing would reflect this and I could still get a decent card for my money. But when my budget had room to upgrade my Radeon 7500, the 9700 had just come out and I couldn't afford it. So I got a very good deal on a GeForce4 Ti4400, and happily put my very first NVidia card into my computer. It is a very good card. Too bad I can't say the same for anything that NVidia has produced since. Unless their upcoming processor is both the bee's knees and comes out on time, my pre-Christmas upgrade (because we all know that Doom 3 and Half-Life 2 are going to need bad-assed new cards) is going to be made by ATI.
giga = 10^9, and an 80 GB hard drive has 80 x 10^9 (10 billion) bytes. This is standard notation that has been in use for at least a hundred years. Perhaps what you're looking for is 80 GiB, which the hard drives are not advertised as.
This is standard even in most other parts of computing (anything engineering-oriented especially). For example, that 128kbps mp3 you downloaded is 128000 bits/second, not 128*1024 bits/second.
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
NVida has always stood silent in the race to win benchmarks. Fact: Every video card manufacturer tweeks drivers specificly for benchmarks. ATI scammed people into a 50% performance increase years ago with a new set of drivers. This of course was completely false.
Fact: NVidia is probably the last company to join in this race. when they denouced the use of Futuremarks programs after 3DMark2k3 showed undeserved favorability towards ATI's driver set they were ostracized for not being a big player. It seems to me that they finally said "fuck it, the public wants bullshit drivers that inflate thier benchmarks, then we will give it to them!"
Good for NVidia. They always have been, and for the forseeable future always will be the no compromise 3d gaming solution.
The funniest part of this all is this, in unreal2k3 I personally have seen a 160/80 flyby/botmatch score jump up to 220/103 on a 5800FX based AMD1700+ system. So the drivers are not complete bullshit. Unlike ATI who was chastised in the past for having lower game scores after the fact.
I had an nVidia GeForce 2 card in my previous PIII Win2k box. It wasn't the fastest card around, but it did the job. And it seemed to render game video correctly and true to the game vendor's intent. Eventually I was unable to play certain games with big CPU/video requirements, and bought one of those little Shuttle Mini ATX boxes with a nice P4 3Ghz CPU and an ATI Radeon 9700 AIW Pro. The box screams, and I can play games at untold resolutions now. But that Radeon just isn't quite right.
Playing "Jedi Outcast", it seems to omit the sky in outdoor scenes, which is completely lame. Sometimes you can see through the corner of walls, as if there is a crack. It displays severe performance problems in 32-bit mode, as well as some other behavioral quirks. And in at least one popular game (Raven Shield) you have to completely turn off antialiasing or the mouse doesn't work properly (go figure!).
My nVidia card didn't seem to have any issues at all, at least none that I could detect. Certainly nothing as plain as what I've found with my Radeon 9700. I would not be surprised if nVidia has some problems with their latest card, nor would I be surprised if they were consciously cutting corners. But there are enough issues with the products their competitors put out that I have to wonder why nVidia is being singled out here?
The shutter isn't instantaneous.. it's left open for a period of time as things are moving/rotating the film is recieves a perfectly (as perfectly as film can be) smooth blur.
I meant "perfect" motion blur in that it's not a hack. It doesn't look like two or three images mixed (a common PS2 trick) and it's not smudged (common in 3d renderers... basicly saving movement data and "smudging" colors around as a post process. A lot faster than the more accurate method of rendering the object 10+ times/per frame and mixing them.)
Video cards show one instance of time per frame. Occasionally faked motion blur is added, but this typicly doesn't look very convincing.
-Derick
Well, Some fella a long time ago (don't remember his name) did a large number of experiments and found that 24FPS is the lowest number where the eye could be convinced it was in motion. The desire for the lowest number is because film is expensive (especially back then.)
When television was being made it was found that the eye could see a bit faster than the 24 ( around 30 fps)
So now TV goes at 29.97 FPS.
What's funny about people making their games go at 200 FPS is their monitor is probably set to 80 Hz or less... heheheh
-Derick
Drugs are a cheaper short-term solution, but a more expensive long-term solution.
Slashdot: Failed Car Analogies. Amateur Lawyering. Anecdote Battles.
While you're absolutely right, one has to also wonder who would actually go to the huge amount of effort required to provide free web sites (updated almost daily, no less!) with good data and benchmarks on all the new hardware.
Of course, the only logical answer that comes to mind is the folks getting some sort of financial benefit/kickback from the hardware manufacturers.
All I'm saying is, perhaps you're being a bit too hard on sites like Tom's Hardware - because "unprofessional" and "inaccurate" as it may be, it's provided for free to the public, and I've seen much worse info that people pay big $'s to receive (cough, Gartner Group, cough).
Like anything you read, you have to consider the source, and take the data with a few grains of salt. I'm far less concerned with benchmark number they publish anyway. All I look for is general information, such as "video cards Y and Z are outperforming A and B by a wide margin, although the new card Q seems to run as slowly as the cards made 2 generations ago".
Why not (immediately before testing) record a sequence in a 3D game and use this as your very own benchmark? Then at least the graphic card manufacturer has no chance to trim the driver in advance.
By the way, this reminds me of the days in the early 90s where certain graphic cards could recognise the Ziff Davis 2D Benchmark. In one test sequence, a lot of rectangles were drawn above each other - not a real world application and thus very easy to detect. This lead to the upcome of becnhmarks that consisted of real world applications (like scrolling a big text in Word and drawing diagrams in Excel).
Excellence: Moderate (mostly affected by comments on your karma)
A valid point, especially when the same accusations have been made about ATI's Quake3 benchmarks in the past.
:)
I'm an eye-candy/FPS nut too, and I will spend the money on the card that blows me away. But do I really care if the benchmark gives me an extra 5 to 10% improvement over it's competition in a specific game? I read over what people have to say about the drivers stability instead. I know I'm going to have a fast card already if I spend top dollar. And getting stuck with drivers that make the card's performance unreliable is going to sway my opinion mightily. If they cheat a little, that's okay. As long as they perform well consistently.
That's why I stopped buying ATI a couple of years ago. Consistently crappy drivers on decent cards. Now they've improved significantly, and I'm probably going to buy their products again. It won't make me a better gamer either. But at least I'll have the visual quality I enjoy to go along with the speed
and got caught red handed.... you would think ppl would learn (if this is indeed true which I do not doubt)
I do realize that using base-2 has been standard in measuring storage space in computing for a while; I was referring to the prefix "Giga" itself, which has been an SI prefix for "10^9" for quite some time, with common usage in the sciences dating back at least 100-200 years. I'd imagine 2^10 was called "kilo" because it was close enough to an actual kilo, but 2^(10x) and 10^(3x) diverge as x increases, so that's getting increasingly confusing.
And the base-10 version is standard in everything now except for storage space, even computer-related stuff. Audio file bitrate as I mentioned is in base-10, network bandwidth is in base 10 (that's why 10 Mbps != 1.25 MiB/s. 10 Mbps = 10 000 000 bytes/s = 9.56 MiB/s), and so on.
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
10 Mbps = 10 000 000 bytes/s = 1.12 MiB/s.
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
I think it's pretty obvious that nV has joined the long, proud list of video card manufacturers that have fudged on their drivers to make their card look better then it is. The question I've got is... why?
I mean I understand best-case. You cook the benchmarks, you don't get caught... profit! But I'd think that getting caught cheating at benchmarks would really screw sales, not to mention a companies reputation. Is the risk worth the possible payoff?
If it is, then I can only figure the either A) companies cheat at benchmarks all the time, and rarely get caught, or B) Companies don't really notice any sort of backlash when they do get caught. If getting caught cheating won't hurt sales, there's really no reason (from a business standpoint, anyway) not to do it.
The only other reason I can come up with is that nV feels the market slipping away from them. If they feel that they can't compete fairly, they might decide that it's either fudge on a few benchmarks, or go out of business.
I am NOT a man!
I am a free number!
First, faster video cards are not designed to make you a better gamer, they are designed to make your gaming experience better. If they are not doing that for you, then you're not playing the games that need the improvement, and you don't need the card. Which, I'm sure, is true for a lot of people out there.
On the other hand, ATI sold over 1 million Radeon 9700s in first few months of it being out, so there are definitely a lot of people out there who do need and want the best card the money can buy.
So, that gets us to your question of whether nvdia cheating really makes a difference. Obviously, it doesn't make a difference to you, because you don't want the buy any of the high-end cards in the first place. It should be obvious in the same way, though, that it does make a big difference to somebody who will buy a high end card.
If 9800 and FX5900 have the same price, and speed is what you're after (and it should be, since you're buying these cards), then you want to buy the faster one. The only way to figure out which one is faster is to check the benchmark results (unless you buy both and try them tyourself). If one of the companies cheated in a benchmark, they have tricked you into thinking that you're buying a faster card, while you're really buying a slower one.
Imagine you're picking between two equally expensive cars, and you want to buy the faster of the two. One claims to do 0-60 in 5s, and the other claims to do it in 3s. You'll go ahead and buy the latter one, only to learn later that they were testing the car going downhill while the other was accelerating on level ground! I think enraged would only begin to describe your reaction to that.
Never underestimate the bandwidth of a 747 filled with CD-ROMs.
I used to work there, they were always optimizing drivers to look good in winbench and macbench.
Matrox did it too....
But it is sort of valid. Take winbench, decompile, see what you need to do really quickly (ie. what kind of blitting etc.) and optimize the crap out of that. At any rate, the end user gets some aspects of the driver that are fast, and some that are really really fast.
Unfortunately optimizing for benchmarks is a fact of the industry
All I need in a card is >75 fps. why so low? 75fps is the most a human can distinguish. so get me 80 fps and wow! no flicker! and you can't tell the difference between anything higher, so why bother? I'm not sure on the resolution figures, but 1600*1200 is pretty close to not being able to see pixels anymore. so why go faster and higher when you can't tell the difference?
Not a sentence!
You are right - it is completely beside the point that Ballmer has financed a goat farm. However, it is interesting.
Remember that 4.9 MB .jpg of the Hubble deep space shot last week? X took it just fine as a background with my Radeon 8500, but Windows 2000 choked.
I already told everyone that the Nvidia beta drivers would run "faster".
1 659
Look at the post I made from a few days ago: http://slashdot.org/comments.pl?sid=64007&cid=594
You can cut corners, get the performance numbers, and if there are any problems, you have the safety net of "Hey, these are BETA drivers on BETA hardware.. so sue us if there are "bugs"".
- Jeff
P.S. I'm pissed off.
Modesty is one of life's greatest attributes
I hope this finally proves that synthetic benchmarks are worthless. Reviewers need to stick to running a series of games to benchmark a system/card, and not just spouting off some 3d Mark number [further causing people to go "my penis is bigger than yours!" and trying to get just 5 more points than their friend.] Game developers seem to anticipate benchmarking and most often put benchmarking utilities/commands into their code to assist with such things.
Not All Who Wander Are Lost
Im not surprised, any company that does it's own assesment of performance is going to fudge data. Self-serving research is not going to get you accurate data.
Personally, I have nothing but good things to say of Radeon. While it can be argued that their cards don't quite measure up and driver issues are common with some software, the fact Radeon Cards are on average $100 cheaper than their Nvidia equivalents makes it easier for me to look the other way on those sorts of things.
I still have a hard time trusting Nvidia after that garbage with the GeForce-4s that were really Geforce2s with more RAM.
On Wall Street they say "buy low, sell high" On the pad we say, "buy high, sell high" Isn't that somehow better?
I would agree, but only to a certain point.
To exagerate my point: I would challenge, and likely beat, anyone who still used a 486 to play a graphically intensive online game, even though I would most likely lose to someone who played on my same hardware.
I would agree, however, that people spend good money to change from 75fps to 120fps, when the human eye is unable to distinguish between the two. That said (to ramble on) if I am the only one in the game and get 120fps, how many fps would I get when there are 40 other players in my same area? I think that's where the extra horsepower, graphical or elsewhere in your box or connection speed, comes in. That's certainly why *I* would spend the extra money...just to compete. People just need to understand when they *should* make that purchase.
If all you have is a hammer, everything looks like a nail.
I seem to remember a few other choice nvidia cards (ahem) with, shall we say, augemented benchmark results?
I call bullshit.
--Yep, seems to me like ET is spreading crap about NVidia and assuming bad intentions. I don't think NV would do something like this intentionally, seems like an honest driver bug.
From the article:
>The problem is that, because a given workload is identical from run to run, hardware vendors can carefully study it and make optimizations to maximize performance, some legitimate, and others unsavory. In these types of deterministic tests, the camera movement is essentially "on a rail," (think of the opening credits sequence in Half-Life where you ride the tram into the depths of the base). And because the camera's every move is a known quantity, hardware vendors are able to figure out exactly what will, and won't be visible to the camera. This is where vendors can look for places to cut corners.
--Here's the places in the article that *I* find suspicious: (emphasis mine)
> A developer version of 3DMark03 version 3.2 allows the tester to pause playback, and then move freely through the scene -- in much the same way you'd move through a first-person shooter like Unreal Tournament 2003. nVidia, because they are not a member of FutureMark's beta program, did not have access to that developer version when testing the 5900 Ultra with 3DMark2003.
--And:
> During our analysis of Game Test 4, we paused the benchmark, went into the free-look mode, and moved "off the rail" in the 3D scene. Once we did that, serious drawing errors were readily apparent. We interpreted these results to indicate that nVidia was adding static clip planes into the scene. These static clip planes reduce the amount of sky that the GPU has to draw, thus reducing the pixel shader workload and boosting performance.
--I see no reason why NV would add optimizations for specific benchmarks into their drivers - it wouldn't make sense in the long run.
--My advice is to kindly suspend judgement on Nvidia on the basis of this so-called "evidence", at least until they come up with a driver upgrade.
.
== WolfriderV6 == I'm willing to admit that *I just might* be wrong... Are you??
Sweet, 3DMark should now include a "Set as Wallpaper" benchmark.
Well, in all fairness, there's gameplay reasons to do it too. The classic example is Quake3 -- there was a bug in the engine where your in-game avatar could literally jump higher if you were going at least 120fps, because of inaccuracies in the physics engine. As a result there's a number of tricks -- jumps from one area to another, etc. -- that are extremely common in tournament play in Quake3-based games but that are impossible to do unless you can get at least 120 fps out of your system.
Of course, if programmers did their job, you'd be absolutely right.
8x Anti-Aliasing and Antitrospic Filtering greatly enhances the look of any game. Newer games have a lot more textures, pixels to shades, and greater view depth. You want the latest and greatest card if you want to play the latest and greatest game for the next year at maximum settings. And some of the features such as AA and FA can greatly enhance some of the older games (making older games look great). Compare an ATI card with an nVidiot card on the later, the nVidiot card just can't do the job ATI can with AA and FA. They start to become unplayable if you are above 2X AA/FA at a decent resolution.
I'm not exactly sure why this is considered bad. GPU and driver developers are *always* designing and implementing optimizations and tricks one can use to avoid doing any work possible. That's simply part of computer science. Is using a Z buffer cheating? Is letting hardware take advantage of clipping planes cheating? Now, it's entirely possible that NVidia simply made an optimization that went too far and causes problems, and didn't pick up on it.
The way I see it, testers get cards. They then play games with them (Quake 3 used to be a popular one, but I suppose there are newer ones now). They then examine the frame rate, the smoothness, image quality, etc. If NVidia is using cards that produce a better image quality * smoothness, then more power to them -- I hope they make more of them. If they're using an optimization that degrades image quality...then the tester can complain about poor image quality. I don't think that claims of "cheating" are really warranted unless the card really is not rendering what a benchmark is feeding back -- like the driver looks for benchmark programs and actually modifies the benchmark score in memory. Hell, I wish testers wouldn't *use* benchmark programs in the first place -- if developers then optimize for that benchmark, the testers more than had it coming. Use Quake III or Tribes or something, not 3dmark2000, to do testing. Something that the end user will actually use the card for.
Consider lossy texture compression. It's becoming the standard way to deal with textures. Is that "cheating"? It degrades image quality to improve performance.
Now, I have my issues with NVidia. I won't buy any of their cards as long as their (usable) drivers are closed-source. I'm using Matrox cards (which have good open source support). I see the issue of open source drivers as a far more legitimate issue with NVidia than some claims of "cheating" on a benchmark.
Anyway, I suppose it's up to everyone to assign their own weight to these claims, but they aren't particularly interesting to me.
May we never see th
Thank you for being so bluntly honest. I suspect that you'll get modded down as a troll or flamebait, but I certainly agree with you.
Tech publications try very, very, very hard to appear to be ultimate gurus in all areas. When you talk to some of the people involved, you realize that they very much are not. They pick up a few anecdotes from people who are actually in the know, throw them down on paper in a knowing tone, slap some numbers on a piece of paper, and are done.
You know what I information I respect the most? Points that come from people that prefix some of what they say with "I think" -- which, of course, *never* occurs in journalism, even if the journalist in question does *not* know. I *think* he's expected to know because other folks do the same thing.
May we never see th
This is a big deal to people who care -- it insults the reviewers who spent hours benchmarking their card, and it insults the users who bought/will buy their card. There are people who care, and people who do want the fastest card for a reason, and they are interested to hear from other people who care, and not the people who don't!
These are the same people that would buy Porches or Armani suits. They're status symbols. In a social vacuum, the items would have very little value relative to their far less expensive variants.
May we never see th
The cooperative video game is a hideously underused concept. It's a ton of fun.
Try playing Halo (Bungie had a long tradition of cooperative games before Microsoft bought them) in cooperative mode against the computer. It's fun. Both people can win, and still have a challenge.
May we never see th
At some point, you can start doing motion blur (I'd estimate when you can render at 30x the vertical refresh rate), which reduces the choppiness in even a 60 fps game markedly.
May we never see th
I want open source drivers too. But that's *only* so that I can be sure that the product will keep working with future versions of the Linux kernel.
As has been well demonstrated in the past, 3d drivers are complicated enough and enough of a PITA that volunteers are relatively uninterested in working on them. Much of the existing DRI acceleration was funded by vendors and done under contract by people like Precision Insight.
Open source doesn't provide nearly as many benefits if it's hard to drop into a program and write a patch.
May we never see th
Your video card isn't doing that.
xv or whatever you used to set the background scaled it ahead of time. Your graphics card isn't doing any heavy lifting at all.
The difference is that Windows 2k's sucks, not that the Windows ATI drivers are worse than the Linux ATI drivers.
May we never see th
Well, yeah - it's the same video card. X vs. Windows 2000 is the difference, but the driver support under X is obviously fine.
Sorry to break it to you, but you're not just wrong, you're INCREDIBLY wrong.
Black-and-white television in the United States was broadcast at 60 Hz (60 fields per second, 2:1 interlace) because the mains cycled at 60 Hz. In the original televisions, if your mains frequency (the frequency at which the alternating current coming into the TV alternates) and your scanning frequency were different, you'd get an oscillating hum bar, or a moving visible distortion of the picture. If you scanned in sync with the mains, however, you got either no hum bar or a stationary hum bar; a stationary hum bar is far less noticeable than a moving hum bar.
The same story played out in Europe, where they use 50 Hz power and 50 Hz scanning in their TV system.
When America went to color television, the scanning frequency was changed slightly to 60/1.001, or about 59.94 Hz, in order to deal with an interference problem between the video and audio carrier signals. In Europe they used a different combination of carrier signals for their color broadcasts, so they didn't have the interference problem. Their TV's still run at 50 Hz today.
None of this has anything to do with what the eye can and cannot perceive. It's impossible to measure that in any objective way anyhow; the best you can do is take a survey and look for a trend.
But your final point was dead on: if you're letting your game free-run at a frame rate higher than the vertical refresh of your monitor, you ARE dropping frames. (Do the math. If your game frame window is shorter than your vsync window, you will eventually drop a frame. Precisely when depends on how far out of phase the two rates are. The best-case scenario is for your game to run at an integer multiple of your vsync, in which case you drop one or more frames for every video frame. Of course, since the graphics card is rendering twice or three times as many frames as you're actually seeing, the best way to characterize such a situation would be "vast waste of money.")
I don't think you can give one good example of how society is hurt by paying for the work of our hands and our minds.
What about films that deteriorate into dust because restoration societies cannot locate the copyright owner or because the copyright owner refuses to let the film be restored? Read this PDF for details.
Will I retire or break 10K?
5) Liability.
The GNU General Public License disclaims liability. Even with the Magnuson-Moss Warranty Act, which bans tying a warranty to exclusive use of products identified by brand name, NVIDIA can still void a warranty if it can show a preponderance of evidence that the third-party driver ruined the video card.
6) Nivida's Programmers Don't Want This.
What they want has no legal bearing. NVIDIA's drivers are works made for hire and belong to NVIDIA not to the individual programmers.
Will I retire or break 10K?
Careful saying ATI doesnt support alternate OS's, cause APPLE uses them and OS X is certainly an alternate OS!
Grandparent said "alternate" but meant "free software". Does ATI support free software operating systems and their windowing systems?
Besides, Mac OS X isn't an "alternate" operating system because it's the primary operating system for PowerPC architecture desktop and laptop computers. It ships with 99 plus percent of Macintosh computers.
Will I retire or break 10K?
Of course, since the graphics card is rendering twice or three times as many frames as you're actually seeing, the best way to characterize such a situation would be "vast waste of money."
Either that or "opportunity for a motion blur option in the Control Panel."
Will I retire or break 10K?