Domain: futuremark.com
Stories and comments across the archive that link to futuremark.com.
Comments · 65
-
Not a fair testApparently both machines were running Linux and had 512mb ram each, but not much else was the same. Did the hard drives have the same filesystem even? Could one have been more fragmented (i.e. the one the author of the article uses all the time before copying the files to the SCSI test system?)? Before buying any hard drive you should check out benchmarks on it. IDE drives with the same size, cache, and rotational speed can have more than 2x difference in benchmarks.
I'd also like to know what IDE controller was used. I was sorely dissappointed in the performance of the hard drive that came with my Dell computer using the on-board controller. I could really tell it was slow switching applications and booting up. It showed a rather pathetic score of 398 on PCMark 2002. I thought "to hell with that" and bought a new 160gb hard drive that was rated really well on tom's hardware. My score actually dropped to 244. That didn't make any sense. I updated all the drivers I could to no avail. Finally I bought a PCI IDE card at CompUSA for about $40 to try it out. The score jumped to 1216. That's almost up to the 1400 I have at home with my IDE raid setup, and much more than the 800 or so my brother got with his 10,000 rpm SCSI Cheetah drive.
Download the free version of PCMark 2002 and check your own hard drive scores.
-
Re:This is fake
planethalflife think it's fake as well, and there isn't the firestorm I'd expect there to be if this was for real over at futuremark. Basically I think someone is playing silly buggers.
On the other hand, Steam will have the ability to read and write to your half-life 2 directory - it'll be able to upgrade your game or install mods, cf interview here. Which is nearly as potentially troublesome.. -
You actually *believe* hardocp?
HardOCP's coverage of all this is disgraceful. When Extremetech originally broke the story, HardOCP practically accused them of making it up, and said they had "motives of their own" for writing the article outlining the problem. Instead of investigating on their own, apparently the procedure at HardOCP is to question the findings of the other, more competent, tech sites.
Then, when the fix is posted, they write "This is in response to the news item we posted last week."
... As if _they_ broke the story. As if _they_ are responsible for causing a patch to be posted. No apology to Extremetech, either (in fact, no mention of them at all)
And now, they're making unfounded accusations that 3DMark is taking bribes to skew the benchmark results? WTF? Why doesn't HardOCP just hire Jayson Blair to write their "articles"? At least then, they'd have less spelling errors. -
Re:Apple's SPEC scores are false.
After all, writing special cases in the compile for good benchmark results is a _very_ old trick (Sieve OF Erasthones)...
Not to nit-pick too much, but it's the Sieve of Eratosthenes. :-) Or, "One of them Greek guys."
On a more serious note, I remember studying benchmarks in grad school, and how crappy benchmarks are, and how meaningless. Compiler vendors routinely tweak compilers to recognize blocks of code used in benchmarks and generate highly tweaked code to make those benchmark results look better than they should be. Case in point, there was a Fortran benchmark that was used heavily in comparing hardware, and one Fortran compiler was tweaked by the hardware vendor to super-optimize a block of code so it ran far, far faster than usual. The problem was, the optimization, although valid for all possible inputs, was something only a human with advanced knowledge of calculus and transcendental math could have seen/figured out. And this optimization only worked for the benchmark suite, IIRC.
As we've seen lately with video card vendors, this game is still played even today. I, for one, am glad that Apple did their comparisons using the same compiler (and an Open Source one at that) on both platforms. It's a very fair comparison. -
Alas, not legit......
NVidia did things that were clearly NOT legitimate, and FutureMark caught them at it. There's a PDF report on FutureMark's Web site (assuming it hasn't met with an "accident" by now) detailing the dirty deeds. Chief among them, IMHO, was a trick where the driver was supposed to draw and update positions of stars in a night sky (involving clearing the background) as one moved along a 3D path; if one stays on the exact preprogrammed track of the demo, it looks OK. BUT... if you turn around (possible in the beta mode of the benchmark) you see that the driver SKIPPED clearing the background; the stars smear like mad. There is NO POSSIBLE WAY their driver was behaving legitimately. (Especially since changing the benchmark's fingerprint oh-so-slightly caused all these quirks to vanish; they were detecting the demo and screwing with things if it was being run...) The rest is just fear-of-pissing-off-the-800-pound-gorilla. A FutureMark developer admitted as much in a newsgroup posting. Sigh...
-
Article text:
Futuremark Statement
For the first time in 6 months, as a result of Futuremark's White Paper on May 23rd, 2003, Futuremark and NVIDIA have had detailed discussions regarding NVIDIA GPUs and Futuremark's 3DMark03 benchmark.
Futuremark now has a deeper understanding of the situation and NVIDIA's optimization strategy. In the light of this, Futuremark now states that NVIDIA's driver design is an application specific optimization and not a cheat .
The world of 3D Graphics has changed dramatically with the latest generation of highly programmable GPUs. Much like the world of CPUs, each GPU has a different architecture and a unique optimal code path. For example, Futuremark's PCMark2002 has different CPU test compilations for AMD's AthlonXP and Intel's Pentium4 CPUs.
3DMark03 is designed as an un-optimized DirectX test and it provides performance comparisons accordingly. It does not contain manufacturer specific optimized code paths. Because all modifications that change the workload in 3DMark03 are forbidden, we were obliged to update the product to eliminate the effect of optimizations identified in different drivers so that 3DMark03 continued to produce comparable results.
However, recent developments in the graphics industry and game development suggest that a different approach for game performance benchmarking might be needed, where manufacturer-specific code path optimization is directly in the code source. Futuremark will consider whether this approach is needed in its future benchmarks.
NVIDIA Statement
NVIDIA works closely with developers to optimize games for GeForceFX. These optimizations (including shader optimizations) are the result of the co-development process. This is the approach NVIDIA would have preferred also for 3DMark03.
Joint NVIDIA-Futuremark Statement
Both NVIDIA and Futuremark want to define clear rules with the industry about how benchmarks should be developed and how they should be used. We believe that common rules will prevent these types of unfortunate situations moving forward.
About Futuremark\x{00AE} Corporation
Futuremark\x{00AE} Corporation, formerly known as MadOnion.com, is the leading provider of computer performance analysis software and services. Futuremark\x{00AE} is known around the world for its benchmark products, including the 3DMark\x{00AE} Series and PCMark2002 (with more than 30 million copies distributed worldwide) and value-added services powered by a database of over 5 million real life benchmarking results. Futuremark\x{00AE} has offices in Saratoga, California and Helsinki, Finland. For more information, please visit http://www.futuremark.com.
\x{00A9} 2003 Futuremark\x{00AE} Corporation. 3DMark\x{00AE} and PCMark trademarks and logos, Futuremark\x{00AE} character names and distinctive likenesses, are the exclusive property of Futuremark Corporation. DirectX\x{00AE} is a registered trademark of Microsoft Corporation. All other trademarks are property of their respective companies.
About NVIDIA
NVIDIA Corporation is a market leader in visual computing technology dedicated to creating products that enhance the interactive experience on consumer and professional computing platforms. Its graphics and communications processors have broad market reach and are incorporated into a wide variety of computing platforms, including consumer digital-media PCs, enterprise PCs, professional workstations, digital content creation systems, notebook PCs, military navigation systems and video game consoles.
NVIDIA is headquartered in Santa Clara, California and employs more than 1,500 people worldwide. For more information, visit the Company's Web site at www.nvidia.com.
Certain statements in this press release, including any statements relating to the Company's performance expectations for NVIDIA's family of products and expectations of continued revenue growt -
Re:time?
www.futuremark.com
I was using 3dMark 2001, but 2003 is out. Download the demo (if they still offer it) and sign up with the profile manager.
Then you can do 2 neat things, Compare by CPU and Compare by GFX Card. So if can see what a faster CPU does with the same GFX card, or the same cpu with Faster GFX cards. Really ace. -
Re:To add an optimization is fraud? Ridiculous
See the section marked 'Aren't These Cheats Just Optimizations...' here. These aren't driver optimisations. The drivers are claiming to do work they are not actually doing.
-
Re:3D Mark Score
I haven't done nearly as much as I can to up the score yet, I'm sure I'm capable of breaking 5000. As it stands, my score is 4903. Here is a compare link for all those interested. I honestly believe that nVidia is just disappointed that their card performs so poorly in 3DMark03, and it's probably more of a problem on nVidia's side then on Futuremark's side. FM spent a lot of time working on this benchmark, and from everything that I've seen officially stated, the benchmark should be an accurate representation of future games. nVidia still has time to make their drivers more mature anyway...
-
A Radeon would make my system three times as fast?
My box:
CPU: AMD Athlon(tm) XP/MP/4 1532 MHz
GPU: NVIDIA GeForce4 Ti 4200
Score: 1294
Now, if I go to the Futuremark Project Search, enter my exact system, and change out my GF4 for a Radeon 9700, the top results are in the 4500 range. Which would suggest that I could get a 200% performance boost by buying an ATI card. However, game benchmarks suggest the 9700 would buy me a much smaller gain (5-50%, depending on the game/resolution/options). IMO, 3dmark places a bit too much emphasis on the newest DX9 features to be useful as a general performance benchmark. -
Lack of Cinfx featuresWhen T&L came out FutureMark jumped at it. SSE, 3dnow, etc... are all included in FutureMarks releases. FutureMark suggesting that 3dmark isn't bias and as policy shuns proprietary extensions is misleading. BABCo (a collection of company representatives to insure fair testing) is listed on FutureMarks homepage here but uses an outdated logo which neglects to show ATIs membership (BABCo membership page look under the logo shows ATIs membership). Nvidia isn't as of yet a member of BABCo, whether it be money or other, I'm not sure.
At present the ATI card is faster, but when you begin adding more shaders/textures the nvidia card is vastly superior and leaves ATIs card severly underpowered. Look at multitexture (textures per pass) capabilities here of each card. In hindsight, I wish Nvidia did go for the 256 bit engine, much of this argument would not currently exist.
I am a Nvidia fan but not a fanatic, FutureMark by design does favour the newer Radeon cards over the newer Nvidia cards, whether or not it is by intent, I'm in no position to say. But, if I was allowed to speculate, I'd suggest a little industrial espionage (meaning indepth knowledge of the FX card) on ATIs behalf existed long before 3dmarks release, there's certain aspects of 3dmarks release that are questionable or at least circumstantial. Which does suggest "a spy amoung us" scenario.
I do believe ATI is pawning off a substandard card and is once again lying about it (Rage Maxx).
Note:I accidently posted as AC (cookies), thus the repost.
-
Cinfx and T&LWhen T&L came out FutureMark jumped at it. SSE, 3dnow, etc... are all included in FutureMarks' releases. FutureMark suggesting that 3dmark isn't bias and as policy shuns proprietary extensions is misleading. BABCo (a collection of company representatives to insure fair testing) is listed on FutureMarks' homepage here but uses an outdated logo which neglects to show ATIs' membership (BABCo membership page look under the logo shows ATIs' membership). Nvidia isn't as of yet a member of BABCo, whether it be money or other, I'm not sure.
At present the ATI card is faster, but when you begin adding more shaders/textures the nvidia card is vastly superior and leaves ATIs' card severly underpowered. Look at multitexture (textures per pass) capabilities here of each card. In hindsight, I wish Nvidia did go for the 256 bit engine, much of this argument would not currently exist.
I am a Nvidia fan but not a fanatic, FutureMark by design does favour the newer Radeon cards over the newer Nvidia cards, whether or not it is by intent, I'm in no position to say. But, if I was allowed to speculate, I'd suggest a little industrial espionage (meaning indepth knowledge of the FX card) on ATI's behalf existed long before 3dmarks release, there's certain aspects of 3dmarks release that are questionable or at least circumstantial. Which does suggest "a spy amoung us" scenario.
I do believe ATI is pawning off a substandard card and is once again lying about it (Rage Maxx). -
Re:Silly arguments...One of the primary reasons for the criticism of 3DMark2003 is the fact that it *DOESN'T* use DX9 extensively. Pixel shader 1.1 and 1.4 are primarily used, which is absolutely laughable
Uhh.. You didn't read the reply, did you? OK, I thought so. Here's an excerpt from it:
The argument here is that game test 4 is not "DirectX 9 enough". Once again, a good application should draw a scene as efficiently as possible. In the case of game test 4 this means that some objects use Pixel Shaders 2.0, and some use 1.4 or 1.1 if a more complex shader is not required. Because each shader model is a superset of the prior shader models, this will be very efficient on all DirectX 9 hardware. In addition, the entire benchmark has been developed to be a full DirectX 9 benchmark[...]
(emphasis mine)Do you think your web browser should use DirectX 9 pixels shaders to render text, too?
-
You want scores?
IF you've run the benchmark, post your scores here, and we'll all compare.
Or you could just go directly to the futuremark forums instead. -
Re:Actually...
prove what? and since you replied nameless i don't think you even want the proof.. and since you could easily look up some benches..
well.. eh.. here's some info.
http://www.futuremark.com/community/halloffame/
at all the lists this order is common:
--
5. ATI RADEON 8500/LE
6. NVIDIA GeForce3 Ti 500
--
as to 'came out months before' i don't actually know, and i don't even care.
the nvidia card that is priced like 8500le is gf4mx, which is just another mx joke as far as performance goes.
and 8500's are about the same price or cheaper than the few gf3's still for sale.