Slashdot Mirror
New GPU Testing Methodology Puts Multi-GPU Solutions In Question
Vigile writes "A big shift in the way graphics cards and gaming performance are tested has been occurring over the last few months, with many review sites now using frame times rather than just average frame rates to compare products. Another unique testing methodology called Frame Rating has been started by PC Perspective that uses video capture equipment capable of recording uncompressed high resolution output direct from the graphics card, a colored bar overlay system and post-processing on that recorded video to evaluate performance as it is seen by the end user. The benefit is that there is literally no software interference between the data points and what the user sees, making it is as close to an 'experience metric' as any developed. Interestingly, multi-GPU solutions like SLI and CrossFire have very different results when viewed in this light, with AMD's offering clearly presenting a poorer, and more stuttery, animation."
My AMD is cranking out Bitcoin hashes 15 times faster than an equivalently priced Nvidia so I'm okay with the results of this article.
I'm sure that AMD, the losing party, will dispute the results and come up with its own methoology to counter this.
Then again, everyone knew nVidia high end cards are better, so was this new test really necessary??
The point of the "new test" is that framerate is a terrible metric because it averages out what you care about.
When you measure frame times individually you can then quantify how often a game slows down and by how much.
You don't just have an average FPS, or a MAX/AVG/MIN.
As an owner of a Crossfire setup, it's obviously not a 2x improvement over a single card; however, it's also a marked improvement over a single card. When I first set up this rig (August), I had problems with micro-stutter.* Now, though, after AMD's newer drivers and manually limiting games to 59 FPS, I don't see it anymore; games appear smooth as silk.
At a mathematical level, it may not be a perfect solution, but at a perceptual level, I am perfectly satisfied with my purchase. With that said, buying two mid-line cards instead of one high-end card isn't a good choice. Only buy two (or more) cards if you're going high-end.
*I was initially very disappointed with the Radeons. That's no longer the case, but I will probably still go nVidia the next time I upgrade, which hopefully won't be for years.
If you can't convince them, convict them.
It started when people began to look not only at average frame rate, but at *minimum* frame rate during a benchmark run. That shows how low the FPS can dip, which was the beginning of acknowledging that something in the user-experience mattered beyond average frame rate. It has gotten a lot more advanced, as pointed out in the article here, and this sort of information is very helpful for people building or buying gaming computers. I use info like this on an almost daily basis to help my customers get the best system for their needs, and I greatly appreciate the enthusiasts and websites which continue to push the ways we do testing!
William George
Mind you, the only thing worse than stuttery animation is jerky sound.
Ezekiel 23:20
99th percentile frame times. That gives you a realistic minimum framerate, discarding most outliers (many games, particularly those using UE3, tend to have a few very choppy frames right on level load, that don't really affect performance).
This is interesting from a developer standpoint as well. This means we are wasting processing time rendering frames and are only displayed for a handful of milliseconds. These frames could be dropped entirely and that processing time could go to use elsewhere.
It's just tragic that all the other 3D graphics vendors - 3D!fx, 3DLabs, Matrox, et al went out of business or exited the market. The NVidia-ATI duopoly (or the NVidia-ATI-Intel triopoly) has really trimmed things down.
The new methodology shows that nVidia SLI gets excellent results. However, I've owned SLI solutions, both two 285s and a 290. Neither one was satisfactory. The higher failure rate was of particular note. One of the 285s failed, I RMAd it, got a 290, eventually that failed... On top of that, even when things were working, there were lots of problems. Some games didn't run correctly. Crysis, for example, didn't seem to like SLI; it would get into a bizarre state where the screen would begin flickering rapidly between black and normal frames, and you'd have to restart the game to recover. The problem never occurred in the same machine when I disabled SLI.
After that experience, both with dual-card SLI and single-card SLI, I decided it just wasn't worth it, and that my money was better spent on a good single-GPU solution. I currently have a single GTX 670, which works fine.
Mind..... you, the o......only thing w..w....w....w...orse thanananananan stutttttttttttttttttttttttery animation is jerkyerkyerky so........und.
FTFY
Ezekiel 23:20