Slashdot Mirror
New GPU Testing Methodology Puts Multi-GPU Solutions In Question
Vigile writes "A big shift in the way graphics cards and gaming performance are tested has been occurring over the last few months, with many review sites now using frame times rather than just average frame rates to compare products. Another unique testing methodology called Frame Rating has been started by PC Perspective that uses video capture equipment capable of recording uncompressed high resolution output direct from the graphics card, a colored bar overlay system and post-processing on that recorded video to evaluate performance as it is seen by the end user. The benefit is that there is literally no software interference between the data points and what the user sees, making it is as close to an 'experience metric' as any developed. Interestingly, multi-GPU solutions like SLI and CrossFire have very different results when viewed in this light, with AMD's offering clearly presenting a poorer, and more stuttery, animation."
That unpossible. :-P
Lost at C:>. Found at C.
My AMD is cranking out Bitcoin hashes 15 times faster than an equivalently priced Nvidia so I'm okay with the results of this article.
I'm sure that AMD, the losing party, will dispute the results and come up with its own methoology to counter this.
Then again, everyone knew nVidia high end cards are better, so was this new test really necessary??
As an owner of a Crossfire setup, it's obviously not a 2x improvement over a single card; however, it's also a marked improvement over a single card. When I first set up this rig (August), I had problems with micro-stutter.* Now, though, after AMD's newer drivers and manually limiting games to 59 FPS, I don't see it anymore; games appear smooth as silk.
At a mathematical level, it may not be a perfect solution, but at a perceptual level, I am perfectly satisfied with my purchase. With that said, buying two mid-line cards instead of one high-end card isn't a good choice. Only buy two (or more) cards if you're going high-end.
*I was initially very disappointed with the Radeons. That's no longer the case, but I will probably still go nVidia the next time I upgrade, which hopefully won't be for years.
If you can't convince them, convict them.
It started when people began to look not only at average frame rate, but at *minimum* frame rate during a benchmark run. That shows how low the FPS can dip, which was the beginning of acknowledging that something in the user-experience mattered beyond average frame rate. It has gotten a lot more advanced, as pointed out in the article here, and this sort of information is very helpful for people building or buying gaming computers. I use info like this on an almost daily basis to help my customers get the best system for their needs, and I greatly appreciate the enthusiasts and websites which continue to push the ways we do testing!
William George
I thought having a second GPU would be a good upgrade path. Boy was I ever wrong.
None of the soulutions scale gracefully. Both games and drivers have to be tweaked on a case by case basis before performance becomes good.
Often new cutting edge games work like shit on launch day.
I decided to ditch the scheme when the only way to make both max payne 3 and skyrim playable on launch day was to pull out one of my radeons.
99th percentile frame times. That gives you a realistic minimum framerate, discarding most outliers (many games, particularly those using UE3, tend to have a few very choppy frames right on level load, that don't really affect performance).
This is interesting from a developer standpoint as well. This means we are wasting processing time rendering frames and are only displayed for a handful of milliseconds. These frames could be dropped entirely and that processing time could go to use elsewhere.
nVidia and AMD both have to bribe game developers to get them to consider threaded rendering at all. Coupled with the bugs in the respective companies' implementations you have lost entire generations of hardware purchases to the simple economics of "if I buy a better single-GPU card now, I get more value for my money than multiple lesser GPUs that might not even work."
What is a multi-GPU solution? Liquid Hydrogen?
Solutions are all tech articles talk about these days. Shouldn't electronics be kept safe from liquids?
If I'm buying a GPU for a video game, I only care about how it benchmarks on video games.
Why would the study of methods to ascertain representative GPU speed change multiple GPU solutions?
Do you mean a new testing method perhaps?
It seems to me that lots of times when AMD or ATI back in the day made a damn good product most testers would go out of there way to emphasize Intel/Nvidia's good points, while emphasizing the bad. It's made me suspicious as all hell.
Personally out of all the different products I've used I've been most impressed by AMD and ATI.
Hell, I am running on an i7 and GTX 260 and it stutters while playing Synthesia. It's terrible.
But the Radeon 7570 (I think thats the model) that I put in my buddies old Core2 duo setup runs everything he's thrown at it.
As a long-time GTX 295 owner, I've known for quite a while that my eyes are really good at seeing stuttering. For a few years, my GTX 295 did a splendid job keeping up with games, and as long as I could manage 60 FPS everything went seemed pretty smooth. I did have a few moments where I did see micro-stuttering but I found that either enabling V-sync or enabling frame limiting solved the problem. As you can see in this diagram http://www.pcper.com/files/review/2013-02-22/fr-3.png it's very possible that your GPUs synch up perfectly, producing two frames at the same time effectively causing every other frame to never reach the screen. Forcing games to synchronize to a certain FPS either via V-sync or frame limiting therefore helps the GPUs render at perfect intervals, assuming your hardware can render at that speed (usually 60 FPS).
And there's my point. SLI and Crossfire works perfectly fine as long as your hardware can pump out 60 FPS so you can synch it to that frame rate. As soon as it drops below that for even just a few seconds, it's easy as hell to spot a drop to 55 FPS that looks like 30-40. Therefore, SLI and Crossfire has tremendous value at their time, but are far from future-proof since as soon as you go under 60 FPS you basically drop to singe GPU framerates. For me, that wall was hit around when I started playing BFBC2, and I'm forced to play BF3 at low to ensure a minimum 60 FPS at all times.
A multi-GPU solution is a great investment for enthusiasts who change upgrade their hardware every year or two, but a horrible mistake for people expecting it to last several years.
The methodology to use FRAPS to capture the frame rate is flawed at several levels.
To start with, FRAPS will take system resources, IO, CPU and probably on certain configurations even GPU resources.
The "overlay" is sort of interesting, BUT doesn't reveal what is actual frames or not.
They will need to get back with a better solution for measuring real frames per second.
Besides, 60 fps, which is the refresh of the "image" is more then the human eye can distinguish (i will not enter into the actual details of the maximum screen refresh rates, i will leave that to you, to do the math of screen mhz into max FPS that they can render).
There is also the issue of the actual screen materials that have a max capability of changing luminescence (which may or may not be in sync with the max screen refresh rate).
So interesting analysis, but i fail to see where the result is (if you are measuring actual frames, or actual limitations of any of the mentioned issues).
I discovered this about a year ago, when i wanted to add a 3rd monitor to my system, and discovered I couldn't do it in Crossfire mode with my dual 4850s, but COULD do it if i turned it off. Productivity being slightly more important to me than game performance, I turned it off and hooked up my 3rd monitor.
A few days later I decided to fire up Skyrim, and didn't notice any discernible drop in performance at all. My settings were all on medium, just because the cards were a few years old, but still, I expected to see *some* performance hit going from 2 GPUs to 1.
I have since upgraded to a 7950 that supports 3 monitors, but I'm never doing that SLI/Crossfire shit again.