We're always on the lookout for augmenting our testing suite with benchmarks that do something different. The folks over at Futuremark have updated the popular 3DMark benchmark - one that we use in our graphics-card reviews - to include a stability tool, and it works in an interesting way.
Rather than tell you the hardware has an absolute pass, as a program like Prime95 or wPrime would do for the CPU, the 3DMark Stress Tests, available to users with registered copies of either the Advanced or Professional Editions, look at the average frame rate during a loop and compares it against the scores obtained in other loops. The premise is to weed out graphics cards that throttle frequency once a certain temperature limit has been reached, thus lowering performance compared to usually the first run.
The Stress Test runs 20 loops over a 10-minute period for Advanced Edition and up to 5,000 loops for the Professional Edition. A 'stable pass', defined as 97 per cent, requires that the average frame rate of the worst loop is within three per cent of the best loop. In other words, if the card scores 100fps during the best run then it cannot score less than 97fps during the worst-performing loop. Three per cent is a reasonable degree of variation.
If the graphics card scores less than 97fps, going by our example, due to it throttling under sustained load or for any other reason, then it is deemed a failure. So we thought it made sense to put in a reference Nvidia GeForce GTX 980 Ti, one known to run into the odd throttling issue, into one of our quiet systems and see what transpired.
Ouch!
Tested on a hot day with minimal airflow through the chassis, the card returned a resounding 'failure' in the Stress Test. Does this mean that we should castigate Nvidia for a substandard product? The answer, actually, is no, because, looking at GPU-Z logs, the slowest boost speed of our sample during the test is still higher than the 1,216MHz prescribed on the reference board. The card is performing within Nvidia-mandated tolerances.
What this kind of number really calls into question is the frame rates reported by websites when benchmarking cards. We let each card warm up for 60 seconds or so - usually by leaving it on the game's menu screen - before running our benchmarks, thus bringing a semblance of throttling into play. Cooling, too, becomes a concern if too many cards fail this test.
If you have the appropriate 3DMark on your system then give it a go and report back with your own findings below.
