Principled Technologies, the company behind the benchmark results used by Intel to back up claims that its 9th gen Core i9-9900K processor is the “world’s best gaming processor,” came under fire earlier this week after the tech and gaming press had had time to digest the small print in Intel’s presentation slides. HEXUS didn’t share the slides comparing the Core i9-9900K with the AMD Ryzen 7 2700X, only an intergenerational performance comparison slide. Then we noted that the 9th gen processors would start shipping on 19th October, on the very same day our review embargo is lifted. Not a great situation for any product – third party reviews are important, and perhaps this Principled Technologies (PT) episode proves that…
A few hours ago PT issued a statement about its testing methodology (PDF) in response to various questions. From the PDF statement we read that PT, with its 15+ years of testing products for clients such as Intel and AMD, Microsoft and Google, Dell and HP, and many others, denies any wrong doing. It highlights that for the purpose of transparency it previously published an interim summary report (PDF) on its tests (on 8th Oct). You should look at this PDF link to see all the test findings from PT comparing eight Intel and AMD processors in 19 popular modern games.
As an example of the test results I’ve reproduced just one of the results tables below, for Assassin’s Creed Origins. And it all looks good so far... It is only when the test settings later on in the document come under scrutiny that people start to see some potential for unfairness. And that is what PT’s follow up PDF seeks to address.
Key points of contention in PTs test methodology are; its use of Game Mode on AMD Ryzen processors (halving core/thread use on the 2700X), air cooler cooler choice on AMD platform, use of 4x 16GB of underclocked RAM in the test systems, and 1080p resolution gaming on the GTX 1080 Ti graphics card. On the first three of these concerns PT answered as follows:
- Use of "Game Mode" on the AMD Ryzen 7 2700X: Some inquiries we have received concern the use of the Ryzen utility and the number of active cores in the AMD-based systems. Based on AMD's recommendations and our initial testing on the Threadripper processors, we found installing the AMD Ryzen Master utility and enabling the Game Mode increased most results. For consistency purposes, we did that for all AMD systems across Threadripper and Ryzen. We are now doing additional testing with the AMD systems in Creator Mode. We will update the report with the new results.
- Cooler choice: We chose Noctua for the CPU coolers, due to having almost identical systems in the NH-U14S (Intel) and NH-U14S TR4-SP3 (AMD), which allowed us to maintain a comparable thermal profile. Because we were not performing any overclocking on any configuration, and because AMD has said it was a good cooler, we stuck with the stock AMD Ryzen 7 2700X Wraith Prism cooler.
- Memory speeds: To have complete parity across all systems, and to allow the Intel Core i9 X-series and AMD Ryzen Threadripper to fully utilize memory bandwidth, we used four 16 GB DDR4 DIMMs on all configurations.
On the topic of gaming at Full HD, PT said that this is the most common resolution in the most recent Steam Hardware Survey and allowed it to minimise the possibility of GPU-based bottlenecks. Overall, PT stands by its methods and denies any bias towards Intel.
If you are interested in the topics mentioned above and have 45 minutes to skip through a YouTube video interview it might be worth a gander at Gamer’s Nexus exclusive chat with one of the company owners.
As usual, HEXUS will be publishing a full and thorough review of these latest Intel processors, and new motherboards, as soon as we can. This might be the best way for potential purchasers to see how these chips compare in the real world and stack up against the competition before parting with their hard earned cash.
UPDATE:
PT has published revised benchmark scores including test results for the AMD Ryzen 7 2700X in default mode (i.e., Creator mode). It makes quite a considerable difference. As I previously shared the Assassin's Creed Origins scores (image above), I have included the updated set of results below. There are many more tests in the updated benchmarks PDF for those interested to delve further.
