Update | Intel's commissioned 9900k benchmarks have serious flaws, says PC hardware reviewer

Walton's frustration with Intel is even visible in the thumbnail of his video titled "Intel's New Low." (Source: Hardware Unboxed)

When Intel commissioned Principled Technologies to benchmark its as of yet unreleased i9 9900k CPU, many voices in the tech press spoke out against such a practice, such as Steve Walton, saying it "undermines what we do." Now, more reliable figures have come out, and it seems like Principled Technologies did not run an unbiased test.

Matthew Connatser, Published 10/09/2018

Well before reviews will be coming out for the 9900k, Intel has commissioned Principled Technologies to benchmark the company's new CPU. In response to this study, some PC hardware reviewers have spoken out against the practice; Steve Burke of GamersNexus said, "it's fucked up. Full stop," and Steve Walton of Techspot and Hardware Unboxed said the study "undermined what we do" and that it was "misleading." Walton actually published his own benchmarks of the 2700X and the 8700k to compare with the results Principled Technologies found.

Walton's criticisms are directed towards the testing methodology found within the study and not necessarily the results in particular. He claims that the study gave Intel an unfair advantage by enabling XMP (or for AMD, AMP) only for Intel platforms, which gave Intel CPUs tighter memory timings. AMD's processors, including the 2700X, had to run on the default timings, which are much looser for the sake of stability. Walton also points out that the type of memory used was not optimal for gaming on Ryzen: dual rank, fully populated slots as opposed to single rank, half populated slots.

The most egregious act in Walton's opinion, however, was setting the 2700X to "Game Mode," a setting intended for Threadripper CPUs to perform better in games. While this clearly benefited the 2950X and 2990WX (which beat the 2700X, an unusual instance), the 2700X was actually demoted from 8 cores to 4 cores. When running games such as Assassin's Creed: Origins, using only 4 of 8 cores hampered the 2700X, causing what Walton estimates to be a nearly 20% performance penalty, just by turning on "Game Mode."

Upon testing "Game Mode," Walton said:

This is unbelievable, I don’t know if they are being extremely malicious or it’s just incompetence of the highest order. How do you take note of every last setting to be documented but not realize just 4-core/8-threads are active on an 8-core/16-thread processor?

While Walton concedes that the 9900k will be faster than the 2700X, he said that it wouldn't be by the 50% shown in some of the commissioned benchmarks, and that you might be looking at about double the cost of a 2700X once a motherboard is factored in for both CPUs; it should be noted however that 9th generation CPUs are compatible with existing 300 series chipsets. Walton had also previously tested the 8700k and the 2700X against each other and found that the 8700k was only 9% faster at 1080p across dozens of games.

Update: Intel's statement:

We are deeply appreciative of the work of the reviewer community and expect that over the coming weeks additional testing will continue to show that the 9th Gen Intel Core i9-9900K is the world’s best gaming processor. Principled Technologies conducted this initial testing using systems running in spec, configured to show CPU performance and has published the configurations used. The data is consistent with what we have seen in our labs, and we look forward to seeing the results from additional third party testing in the coming weeks.