OnePlus and Meizu caught cheating on benchmarks

Cheaters never prosper.

The OnePlus 3T has been touted as a solid choice for Android users looking for an affordable yet powerful phone. The 3T is one of the few Android devices powered by the powerful Snapdragon 821, which is currently Qualcomm’s most powerful SoC on the market. Upon its release, it blew reviewers away in synthetic benchmarks like AnTuTu and Geekbench. In our tests, it topped the list in our AnTuTu tests, but now there seems to be more to the picture.

Thanks to some investigative work by the team over at XDA Developers, we now know that the OnePlus 3T has been cheating on benchmarking tools. Mario Serrafero, editor-in-chief at XDA, noticed some odd behavior in the 3T that wasn’t present in other devices packing a Snapdragon 821. He decided to use some diagnostic software from Qualcomm to see how the SoC handled clock speeds in different apps. Serrafero found that in Geekbench and AnTuTu, the 821 in the OnePlus 3T would run at full tilt even when the CPU load dropped to 0%. Other devices, like the Google Pixel XL, would idle during these times.

To check his hypothesis, Serrafero reached out to Primal Ape, the developer behind Geekbench, to get to the bottom of the problem. Primal Ape quickly developed a new benchmarking version of Geekbench specifically for this test. When running the modified version of Geekbench, the OnePlus 3T would idle like normal.

Running the CPU at full speed during the entire benchmark gave the 3T a higher score in the benchmark, making it appear to be a much faster phone. Serrafero and the team at XDA determined that the OnePlus 3T recognizes app package names and runs the processor at higher speeds. The 3T detects when games and other apps were running and keep the SoC at boost. Two of the apps that trigger this behavior are AnTuTu and Geekbench 4.

OnePlus responded to the claim, saying:

In order to give users a better user experience in resource intensive apps and games, especially graphically intensive ones, we implemented certain mechanisms in the community and Nougat builds to trigger the processor to run more aggressively. The trigger process for benchmarking apps will not be present in upcoming OxygenOS builds on the OnePlus 3 and OnePlus 3T.

Meizu has also been fibbing on benchmarks with their flagship Pro 6 Plus, though in a different way. The Exynos 8890 powered device would recognize when Geekbench had started and prompt the user to enter “Performance Mode.” This would switch processing to the more powerful cores in the SoC, inflating the benchmark numbers. In other apps, the big cores would be disabled and work would be done by the lower-clocked small cores. Essentially, Meizu’s Performance and Balanced Modes are nothing more than a “parlor trick,” according to XDA. The modes essentially transfer work to the more powerful cores to trick benchmarking tools.

The saddest part about this whole debacle is that these software tricks would net minimal gains for the devices. There’s no telling what the fallout will be now that the deception has been unearthed, but the gains made by cheating the system probably won’t be worth the cost. The OnePlus 3T is still an excellent phone; it’s sad to see them resort to cheating for the smallest advantage over the competition.

Source(s)