Notebookcheck

OnePlus and Meizu caught cheating on benchmarks

The OnePlus 3T has been found to have cheated on benchmarking tools like AnTuTu. (Source: Phone Arena)
The OnePlus 3T has been found to have cheated on benchmarking tools like AnTuTu. (Source: Phone Arena)
Both companies' flagships were found to maintain high clock speeds instead of idling based on which app was currently opened. This happened with benchmarking apps like Geekbench and AnTuTu.

Cheaters never prosper.

The OnePlus 3T has been touted as a solid choice for Android users looking for an affordable yet powerful phone. The 3T is one of the few Android devices powered by the powerful Snapdragon 821, which is currently Qualcomm’s most powerful SoC on the market. Upon its release, it blew reviewers away in synthetic benchmarks like AnTuTu and Geekbench. In our tests, it topped the list in our AnTuTu tests, but now there seems to be more to the picture.

Thanks to some investigative work by the team over at XDA Developers, we now know that the OnePlus 3T has been cheating on benchmarking tools. Mario Serrafero, editor-in-chief at XDA, noticed some odd behavior in the 3T that wasn’t present in other devices packing a Snapdragon 821. He decided to use some diagnostic software from Qualcomm to see how the SoC handled clock speeds in different apps. Serrafero found that in Geekbench and AnTuTu, the 821 in the OnePlus 3T would run at full tilt even when the CPU load dropped to 0%. Other devices, like the Google Pixel XL, would idle during these times.

To check his hypothesis, Serrafero reached out to Primal Ape, the developer behind Geekbench, to get to the bottom of the problem. Primal Ape quickly developed a new benchmarking version of Geekbench specifically for this test. When running the modified version of Geekbench, the OnePlus 3T would idle like normal.

Running the CPU at full speed during the entire benchmark gave the 3T a higher score in the benchmark, making it appear to be a much faster phone. Serrafero and the team at XDA determined that the OnePlus 3T recognizes app package names and runs the processor at higher speeds. The 3T detects when games and other apps were running and keep the SoC at boost. Two of the apps that trigger this behavior are AnTuTu and Geekbench 4.

OnePlus responded to the claim, saying:

In order to give users a better user experience in resource intensive apps and games, especially graphically intensive ones, we implemented certain mechanisms in the community and Nougat builds to trigger the processor to run more aggressively. The trigger process for benchmarking apps will not be present in upcoming OxygenOS builds on the OnePlus 3 and OnePlus 3T.

Meizu has also been fibbing on benchmarks with their flagship Pro 6 Plus, though in a different way. The Exynos 8890 powered device would recognize when Geekbench had started and prompt the user to enter “Performance Mode.” This would switch processing to the more powerful cores in the SoC, inflating the benchmark numbers. In other apps, the big cores would be disabled and work would be done by the lower-clocked small cores. Essentially, Meizu’s Performance and Balanced Modes are nothing more than a “parlor trick,” according to XDA. The modes essentially transfer work to the more powerful cores to trick benchmarking tools.

The saddest part about this whole debacle is that these software tricks would net minimal gains for the devices. There’s no telling what the fallout will be now that the deception has been unearthed, but the gains made by cheating the system probably won’t be worth the cost. The OnePlus 3T is still an excellent phone; it’s sad to see them resort to cheating for the smallest advantage over the competition.

The OnePlus 3T was found to cheat on Geekbench for minimal gains. (Source: XDA Developers)
The OnePlus 3T was found to cheat on Geekbench for minimal gains. (Source: XDA Developers)
The Meizu Pro 6 Plus may not be as powerful as benchmarks would have you believe. (Source: XDA Developers)
The Meizu Pro 6 Plus may not be as powerful as benchmarks would have you believe. (Source: XDA Developers)

Source(s)

static version load dynamic
Loading Comments
Comment on this article
Please share our article, every link counts!
> Notebook / Laptop Reviews and News > News > News Archive > Newsarchive 2017 02 > OnePlus and Meizu caught cheating on benchmarks
Sam Medley, 2017-02- 1 (Update: 2017-02- 1)
Sam Medley
Sam Medley - Review Editor - @samuel_medley
I've been a "tech-head" my entire life. After graduating college with a degree in Mathematics, I worked in finance and banking a few years before taking a job as a Systems Analyst for my local school district. I started working with Notebookcheck in October of 2016 and have enjoyed writing news articles and notebook reviews. My areas of interest include the business side of technology, retro gaming, Linux, and innovative gadgets. When I'm not hunched over an electronic device or writing code for a new database, I'm either outside with my family, playing a decade-old video game, or sitting behind a drum set.