Notebookcheck

MediaTek allegedly cheated on a host of popular mobile benchmarking tests

Several MediaTek phones have been found to be cheating in benchmarks
Several MediaTek phones have been found to be cheating in benchmarks (image source: MediaTek)
MediaTek has been accused of artificially inflating benchmark scores on some of their devices. The shortcoming came to light when a reviewer found that a Helio P95 chip outperformed a Dimensity 1000L in PCMark. Some digging revealed that MediaTek has 'whitelists' for popular benchmarking apps. Whitelisted apps are allowed to push the device's SoC far beyond its limit, resulting in higher benchmark scores.

Although synthetic benchmarks seldom reflect the real-world performance of a device, they're a good indicator of what it is capable of under stressful environments. Furthermore, they're the only somewhat reliable way to measure the hardware prowess of a smartphone. Companies often rely on these scores to one-up their competitors. There have been several instances of OEMs cheating to get a higher score in apps such as AnTuTu, GeekBench, and more. Now, MediaTek just got caught artificially overworking its chips to get a higher score on some benchmarking applications.

MediaTek's Helio P95 outperformed the Dimensity 1000L in PCMark

Anandtech stumbled upon this discrepancy when one of its reporters noticed that Oppo's latest Reno3 Pro running an older MediaTek Helio P95 chip outperformed a Chinese version of the Reno3 that shipped with a MediaTek Dimensity 1000L SoC on PCMark. It struck out as odd because the Helio P95's Cortex-A75 CPU cores are two generations older than the Dimensity 1000L's Cortex-A77 CPU cores. Here is where the reporter suspected that there's something fishy going on in the background.

Most users, and even reviewers, will use standard versions of a benchmarking application. It is usually sourced from the Google Play Store or the company's website. OEMs are aware of this, and they 'whitelist' the application packages. It disables safeguards such as thermal throttling and forces the SoC to work at unsafe and potentially dangerous temperatures. As a result, some devices even shut down mid-benchmark. One of the ways to get around this predicament is to use anonymous versions of benchmarking software that is not on an OEM's 'whitelist.' Anandtech did just this with a mystery version of PCMark, and the results were significantly different. The overall score had dropped by 30%, and variances were as high as 75% in some tasks. 

Older MediaTek chips also engaged in similar behavior

It would be easy to blame Oppo for this mess, given that it was their device. Chinese OEMs have done this many times in the past, and this could very well be another instance of history repeating itself. However, the Chinese version of the Reno3 Pro running a Snapdragon 765G seemed to be performing on-par with its peers, which leads us to conclude that MediaTek is at fault here. Anandtech also found out that these 'whitelists' for benchmarking applications exist on a host of other MediaTek-powered devices such as the Oppo Reno Z, Oppo F15, and even the nearly three-year-old Sony Xperia XA1. MediaTek has been at this for a long time, based on the evidence presented. 

Anandtech reached out to MediaTek for a comment. You can read it in its entirety here. MediaTek doesn't confirm or deny the accusations and uses ambiguous language to distort what is happening with its hardware. The company acknowledges that its chips run on 'the highest possible performance levels' under synthetic loads. However, it maintains that the real-world performance of the chips is determined by a 'wide variety of factors' and that they're optimized for efficiency and battery life.

Historically, MediaTek chips have always lagged behind equally-powered Qualcomm offerings. Very often, one would find MediaTek chips exclusively in low-cost devices. Things got better for the company last year when the Helio G90T made its debut alongside the massively popular Redmi Note 8 Pro. MediaTek's new Dimensity line of 5G chips have the potential to take on, and even defeat their Qualcomm counterparts. Transgressions such as this one will do little towards changing a user's perception towards MediaTek-powered phones.

Source(s)

Read all 2 comments / answer
static version load dynamic
Loading Comments
Comment on this article
Please share our article, every link counts!
> Notebook / Laptop Reviews and News > News > News Archive > Newsarchive 2020 04 > MediaTek allegedly cheated on a host of popular mobile benchmarking tests
Anil Satyanarayana, 2020-04- 8 (Update: 2020-04- 8)
Anil Satyanarayana
I've been an avid PC gamer since the age of 8. My passion for gaming eventually pushed me towards general tech, and I've been a stereotypical 'nerd' ever since. I have a degree in mechanical engineering and have worked in the manufacturing industry. When I'm not writing news on Notebookcheck, you can find me playing or watching an intense match of DOTA 2