Three chips in and Google Tensor is on life support

Google’s Pixel smartphones have always been at the cutting edge of the use of machine learning and AI to deliver unique experiences, all by leveraging Google’s software expertise creating clever algorithms. The original Pixel was the first Android smartphone to feature Google Assistant while it was also the first smartphone more generally to feature AI-powered computational photography features like Lens Blur and Smartburst. Each successive phone built on this legacy and right through to the Pixel 5, Google had partnered with Qualcomm to use its Snapdragon chips to power these AI and ML experiences. 

This changed with the launch of the Pixel 6 and Pixel 6 Pro in 2021, which was powered by the Google Tensor, its first customized chip. At the time, Google said its goal was to create its own chip so that it could “…deliver totally new capabilities for Pixel users by keeping pace with the latest advancements in ML.” The implication being, that if it had stuck with Qualcomm, it simply wouldn’t have been able to achieve its goals. Specifically:

Co-designing Google Tensor with Google Research gave us insight into where ML models are heading, not where they are today. This allowed us to build an AI/ML platform that could keep up with our work at Google.

It also highlighted how the Tensor enabled then new AI features like Live Translate to “work on media like videos using on-device speech and translation models.” As far as the original machine learning capabilities of the first Tensor goes, it delivered. If the Google Tensor series of chips was created specifically to support Google’s AI features - both keeping pace with Google’s AI models and supporting them on-device as intended - something has since gone awry. 

Developing market-leading customized chips is an expensive undertaking. Apple’s decision to invest in fully customized Arm-based chips dates back to the Steve Jobs era when it acquired a couple of chip designing companies specializing in low-power, high-performance silicon. Its first Arm-based Mac prototype also dates back to the Jobs era, with rumors of its existence first surfacing in 2011, the year before Jobs sadly passed away from pancreatic cancer. In the same year, Apple's chip team had already been built out to at least 1,000 strong. It has also invested heavily in TSMC to ensure priority access to its latest fabrication technologies. In short, Apple has invested many billions of dollars with its custom silicon efforts for the reason that it would deliver an unrivaled unison between its hardware and software.

Google has also wanted to emulate Apple’s approach by tailoring silicon to better align with its market-leading AI-powered software capabilities. However, to date, its efforts haven’t been quite as ambitious as Apple’s. The first three Tensor chips are considered semi-custom designs, having been co-designed and developed with Samsung LSI, the team behind Samsung’s own Exynos chips. They are also manufactured by Samsung Foundry. The provenance of the original Tensor and its close relationship with Samsung Exynos chips was identified by Andrei Frumansu who was then writing for Anandtech. (Frumansu, has since gone on to bigger and better things and is now a Principal Engineer at Qualcomm working on the Snapdragon X Elite chip -- so he knows a thing or two about chips).

So while Apple has been developing fully customized Arm-based CPUs for many years and, more recently, fully customized GPU cores as well, Google’s Samsung co-designed and developed Tensor chips have been using stock Arm Cortex CPU cores and Arm Mali GPU cores. They do, however, feature fully custom Google IP blocks including the Tensor Processing Unit (TPU), the Titan M secure enclave, and other customized IP blocks as outlined by Frumansu. This builds on the custom silicon IP that Google had developed when it was still using Qualcomm Snapdragon chips including the Pixel Visual Core first seen in the Pixel 2 and the first Titan M secure enclave that debuted in the Pixel 3. This has been the approach Google has stuck with for all its Tensor chips so far - the original Tensor, Tensor G2 and the latest Tensor G3. 

According to the latest rumors, Google has been designing a fully customized Tensor. This will be fabricated on one of TSMC’s 3 nm nodes instead of Samsung’s nodes - even though Samsung seems to be winning back some of the customers it has lost in the past two or three years to TSMC, it appears as though Google has lost its patience with the Korean giant. Unfortunately for Pixel fans, this chip isn’t expected to arrive until 2025, which means we can expect at least one more Tensor chip from its current hook-up with Samsung. Early signs aren’t, however, promising. Like the Tensor G2, which was a modest upgrade over the OG Tensor, the Tensor G4 is expected to be a modest upgrade over the Tensor G3. This could be a problem for Google given the Tensor G3 is unable to handle Google's latest AI algorithms on-device as per its original plans.

Even though the Tensor G3 significant architectural upgrades, its performance in benchmarks puts its performance between the mid-range Qualcomm Snapdragon 7+ Gen 2 and the Snapdragon 8+ Gen 1. While the performance is sufficient to keep the Pixel 8 Pro running the system UI smoothly and other tasks, it is still prone to getting warm doing simple tasks like downloading apps and shooting video in this writer’s experience with it. When phones get warm like this in general use, it is reflective of the type of inefficiency that is a known characteristic of chips produced on Samsung’s 5 nm and 4 nm nodes – including the 4LPP node used to fabricate the Tensor G3. Google has tried to mask these thermal deficiencies by loading the Pixel 8 Pro with lots of graphite. The same inefficiency is also apparent in numerous standardized battery tests found which typically show the Pixel 8 Pro well behind the iPhone 15 Pro Max and the Galaxy S23 Ultra, its two key competitors. 

The biggest issue, perhaps, with the Tensor G3 is (as we have highlighted recently), that many of the latest AI features that Google has advertised alongside the launch of the Pixel 8 Pro cannot be handled on-device. This is despite Google positioning the chip as being able to do so on its official blog. Although Google made it clear at its launch that the new Video Boost feature is off-loaded to the cloud for processing, it has not made it clear exactly which features require an internet connection. The only clue that AI processing is being off-loaded to the cloud for a new AI feature to work is where Google specifically denotes in the small print that a feature "requires Google Photos app". New AI features that the Tensor G3 is not powerful enough to process on-device include Google Assistant with Bard, Night Sight Video, Gboard proof-read, Magic Editor, AI Wallpaper, Audio Magic Eraser and the generative AI update coming to Magic Eraser.

It is clear that just three chips in, and with so many new AI features being dished off to Google Cloud, the Google Tensor project is on life support. You will recall Google said it switched from using Snapdragon chips as it had been doing because:

Co-designing Google Tensor with Google Research gave us insight into where ML models are heading, not where they are today. This allowed us to build an AI/ML platform that could keep up with our work at Google.

The Tensor G3 simply does not come close to delivering on Google’s original raison d'etre for creating its own chips. According to Google the benefits of a chip being able to process AI tasks on-device include: low latency (as the processing is handled locally); privacy (as user data isn't being off-loaded to remote servers); and improved battery life (as cellular or Wi-Fi connections drain power). Instead, by off-loading AI tasks to the cloud, latency on the Pixel 8 Pro is increased, privacy is diminished, and battery life takes a hit every time a user uses one of the new AI features that needs to use a cellular or Wi-Fi connection. 

In launching the powerful Snapdragon 8 Gen 3 just a few short weeks after the Tensor G3, Qualcomm did not miss the opportunity to send a message to Google that it should never have dropped them. During its launch event, Qualcomm placed extra emphasis on the on-device generative AI processing capabilities of Snapdragon 8 Gen 3, calling it a "Titan of on-device intelligence". Qualcomm was not exaggerating -- the Snapdragon 8 Gen 3 is the first mobile SoC with the ability to support Stable Diffusion on-device, and large AI models up with up to 10 billion parameters.

The Snapdragon 8 Gen 3 has also exposed Google’s assertion that "traditional performance metrics" being unimportant as a farce. The reason the Snapdragon 8 Gen 3 crushes the on-device AI processing capabilities of the Tensor G3 is for the very that it also crushes it in benchmarks. Qualcomm calls the combination of the Snapdragon 8 Gen 3 CPU, GPU and NPU has comprising its "AI Engine". This is because CPU, GPU and NPU of a smartphone SoC are utilized extensively when processing the machine learning models that power AI features on smartphones. Benchmarks show clearly that the Tensor G3 has a comparatively weak CPU and GPU, while its custom TPU isn’t enough to come close to compensating for this.

Raw horsepower matters, and it matters more than now than ever thanks to the rise of generative AI over the past 12 to 18 months. At this point, the only benefit of the Tensor G3 for Pixel fans is that Google has promised 7 years of OS updates for the Pixel 8 series, which is substantially more than would have been possible with Qualcomm. However, at this stage, this is merely a promise, and there have to be question marks about the ability for the Tensor G3 to be able to actually support Android 21, which is ostensibly the last of the OS updates the Pixel 8 Pro and Pixel 8 will receive.

Things don't look like improving much for the Pixel 9 Pro when it launches in 2024, as early indications are that the Tensor G4 will continue with the current CPU architecture of the Tensor G3, while it might receive a TPU and GPU upgrade. This would be similar in approach to the Tensor G2, which was a modest upgrade over the OG Tensor. It will only be in 2025 that we may finally see Google deliver a Tensor that can deliver on Google’s original promise, one that is fully customized, but -- crucially -- fabricated on TSMC's 3NE node. This should see the Tensor G5 much more competitive with other flagship chips on the market.

Right now, though, the Pixel 8 Pro is only just hanging on as it is -- can it survive a Tensor G2-like upgrade when an all-new architecture and a much better fabrication process is needed as soon as possible? An obvious short-term fix would be to drop a Snapdragon 8 Gen 3 in the Pixel 9 Pro next year. However, that would present a marketing challenge given the massive effort Google has tipped into the Tensor brand, before taking into account the sunk costs expended in developing the Tensor G4 and the reengineering costs of switching chips. It is so much of a hurdle, that it would appear Google is locked into its present trajectory. If Google proceeds as planned, it could well kill off any cache left in the Tensor and Pixel brands.

Even at this early stage, it appears that sales of the Pixel 8 series are on the soft side. While Google traditionally discounts its Pixels for the Black Friday sales, huge discounts of over $400 AUD were already available on the Pixel 8 Pro just four weeks after its launch and a week and a half ahead of the official start of Black Friday sales. This is extremely unusual. The Pixel series continues to offer great software and excellent camera performance. Google pioneered on-device AI features on smartphones, and its AI features remain class leading. But its Tensor chips are its Achilles heal.