It is a well-known fact that Apple and Nvidia don't get along too well. There was a time when Apple's Mac lineup did make use of Nvidia graphics, but things went south when in 2008, widespread issues in MacBooks with Nvidia's GeForce 8400M and 8600M dGPUs began surfacing due to packaging defects, leading to overheating, graphical glitches, and in extreme cases, the GPU failing entirely. Although MacBooks did continue to have Nvidia GPUs, their relationship was undoubtedly on a nosedive, and within the next few years, Apple switched to AMD and never looked back.
As a result, and no offense to fans of Team Red, Macs suffered from poor graphics performance compared to GTX and RTX-powered Windows laptops. That was until Apple Silicon-powered Macs came along, of course. Despite Mac enthusiasts holding out for Nvidia and Apple to bury the hatchet, it never really happened - until now. Well, sort of. Apple recently announced a collaboration with Nvidia aimed at accelerating large language model inference, marking a welcome armistice of sorts between the two multi-trillion dollar behemoths.
At the heart of this unexpected collaboration is Apple's ReCurrent Drafter (ReDrafter), which is basically a rather innovative speculative decoding method designed to improve LLM token generation launched earlier this year. Nvidia has apparently worked with Apple to somehow incorporate ReDrafter into its TensorRT-LLM framework. Moreover, the AI giant has also updated its framework with new optimized operators, enabling seamless adoption. The results, even to a layman like myself, sound promising -- LLMs using Nvidia GPUs with ReDrafter achieve a whopping 2.7x increase in token-generation speed, thereby reducing latency and power consumption.
Now, of course, whether this collaboration indicates that more is in store from the two of the world's largest companies, or whether it marks more of a one-time truce kind of thing is still up in the air. Either way, any variety of R&D alliance will likely spell positive news for consumers. After all, who doesn't want faster software?