AI without the cloud: This language model now fits on your iPhone

Large language models typically require a data center or at least a powerful PC with plenty of memory. The U.S. startup PrismML, founded by Caltech researchers and funded by Google and Khosla Ventures, among others, released Bonsai 27B on July 14: an AI model with 27 billion parameters whose smallest version takes up just 3.9 GB. For the first time, a model of this class fits into the memory of an iPhone 17 Pro. Our guide AI without the cloud shows you how to set up a language model on your own computer.

Shrunk from 54 GB to under 4 GB

It’s based on Alibaba’s open-source Qwen3.6 27B, which takes up about 54 GB at full precision. PrismML stores each weight of the neural network using 1 to 2 bits instead of the usual 16. The result is two builds: a ternary version at a nominal 5.9 GB for laptops and a 1-bit version at 3.9 GB for phones. According to the manufacturer, the laptop variant retains 95 percent of the original performance across 15 benchmarks, while the phone variant keeps 90 percent. Math and programming remain virtually unaffected. The weights are freely available under the Apache 2.0 license and run via Apple’s MLX framework on iPhone, iPad, and Mac, as well as via CUDA on NVIDIA graphics cards.

What the iPhone can do and where the limits lie

According to the white paper, the 3.9 GB version generates around 11 tokens per second on an iPhone 17 Pro Max (available on Amazon). A full battery charge is good for about 67,000 tokens. After a little over five minutes of sustained use, the chip throttles. However, the benchmark figures come from the manufacturer itself. Its own table reveals the biggest shortcomings in image understanding and tool usage, precisely the areas where AI agents need to excel. In addition, the actual download sizes for the laptop version range from 7.2 to 8.5 GB depending on the runtime, significantly more than advertised. There is no ready-made app for everyday users yet. For now, the target group is developers and tinkerers. Anyone who just wants to try out the model can find a browser demo on Hugging Face.

Apple is already looking into the technology

PrismML CEO Babak Hassibi told CNBC that Apple and other companies are currently testing the models for speed, energy consumption, and performance. He described the talks as very early. For users, what matters most is the direction: the more AI runs directly on the device, the less personal data ends up in the cloud. We’ve already written in detail about what your AI secretly stores and how you can stop it.