PrismML Bonsai 27B Explained: Ternary and 1-Bit Builds of Qwen3.6-27B That Put a 27B-Class Model on an iPhone (Announced 2026-07-14)

Bonsai 27B is an open-weight multimodal LLM released by PrismML on July 14, 2026: a post-hoc (no retraining) compression of Qwen3.6-27B into ternary or binary weights. The ternary build runs at an effective 1.71 bits per weight — 5.9GB, retaining 94.6% of FP16 performance — while the 1-bit build hits 1.125 bits — 3.9GB, retaining 89.5% — making it, per PrismML, the first major 27B-class model that practically runs on a phone. Weights are on Hugging Face under Apache 2.0, with native support for llama.cpp (CUDA/Metal) and MLX.

PrismML is a startup founded by Caltech researchers and backed by Khosla Ventures, Cerberus, Google, and Samsung. Its stated mission — *concentrating intelligence* — is compressing neural networks without sacrificing reasoning ability. The Bonsai family is its flagship line, and the name says it all: a big tree in a small pot.

On-device AI has mostly meant 3B–9B models that start small. Bonsai flips that: keep the 27B-class capability profile — multi-step reasoning, structured tool calls, vision, computer use — and shrink only the weight representation. The base, Qwen3.6-27B, is a hybrid-attention multimodal model (roughly 75% linear attention) with a 262K-token context. Crucially, unlike BitNet-style 1-bit models that must be pretrained from scratch, Bonsai is a post-hoc treatment of an already-trained model.

Each group of 128 weights shares an FP16 scale s_g, and every weight is expressed as w_i = s_g × t_i, where t is {−1, 0, +1} (ternary) or {−1, +1} (binary). Including scale overhead, that lands at 1.71 and 1.125 effective bits per weight — roughly 9.4× and 14.2× smaller than FP16. Notably, PrismML claims 'no higher-precision escape hatches': the ~24.8B language weights are quantized end-to-end, while the 0.46B vision tower ships in 4-bit and the 2.5B embeddings/LM head are handled separately.

The asymmetry matters in practice: math and coding — tasks with strong reasoning structure — survive remarkably well, while tool calling drops nearly 20% at 1-bit. Agentic use of the 1-bit build calls for external schema validation and retries.

Conventional ultra-low-bit quantization like IQ2_XXS (2.8 effective bits, 9.4GB) falls to a 72.7 average — and collapses selectively on hard reasoning (AIME26: 57.5, LiveCodeBench: 56.4). The abilities you most want are the first to break. Bonsai avoids that pattern at even fewer bits, keeping math at 93.4. And unlike BitNet-style models that require pretraining from scratch, the post-hoc approach can track future base-model updates. PrismML frames the efficiency as 'intelligence density of 0.53 per GB — 10× the FP16 baseline and ~2.7× the best low-bit alternative.'

Supported platforms span Apple (Mac/iPhone/iPad via MLX/Metal, iOS via the Locally AI app, plus a WebGPU demo) and NVIDIA GPUs via custom CUDA kernels. Official figures: RTX 5090 at 163 tok/s (1-bit) / 134 (ternary); M5 Max at 87/58. Independent llama.cpp tg128 measurements differ by setup: M5 Max (1-bit) 66.4, iPhone 17 Pro Max 11.0, H100 104.8 — rising to 143.8 with DSpark speculative decoding (1.37×). On iPhone the real story is memory: the 3.9GB model plus KV cache and activations fits within roughly 6GB of usable RAM.

- Offline assistants: 27B-class reasoning with no connectivity — job sites, flights, travel
- Privacy-first local processing: summarize and analyze confidential documents without data leaving the device
- Resident agents: the vision tower reads screenshots, documents, and camera input for on-device computer-use loops
- Hybrid deployments: everyday inference on-device, heavy jobs offloaded to the cloud API

For businesses handling data that cannot leave the premises, the local-AI ceiling just rose: where the practical choices were 9B–31B-class models on workstation GPUs, Bonsai delivers 27B-class inference from a 5.9GB file on a single laptop.

- Day-one, self-reported benchmarks: third-party verification is pending
- Task-dependent degradation: test your own use case — especially tool calling and vision
- 1-bit agentic use needs scaffolding: 66.0 on tool calling means schema validation and retries outside the model
- Sustained phone use unproven: thermals, battery, and memory contention need real-world testing
- 262K context is memory-bound: KV cache still costs memory the device may not have
- It's still Qwen underneath: Apache 2.0, but multilingual behavior and knowledge cutoff follow Qwen3.6-27B

Bonsai 27B reframes ultra-low-bit quantization from 'shrinking with regret' to 'choosing where the same intelligence lives.' If math stays at 93.4 in a 5.9GB file, a GPU server is no longer a prerequisite for many workloads. Agentic degradation and independent verification remain open questions — but as a post-hoc method that can track base-model progress, this release raises the waterline for on-device AI.