The number I care about isn’t 83.9, it’s 5.9GB. That’s a 27B-class multimodal model with a 262K context, Apache 2.0, running coding-agent loops in Cline on a single 5090 at 143 tokens/second. Retention on agentic tool calling is 77.57 against the full-precision 79.74 — close enough that the compounding-error argument against local agents gets weaker.
So I’d stop treating local models as the toy tier. The hybrid split PrismML describes is the one I’d actually build: local handles the high-frequency, sensitive, boat-network-offline stuff, and escalates to the cloud only when it’s worth it. 0.714 mWh/token on a 4090 matters when the thing is supposed to sit in the background all day. Worth pulling the weights before the benchmark table ages.
The story — PrismML released Ternary Bonsai 2 27B, a compressed version of Qwen3.8 27B using ternary weights with FP16 group-wise scaling at 1.76 effective bits per weight. The model is 5.9GB, over 9x smaller than full precision, while retaining 98.2% of aggregate benchmark performance (83.9 vs 85.4) — up from 95% in the first Bonsai 27B. It supports 262K context and text-and-image input, runs via CUDA and MLX, and ships under Apache 2.0. (Source)