bonsai 2 27b just built this from one paragraph of prompt in one shot, all of it out of a 5.9gb file on an rtx 3060 12gb. i did not expect frontend taste at this size. small models usually get the logic right and the layout wrong, this one got the layout right, and it thought for a long time to do it, 41k tokens over 46 minutes, the context ran out to 77k and it never lost the thread. this is a ternary compression of qwen 3.8 27b, 26 tok/s fresh on a five year old gaming gpu, 13 tok/s at 77k deep, the whole 262k window resident. for what it is, on this card, this is insane, and i cannot wait to run it through real agentic coding on hermes agent, the tool loop, the builds that break and have to recover. @PrismML keep going, this is the one that runs on the card people actually own.

Meet Ternary-Bonsai-2-27B: a 27B parameter model squeezed into 2-bit ternary format. Runs on llama.cpp with CUDA and Metal support. 405K downloads and counting. On-device AI just got a whole lot lighter.

This is phenomenal for local AI!! Frontier-level intelligence right on your MacBook! Here is a quick read about how this entire thing works: Absolutely excited for the future of local AI!!

Bonsai 2 at <6GB size and ~Opus 4.6 performance. Live on Darkbloom. This breakthrough is from @PrismML founded by Caltech Prof. @BabakHassibi a leading information theorist. Optimizing intelligence per bit is an extremely valuable objective for Local AI. It is only possible for us to own our own intelligence if it will fit into the devices we own. Thats what the PrismML team has achieved here. It has only been 9 months from a frontier release (Opus 4.6) to getting it to fit in your phone! As we worry about frontier superintelligence concentrating power, we are delighted to see the counterweight emerging from open models that can fit into our phones. The darkbloom team @0xkydo and @gajesh were so excited by this breakthrough that they worked overnight to bring this to life on Darkbloom. Excited to offer the first hosted service for this model to everyone on Darkbloom, the compute grid powered by real people! Go try it on our chat or provision your Macbook to service this model for the world!

Darkbloom is the first model provider to support Ternary Bonsai 2 27B -- concentrated intelligence that fits on your phone. Try it now: First 250 users get 100 million free tokens; PrismML's new flagship model: - a ternary compression of Qwen3.8 27B at 2bits per parameter. - 8.5 GB total, 5x smaller than original - keeps 98.2% of the Qwen's FP16 benchmark performance. - 75% cheaper than Qwen 27B. Qwen3.8 27B already operates comparably with Opus 4.6 and 5.6 Luna on certain tasks. This one does it in the memory of a phone. 262K context, image input, Apache 2.0. From our first run on the network, on a single M5 Max with no caching: - 35 tok/s decode at 1K context, - 31 tok/s at 10K, - 19 tok/s at 50K. But we expect more performance gain coming in a few weeks! That's a full 27B reasoning model running comfortably on any Mac. 1,000+ Macs are serving on Darkbloom right now. Go try it out!! Thank you to @BabakHassibi @SahinLale @HessianFree @rsadri_ml @tushar_bans @evaninwords and the whole PrismML team. This is exactly the kind of model Darkbloom was built for. You can read the essay by Bonsai on Why Local AI Matters:

prism cooked so hard with Ternary-Bonsai-2-27B, utterly insane how good it is for the size have so far converted the mlx to vLLM, added MTP + some custom kernels 🍿 aiming to create the best local LLM for those with ~10-32GB VRAM NVIDIA cards image/weights coming soon... 👀

Speed on Bonsai 2 27b is interesting, especially in relation to the the MoE Qwen3.6-35b-a3b and its anticipated 3.8 version There is a size/speed tradeoff If you have 64 gb of memory for models, then it will probably be better to use the future Qwen 3.8 a3b because it will be faster, around 2x if it uses the same architecture But if you prioritize quality over speed TODAY, then Bonsai 2 27b is a better alternative because it is reportedly a high fidelity compression that would outperform a3b But if you have 32 to 16gb of memory, then rejoice! Your local model just got a huge bump in intelligence! Good day for local models 4 bit quants of a3b are around 20gb in weights, whereas Bonsai 2 27b around 6-7gb, depending on which bit packing you use Don't be fooled by 2 files in the repo. PTQ1_0 and PQ2_0 are the same model, just different ways to do bit packing If you use the smaller bit packing in memory, the PTQ1_0, you need to do more arithmetic operations. That is why prefill PTQ1_0 one is much smaller. But for decode, it's still the memory bytes that are the bottleneck, so the decode with PQ2_0 is lower Let me know if I'm missing something More details on my local profiling on the DGX Spark:

Bonsai 2 dropped hours ago. It's already doing real work in Osaurus. Read two CSVs. Wrote the Python. Drew the chart. Then found the one channel losing money and told me why. 27B. Ternary. Nothing left the Mac.

Local AI in action. ⚡ @PrismML’s new Ternary Bonsai 2 27B is 9x smaller than its full-precision counterpart while retaining 98.2% of aggregate benchmark performance. Watch it power an agentic coding workflow with @Cline on a single GeForce RTX 5090 GPU.

30 Web Designs made with Ternary-Bonsai-2-27B-PTQ1_0 It's a 5.5Gib model that can fit in 8gb GPUs, it's derived from Qwen3.8-27B, it's not just Q1, it uses their custom ternary kernels to achieve a higher accuracy that would not be possible with standard quantization and it needs a custom fork to work.

Today's my birthday, and I could not have asked for a better gift. Bonsai 2 fits on most phones shipping today (anything >8GB). And it outperforms Opus 4.6 and 5.6 Luna -- on tasks that would have sounded absurd to attempt anywhere two years ago. I try to be disciplined about timelines. My most optimistic estimate for something like this was early-2027. It's September 2026. Huge thank you to the @prismml team -- @BabakHassibi @SahinLale @HessianFree @rsadri_ml @tushar_bans @evaninwords -- for putting this into the world. We're bringing Bonsai 2 to @DarkbloomAI tonight. The 1,000+ providers can test it right away. And whenever those machines aren't in use, they'll serve Bonsai 2 to anyone who wants to try it. So few people set up a local model themselves, and a model this good shouldn't be gated by that. More tomorrow.

They shrunk a strong 27B model to 5.9 GB. 9× smaller keeping ~98% of original performance. “Ternary” compression stores model weights using only three values. That means powerful AI can run on much cheaper, smaller hardware.

Bonsai 27B is breaking local AI again! I'm soo excited to try this on my M1 Air 16GB. Getting 98.2% of qwen3.8-27b fp16 on ternary build is just insane no matter how you think about it. You'll need to give it a try.