Excellent LFM3 GGUFs from AtomicChat, as usual. These results also show how tricky it is to quantize small models while preserving BF16 fidelity. I usually aim for 98 or 99 EAR, but that looks tough for this model. 96 EAR might be a more realistic target. I’ll double-check the accuracy on a few task benchmarks. (Also interesting: the difference in next-token probability distributions between FP16 and BF16.)

Qwen3.8-27B Unsloth GGUF is now the #1# most-liked GGUF of all time! The model hit 10M downloads and 3.7K likes in just 24 days on Hugging Face - all thanks to you. 🤗🦥 GGUF: Guide:

Qwen3.8-27B Unsloth GGUF is now the #2# trending model on Hugging Face with 2.7M downloads! 💗 Unsloth also reached #3# trending on GitHub! Thanks so much for the love! Model: GitHub:

Qwen3.8-27B GGUF has reached 1,000 likes in less than 24 hours! ❤️ It's now the #3# trending model on Hugging Face with 1M overall downloads. Run on 17GB RAM/VRAM setups via Unsloth! Model: GitHub:

I'm working on adding GGUF support to vLLM through the Humming kernels. In just 5 hours with GPT-6 Astra, I managed to get full support for quantization formats from IQ/Q2 to Q8, matching llama.cpp's speed in my benchmarks and outperforming it by ~30% at batch size 8. Still significantly slower than native Humming with GPTQ layers, though. A year ago, this would have taken me days, if not weeks, of work. What a time to be alive! Of course, it's still messy and limited in scope: Qwen3.5 architecture only Ampere GPUs tested (Grace Hopper and Blackwell support is implemented but untested) But it already achieves my only goal: significantly speeding up my GGUF evaluations. I'll definitely do this more often going forward. I might publish the code, but with no promises regarding maintenance.

Meet Ternary-Bonsai-2-9B-GGUF: a 9B image-text-to-text model squeezed into 2-bit ternary weights. Runs on llama.cpp with CUDA and Metal, so you can chat about images right on your laptop or phone. Small but mighty!

We released experimental MTP Qwen3.6 Unsloth GGUFs! Qwen3.6 27B MTP now runs at 140 tokens/s. Qwen3.6 35B-A3B MTP gets 220 tokens/s generation on a single GPU. Qwen3.6 27B and 35B-A3B have >1.4x speed-up over the original GGUFs without any change in accuracy. Guide + GGUFs + Benchmarks: In terms of average speedup, we see a 1.4x for dense models at draft tokens = 2 and for the MoE around 1.15 to 1.2x. We do not recommend more than 2 draft tokens because the acceptance rate drops precipitously from 83% to 50% with 4 draft tokens, and the forward passes for MTP become less beneficial. Use `--spec-type mtp --spec-draft-n-max 2` Thanks to Aman for