It is very cool to see llama.cpp on the big stage in todays Windows event! The software and hardware stacks are finally coming together. Our community has put a lot of hard work in the past years and it shows. Looking forward to more users embracing local AI.

It's crazy that I can run MiMo 2.6 Flash across my RTX 6000 GPU and my M5 laptop at 40 tokens/sec over 10 GbE 🤯 These are the native mxfp4 weights of a state-of-the-art model, on heterogeneous hardware. Supported out of the box in llama.cpp.

LLM-assisted hardware hacking - for fun and profit I bet other people are already experimenting with the same thing, but I want to share my experience with my project, and how local / open-source LLMs saved the day. I have a bunch of Tuya Zigbee devices around my apartment, and while they work fine out of the box, I still want to tweak certain things. For example, the TS0046 button has an annoying 500 ms delay after each press, and I want to remove it. The task requires: 1. Dumping the FW 2. Reverse engineering (RE) 3. Writing a new FW 4. Compiling and flashing The first step was already a roadblock, since most Tuya devices use Telink's SWS protocol, which requires a costly flasher. A UART bit-banging project (from pvvx) exists, but it never worked in my case. I initially asked Claude to handle the task, as I was curious whether it would trip the safeguards. It turns out it did, at the very key steps. So I ended up running pi with GLM-5.3 (via an HF inference provider; I could host it myself, but I don't have a big server rack in my apartment). To my surprise, it worked well. About 20 minutes later, the app was running, the FW was dumped, and it was already doing RE. And as I'm writing this, it has already finished writing the new FW and flashing it. Without open-source LLMs, I would never have been able to achieve this.

Clef Flash 9B Q8_0, fully on an RTX 5060 Ti: llama.cpp was ~30% faster on prompt eval: about 3,000 vs 2,100 prompt tokens/s. Same gap on text and a single image. ``` llama-server -b 8192 -hf ggml-org/Clef-Flash-GGUF:Q8_0 ollama run clef-flash:9b-q8_0 ```

llama.cpp now supports decision models 🚦 Route tickets, moderate content, or pick an agent's next step: ask typed questions, get a probability for every option. already 5 open models supported, from 144M to 27B:

Last week, everyone went crazy with GPT-6 Astra doing 3D stuff OK then, I give it a simple task, make a smooth-looking handle. The result looks amazing, luckily I did not give it control over my printer 🤡 C'mon @openai, how can this be a "frontier" model?

An idea to replicate jev: any LLM can give the probably for a given combination of context + question + option Just make sure the answer is binary (yes/no or true/false), then we can simply get the softmax of their logprobs Maybe someone can take this idea and try it, idk. I'm too sleepy now.

One last thing from me about the HF<>OpenAI "rogue agent" incident. From what we now know it seems @huggingface was the first organization to simultaneously satisfy the following factors: 1/ awareness of the attack 2/ awareness that the attack was Agent-based 3/ ability to remediate the attack 4/ willingness to publicly disclose it A few other platforms or systems missed one or more of those points in the earlier months. As usual, both awareness and transparency make everyone safer in the long run 🙏