One of the coolest recent features in Pi is Virtual Models I am using it to route to Qwen 3.8 27B (running locally in @Llama_app) or to Kimi K3 or GLM 5.3 (hosted remotely), depending on my query complexity and my machine's current load 🔥 No more manual model switching!! Kudos @badlogicgames @mitsuhiko + ggml team!!

We at @huggingface have been working on introducing a AutoModelForDecision to the transformers library. Specifically, we want to define an *interface*, not an *implementation*, so users can integrate the input/output structure and authors keep freedom.

Just minutes ago, I had the pleasure to announce Llama for Windows during the .NET conf keynote, thanks @Microsoft for having me! Llama for Windows brings Local AI to the people. Your AI. On your PC. Easy and free. Fully private. No terminal, no set up. App is open source, and powered by llama.cpp Download it today at - and soon from the Windows Store! Congrats @MorganFunto @ggerganov and team for the launch.

A pretty important new primitive just landed in 🤗 Transformers: **decision models**. I think this could matter much more than adding support for any single model. Today, different decision models all approach "making a decision" differently: - @typesafeai’s Jev, the OG: typed decisions + calibrated probabilities, single forward - @AutoTrustAI’s JEV builds open System One models around a learned decision block, w/ multimodal. - @jaredpalmer’s KEV uses a pointer head over the options, sharing the computation of the state across questions. - @Cloudflare’s CLEF uses a joint schema head that routes evidence between the state, questions and options, scoring the whole schema together. - @perplexity_ai’s Decider uses a separate 255-way readout on top of Qwen, with non-causal attention in its full-attention layers. - @liquidai’s d1 is exploring small, multimodal decision models that can run all the way down to the edge. - And @XunzhuoLiu's Semantic Router team is pushing the serving side hard with System One + the D3 family: one typed interface for choice, score and yes/no decisions, with models from sub-1B to 27B. That experimentation is good, and needed! But without a common interface, every training framework, inference engine and evaluation tool has to understand every implementation separately. So Transformers is introducing a shared decision-model layer: `DecisionConfig` `pipeline("decision")` and, most importantly: `model.decide(...)` The idea is **not** to make every decision model work the same internally, but for them to speak the same language to the ecosystem around them. This means that, without running custom or remote code: - inference engines can serve them through one interface - training tools can build against common primitives. - the Hub can package them consistently. - evaluations can compare them without model-specific glue everywhere. This is the part of transformers I'm the most excited about: to grow alongside the ecosystem and the many ideas; and to make all of them more usable by everyone.

On December 10 in Tokyo, Japan, the biggest @PyTorch event so far is happening in the country, with PyTorch Day Tokyo. Several PyTorch core maintainers will be there in person to talk about everything PyTorch, Executorch, TorchTitan, etc. If you're in the region, don't miss it!

The future of software In the limit, of course, software won't mean anything anymore. Everything digital will be instantly created at will. But in the interim period we'll wander through an increasingly odd in-between age for software. This transition periodwill be defined by three new precepts. Everflux software - instead of stepping from release to release, months apart, software now evolves continuously. Software creation has broken the sound barrier of our own keystrokes, and every model release pushes it faster. There is no reason to wait weeks or months for the next release when a single company merges over 30,000 code changes a day. Cyber capabilities are also progressing at an increasing pace. Bug and vulnerability fixes need to ship quickly and continuously. And every piece of software sits in a lattice of dependencies, each one now shipping at a blistering pace. Software will have to move accordingly fast just to keep from breaking as everything around it shifts. Translucent sources – A new territory has opened between open source and closed source. Today Claude will happily pick apart any Electron app, and any binary shipped with debug symbols has become a reverse-engineering playground for LLMs. In other words, much so-called closed-source software is now open source in all but name, and keeping code closed makes less and less sense. Even the notion of IP and copyright is breaking in software: Apps don't even need to be accessed to be reproduced. I'm writing this text on a personal text editor I prompted Claude to vibe-code from a few videos of people demoing editor features they liked. A mere description of the interface is increasingly enough for an LLM to rebuild the internals, whether or not they resemble the original. Software IP seems to be dissolving like cotton candy in a raccoon's paws. Bespoke features - Now that all software is open source, nothing prevents us from modifying everything. And personalization is just one prompt and a couple million tokens away, so there is absolutely no reason anymore to wait for someone else to add this feature you want. Everyone becomes their own product manager. Want a search box, an API bridge, a custom button? It's a couple of minutes away, mostly executed in the background of your daily life. Soon we'll want to personalize everything. A major challenge appears when bespoke features crash into our first tenet: everflux software. How do you customize software without cutting yourself off from upstream? I could fork it for good to add my search box, but I'd lose all future benefit from the software creator, be it new security updates or other updates to the rest of the software. How do you personalize software that was never meant to be personalized, without shouldering all of its future maintenance? Merge conflicts won't be the exception anymore; they'll be a constant of everflux software. Agents will rebase our personal features onto each new release overnight, and software will look less like a product and more like a living species, evolving through constant mutation. Image: M.C. Escher, Metamorphosis II, November 1939 - March 1940

Did you know Xcode includes skills covering SwiftUI, App Intents and Siri, security and memory safety, testing and verification, and performance profiling? Use them in Xcode or export them to use with your favorite coding agent with: $ xcrun agent skills export

We’ve been hard at work trying to make models work with harnesses better. The best thing @yb2698 found: “harness adapters”. Small LoRAs trained with RL inside the actual agent harness they’ll be used with. Same base model. Different adapter for pi, opencode, hermes, etc. Why? Simply because the same model behaves very differently depending on the tools, prompts and control flow around it. Instead of asking one set of weights to be optimal everywhere, we can specialize a tiny part of the model for each environment. Early results are pretty encouraging: on held-out tasks, the adapters make fewer tool calls than the base model while keeping or improving accuracy. And because they’re LoRAs, the operational side is surprisingly cheap too. For one LFM2.5-24B-A2B experiment, we evaluated 9 checkpoints / 1,350 agent sessions on a single H200 in under an hour. The total: ~$5! Merging and serving every checkpoint separately would have cost around $90. There’s another important piece: RL agents will eventually try to hack the reward. So every tool call goes through `safeagents`, a deterministic rule-based guard that blocks things like editing tests, tampering with rewards or bypassing the sandbox before they execute. And the whole loop can start from real usage: your coding sessions -> verifiable environments -> RL -> a better adapter for your agent. The model doesn’t have to change completely. Sometimes it just needs to get better at the harness it actually lives in.

Join us in this livestream next Tuesday, learn how to shrink models to fit in your laptop and make them go brrr 💨 We will have @mervenoyann @maximelabonne @TadimetiAditya @bartowski1182 and @mgoin_ giving a primer and know-hows on quantization with llama.cpp and vLLM ⚡️

we got the inference goats 🐐 I will shortly teach you how quantization works, modern ideas (double quantization, calibration) and llama.cpp quants (k-quants, i-quants) super excited and starstruck 🥵

Apple's Core AI models repo is now accepting third-party contributions 🥳 If you have developed some cool ports, this is the best place to share your recipes with the community. Looking at you @JackdeS11 @mattt @kevinqz 😂

llama.cpp v0.9.0 is out 💃🕺 > Native FP8 (OCP E4M3) > Faster multi-seq serving: flash_attn_ext_rows > Multi-node tensor parallel via NCCL (ggml 0.29.0) > New models: Clover, Limite, Metis-1.6, Mistral Large 4 > Speech: Chatterbox TTS, Nemotron 3.5 Transcribe > Faster Gemma 4 E2B/E4B prefill > much more...

oMLX 0.7.4.dev1 is out! With the new Prefill Accelerator, connect an M5 Max to your big Mac and prefill up to 2.9x faster! Prefill Accelerator (64K prompt, M3 Ultra 512GB + M5 Max 128GB) - DeepSeek V4.1 Flash: 1,060 -> 3,123 tok/s (+195%) - Qwen3.8 Flash-Next: 1,309 -> 3,666 tok/s (+180%) - GLM-5.3 Flash: 540 -> 1,388 tok/s (+157%) - MiniMax M3.1: 297 -> 569 tok/s (+92%) - An M5 Max 64GB still adds 30-118%, and decode speed stays the same. Faster DeepSeek V4.1 with A8 and buffer pool improvements - Prefill on M5 Ultra is 43-48% faster, 32K goes 2,631 -> 3,779 tok/s (+44%). - oQ4e builds now run INT8 Activation Prefill on M5. Faster Qwen3.8 Flash-Next - FP16 builds get the fused kernels. Decode on M3 Ultra goes 38.9 -> 63.0 tok/s (+62%), and +23% on M2 Max. - Decode after long prompts is 3-5% faster on M5 Max. Better for agents - Parallel tool results from Claude Code reach the model in call order, so Qwen no longer pairs a result with the wrong call. - Agent sessions on a cluster reuse the prompt cache across tool turns, 743 -> 9,266 cached tokens by the fifth turn. - Switch oMLX models inside Codex and Copilot with /model. Also in 0.7.4.dev1 - Ming-3.0 TTS with a 2x faster decode loop. - Ask AI can use an oMLX server on another Mac. - Check for model updates in the macOS app. - German across the web dashboard and the macOS app. - Batched decode with 6 requests on 8-bit models is 16% faster. A big thank you to everyone who sent PRs, reported bugs and tested builds for this release, and a warm welcome to the five new contributors!

With @jundotkim, we'd like to measure how much a DGX Spark would accelerate prefill of DeepSeek V4.1 Flash running on M3 Ultra. Something like this below, with the Spark being in charge of prefill. Has anybody done that? cc @Coolmark482, perhaps you know?