A pretty important new primitive just landed in 🤗 Transformers: **decision models**. I think this could matter much more than adding support for any single model. Today, different decision models all approach "making a decision" differently: - @typesafeai’s Jev, the OG: typed decisions + calibrated probabilities, single forward - @AutoTrustAI’s JEV builds open System One models around a learned decision block, w/ multimodal. - @jaredpalmer’s KEV uses a pointer head over the options, sharing the computation of the state across questions. - @Cloudflare’s CLEF uses a joint schema head that routes evidence between the state, questions and options, scoring the whole schema together. - @perplexity_ai’s Decider uses a separate 255-way readout on top of Qwen, with non-causal attention in its full-attention layers. - @liquidai’s d1 is exploring small, multimodal decision models that can run all the way down to the edge. - And @XunzhuoLiu's Semantic Router team is pushing the serving side hard with System One + the D3 family: one typed interface for choice, score and yes/no decisions, with models from sub-1B to 27B. That experimentation is good, and needed! But without a common interface, every training framework, inference engine and evaluation tool has to understand every implementation separately. So Transformers is introducing a shared decision-model layer: `DecisionConfig` `pipeline("decision")` and, most importantly: `model.decide(...)` The idea is **not** to make every decision model work the same internally, but for them to speak the same language to the ecosystem around them. This means that, without running custom or remote code: - inference engines can serve them through one interface - training tools can build against common primitives. - the Hub can package them consistently. - evaluations can compare them without model-specific glue everywhere. This is the part of transformers I'm the most excited about: to grow alongside the ecosystem and the many ideas; and to make all of them more usable by everyone.

On December 10 in Tokyo, Japan, the biggest @PyTorch event so far is happening in the country, with PyTorch Day Tokyo. Several PyTorch core maintainers will be there in person to talk about everything PyTorch, Executorch, TorchTitan, etc. If you're in the region, don't miss it!

We’ve been hard at work trying to make models work with harnesses better. The best thing @yb2698 found: “harness adapters”. Small LoRAs trained with RL inside the actual agent harness they’ll be used with. Same base model. Different adapter for pi, opencode, hermes, etc. Why? Simply because the same model behaves very differently depending on the tools, prompts and control flow around it. Instead of asking one set of weights to be optimal everywhere, we can specialize a tiny part of the model for each environment. Early results are pretty encouraging: on held-out tasks, the adapters make fewer tool calls than the base model while keeping or improving accuracy. And because they’re LoRAs, the operational side is surprisingly cheap too. For one LFM2.5-24B-A2B experiment, we evaluated 9 checkpoints / 1,350 agent sessions on a single H200 in under an hour. The total: ~$5! Merging and serving every checkpoint separately would have cost around $90. There’s another important piece: RL agents will eventually try to hack the reward. So every tool call goes through `safeagents`, a deterministic rule-based guard that blocks things like editing tests, tampering with rewards or bypassing the sandbox before they execute. And the whole loop can start from real usage: your coding sessions -> verifiable environments -> RL -> a better adapter for your agent. The model doesn’t have to change completely. Sometimes it just needs to get better at the harness it actually lives in.

One of the biggest changes in the history of 🤗 Transformers audio is in the works, spearheaded by @eustachelb We’re rebuilding essentially the entire audio preprocessing stack. The goal is to move away from every model carrying its own `FeatureExtractor` implementation, and towards a common `AudioProcessor` architecture where models describe their frontend, while shared backends handle the actual computation. The stack currently covers ~30 audio frontends and 62 model mappings across Whisper, Wav2Vec2, Parakeet, SeamlessM4T, Qwen, Gemma, codecs, TTS models,and many others. A big part of the work is making backends explicit: `backend="torch"` → native PyTorch preprocessing (often super fast!) `backend="numpy"` → portable NumPy preprocessing That means audio preprocessing can increasingly live in the same framework as the model itself, while still keeping compatibility with the numerical behavior existing checkpoints expect. It also gives us a much better foundation for what comes next: new hardware backends, faster kernels, compilation, and optimizations that can benefit many audio models at once instead of being implemented model by model. ~30 frontends migrated. 62 model mappings. One common preprocessing architecture. The entrypoint, common across all architectures: `AutoAudioProcessor.from_pretrained(...)` Audio in Transformers is about to get a much stronger foundation! 🎧🎧🎧

Fun addition to transformers serve: you can now turn any of the 250 LLM & VLM model into a JEV-like decision model. /v1/systemone scores a set of decision with the model's logits. No need to generate text! Which one of them can get microducks across the finish line fast?

Transformers has supported loading GGUF files for a few years now, by unquantizing them. Thanks to @_marcsun, we're now using GGML kernels through the `kernels` library to run at the same performance as llama.cpp Huge kudos to the entire @ggml_org for making these kernels!

Tokenizers v1's first release candidate is out 🔥 absolutely huge release across the board: Up to 30x faster tokenization, rivaling the best in the ecosystem. Scaling across threads, much better latency, less memory consumed, and microscopic crate size.

This is going to be a little bit long, but I want to give hope to my fellow anxious ML engineers. We see a lot of propaganda on how this or that AI one shotted something, about how incredibly strong the models are getting and how we don't even need to review PRs and we can just ship to production. Although this can be true for some cases, its also far from being representative of all the challenges we have to face. I started using claude code 4 month ago, and quickly realized how it really does change the way we work. I can experiment 10x faster, fix small issues without coding and refactor code without sweating. BUT, these tasks were "just" tedious and not hard. The challenge in my day to day work is to take a research code and integrate it into transformers using our standards. Its challenging because code beauty is abstract and subjective just like a philosophy. By relying too much on claude, and on how seemingly good the code it produces look, I pushed the deepseekv4 integration without realizing that claude really did not understand the model. I gave it access to `transformers`, the original paper, the original code, the different blog posts and my past chats and skills created to add a model, a b200 node node and a LOT of tokens, but it did NOT nail it. It did not understand the eager attention path, it did not understand the basics of causal attention. It was even wrong implementing the manifold constrained hyper connections. It helped to reduce the burden of exploring implementation and debugging but it did not help reason around the model. I am not a doomer, I think our job as Software Engineers has never been this great, I am just saying that we still have a job, and we should still be a bit careful when it looks to good to be true 😉

I want to live in a world where I only need to think about the cost of my infra for my tooling to run; not my token limits. Excited to work with @onusoz on enabling open agent harnesses on local hardware: handle your own setup, ensure reliability, get more out of your agents.

I have a new job! Excited to announce that I will be working with Hugging Face to make local models work great in OpenClaw and other open agent harnesses! I will be building in public and documenting everything along the way, stay tuned!

Reading @deepseek_ai 's v4 paper.... absolute hats off. Every problem has a mathematical solution, nothing is left to chance. I have so much respect for them, putting out months or years of efforts entirely for free, in the open for anyone to benefit. Real goats 🫡

This marks the end of my first week at @huggingface! I'm joining as a founding engineer on HF's PyTorch team. My first project: safetensors on Mac is up to 3x faster🚀 Parallel reads straight into MPS unified memory, no CPU staging. MB Pro M5 Pro - Cold 16 GB: **2.97 → 8.23 GB/s** (2.8×) - Warm 3 GB: **10.3 → 26.6 GB/s** (2.6×)

DSv4 genuinely shines in 1M context window and peak efficiency to run many agents/users 😍 shortly coming to transformers and we're making sure you get all the peak efficiency 🔥 @art_zucker

🔈 Every model added to transformers has to be available on Apple Silicon 🍎 at once. We built a Skill and test harness for mlx-lm to get us closer 🔥 It's designed to help contributors AND support reviewers. Read on to see what we did and why it matters.

Great to see inference engines starting to leverage kernels on the Hub, in this case sglang. It's probably the easiest and fastest way to install flash attention and other specialized kernels right now.

We shipped a new repo type called "kernel" on the Hub. We want to democratize the whole ping-pong around packaging, distributing, and using custom kernels. This repo type is only available to a few community partners, @sgl_project being the first! Hop in 🧵for more details.