Zero Train–Inference Mismatch — now for linear attention, and under async RL 🎯 We got bitwise-exact trainer/generator parity for Gated DeltaNet (Qwen3.5-9B / 35B-A3B) on TorchTitan RL + vLLM, then asked the question nobody had actually tested in open source: does it help async RL? Two firsts (as far as we know): 1️⃣ First open-source stack to hit train/inference logprob diff = 0 on a linear-attention model 2️⃣ First to measure what zero mismatch is worth under async off-policy RL How — smaller than it sounds: • trainer and generator share one model definition (TorchTitan unified model). No more aligning trainer & generator op by op • recurrent kernel everywhere on the forward (prefill + decode + trainer), chunked only for backward • split-K off, batch-invariant GEMM, + a batch-invariant bmm for the MoE router • we never touch FLA kernel internals — the recurrent kernel is intrinsically batch-invariant: one sequence's own state, fixed order, no cross-sequence reductions • vLLM's linear-cache management + prefix caching keep working as-is 📊 Qwen3.5-9B + DAPO-Math, async at offpolicy = 4 / 12 / 32: • logprob_diff = exactly 0 at step 0 ✅ every nonzero after that is staleness, not precision • at offpolicy=32 the stock stack runs away past 0.065; ours holds ~0.035 • the unified model is free — the entire bill is the kernels • also verified on MoE 35B-A3B and a 64-turn / 64K-context terminal agent 🤔 Now the honest part: it depends on the config. On math at offpolicy=12, BI leads on train reward and peaks highest on held-out AIME. On the terminal agent it's slightly ahead. On search, and on other math configs, the curves are indistinguishable. What is consistent is the numerics — BI keeps the gap from exploding as you widen the window. Cost: 2–3× trainer throughput (≈5× on the agent). Our take: bitwise parity is a debugging tool, not a production default. Flip BI on for 20 on-policy steps — if the gap is 0 and your run is still broken, the infra is exonerated and the bug is in your data or your algorithm. That's worth 20 steps, not a whole run. Next: • make BI cheap (kill the 2–3×) • more workloads, more seeds • stay tuned for more complex agentic training recipes/tests 👀 Full writeup 🔗 We collaborated closely with the TorchTitan team, and we sincerely encourage everyone to check out TorchTitan RL. With its clean, efficient unified model architecture and native async RL design, we believe it is a strong foundation for your next RL engine. #LLM# #RL# #agentic# #pytorch# #torchtitan#

Try our new image browsing plugin for Claude Code, built by the legend @andylizf 🚀 It started as the ultra-efficient screenshot crawling pipeline we built for PixelRAG. Turns out screenshot-based browsing is often more token-efficient, more accurate, and preserves layouts/tables/images better than text-only browsing. Now you can plug it directly into Claude Code with one command. Code: Install: uv tool install pixelrag Give it a try and let us know what you build! 🔥

LEANN just won the Best Paper Award at #MLSys26# 🥹 still processing this. paper: repo: huge thanks to all the amazing collaborators, advisors, and open-source contributors who made this possible ❤️

Today I shared some personal practice use cases (for research and maintaining open-source) of Claude Code at @BerkeleySky, UC Berkeley. Here is the link to the presentation: Some personal hot takes/ discussion conclusions: 1. IDEs might not be the eventual best way to interact with code. CLI-based agents require less human supervision and scale better (compared to tools like Cursor). People may eventually only review the PR rather than the intermediate coding steps (even more aggressive!). 2. Whether a problem can be solved (the boundary!) by coding agents (today) largely depends on whether there is a clear signal for the task. For example, @karpathy’s autosearch is relatively easy because the signal is clean, a very good example. (Good news for researchers: coding agents still struggle with open-ended questions. So… our jobs are probably safe for a little while.) 3. Human supervision is the bottleneck. Designing agent products that minimize human effort while leveraging humans’ advantage in context switching is a very interesting direction.