As generalist robot policies make their way into our homes, we can’t expect everyday users to rely on teleoperation devices or even write code to guide them. We need interfaces that feel natural and effortless. Introducing our latest work, RoboPrompt: simply point, circle a target, or sketch a trajectory to steer a robot—and turn those corrections into data for continual policy improvement! 🎉🎉🎉

Real2sim has made impressive progress with coding agents. But what about sim2real, especially the visual gap? Meet RoboRender: turning simulated trajectories into photorealistic videos for policy training, achieving 71% average success in zero-shot real-world robot deployment.

Really excited to finally share ThunderSyncRL ⚡, a project I co-led with @seilk_! We show that synchronous agentic RL can be up to 1.9× faster without sacrificing on-policy updates, simply by overlapping gradient computation with rollouts. Huge thanks to @Azaliamirh and the entire team!

Last week, I directed K3 to optimize mjwarp kernels for a workload and got 77% speedup. Today I asked GPT-6 Astra to optimize it further. It first tried an obviously wrong direction and failed. Then it tuned num_threads and got 0.38% speedup.😅 It's pretty good on other tasks.

Excited to introduce QF3: Fast Flow RL with Filtered Q-Gradients! Train humanoids from scratch, fine-tune VLAs, even fine-tune image models... all with one simple off-policy update ✨

Since my nanoGPT record was published, the question I’ve been asked the most is, “Does it scale?” The true answer is, while some aspects of the speedrun scale, the goal of this specific endeavor was not to scale it up. The point of the challenge is to utilize clever engineering and creative ML intuition to increase efficiency. But while nanoGPT is just about hill-climbing one benchmark, the rest of my work has been focused on scaling to the frontier. The main projects I’ve worked on are some architectural changes and ANVIL III. ANVIL III is a significantly stronger optimizer than ANVIL II, the optimizer I used to beat the nanoGPT world record. In fact, for the record, we removed the parts of ANVIL III that scale. ANVIL III is significantly stronger than Muon, while nearly matching its speed. For the same token budget (same compute), ANVIL III reaches final validation loss 20-28 millinats (.02-.028 nats) lower than Muon, holding at all scales from 124M to 1.2B, and different overtraining depths. As we scale up the model size and training depth, the 20-28 millinats becomes worth a larger fraction of compute. At deep overtraining (8x-Chinchilla) and 1.2B parameters, the fitted scaling curves imply that ANVIL III reaches the same loss numbers at 50% less compute. Of all the scales we tested, the gap is largest at 1.2B, staying between 26-28 millinats. The first remarkable aspect of ANVIL is hyperparameter consistency. The optimal hyperparameters remain highly consistent across different scales and training depths. In most of our tests, varying hyperparameters rarely changed ANVIL III’s loss, or did so within noise. In fact, as shown in the attached image (from the blog’s appendices), we tuned Muon significantly more than we tuned ANVIL III. We performed over 3 times the aggregate amount of runs of Muon than ANVIL III, and at each scale our loss margin uses only the optimal parameters for both. At 1.2B, we used 51 tuning runs of Muon (including muP proxies), compared to only 7 for ANVIL. We wanted to guarantee that our improvement was not a tuning artifact, so we meticulously tuned Muon’s parameters, and had to apply far less tuning to ANVIL III. Our hypothesis is that the high SNR of ANVIL III makes it very resistant to parameter changes. That SNR is the second remarkable aspect of ANVIL III. We generalize the Adam SNR (SNR = m̂ / (√v̂ + ε)) for each singular direction (uᵢ, vᵢ) in an update, and measure how strongly held-out gradients (G_b = gradient on held-out batches b) point along it: SNRᵢ = mean_b(uᵢᵀ G_b vᵢ) / rms_b(uᵢᵀ G_b vᵢ) In our tests, Muon takes a lot of steps with low SNR, but ANVIL III doesn’t, despite being a Muon variant. For example, at 10% of the training tokens into a 1.2B (approximately Chinchilla) model, 90% of singular directions in ANVIL III’s steps go downhill on held-out data, while Muon gets only 67% (random = 50%). Pretraining Blog: Another recent post from Hyperstition contextualizes the importance of this SNR gap. In The Signal and the Noise, we generalize Adam (in which the update is just a per-coordinate SNR) to every direction in parameter space. Almost every direction is nearly orthogonal to the gradient history, so averaging that over all directions just collapses back into plain momentum. But the single best direction, C⁻¹ḡ, reaches SNR=1 whenever recent gradients are linearly independent, which is almost always true in LLM pretraining. By solving that with a s-by-s Gram system with recency weighting, we get a basis-free optimizer that matches AdamW at 286M parameters. While this is far from optimal, it represents one avenue for possible gains in pretraining optimization that are yet to be realized. In another fun but mostly unrelated blog, Stuck in a Loop, we show that attention in pretraining creates feedback loops and self-reinforcement. A key’s gradient is proportional to the attention it already gets, so keys with early inflated attention receive stronger gradients, overinflating them. Breaking that loop spreads attention out, with the average entropy of attention heads goes from 3.75 to 4.69 nats (effectively 43 vs 109 tokens), but it makes the model worse. Similarly to The Signal and the Noise, this suggests that putting more weight in high-signal directions can beat spreading it out broadly. These discoveries are just a small part of the upcoming releases from @hyperstition_cc. Can’t wait until we release more. Big things are coming!

Can LLMs generate novel research ideas? We propose Priced Guidance to quantify this via compression! We lower bound P(LLM forecasts a future paper’s idea X). Instead of sampling ideas until X is matched, the LLM plays “20 Questions” to recover X, charging each hint by surprisal.

SwiLA's choices: ① Function class: a mixture of J linear regressions; each value coordinate picks one, with a learned prior. ② Optimizer: online SGD on the likelihood-based loss; each regressor takes a delta-rule step weighted by its responsibility. J = 1 recovers DeltaNet.

2/ As you scale the population, Dust beats backprop at 100k and 1M tokens and comes close at 10M and 20M. At 20M, the current trend even puts Dust's asymptote below backprop! EGGROLL is way worse at every budget. We describe in the post how Dust also integrates well with Adam.

RL with LLMs is very unstable when training and sampling policies differ. Standard fixes (matching numerics, importance sampling) work around the problem. We find the root cause of this instability from first principles and propose a way to directly cancel it. Score Centering is competitive and compatible with existing approaches — while simple to implement! 🧵 [1/6]