(1/5) New preprint: TreeOPSD, critic-free RL for long-horizon LLM agents 🌳 Long-horizon agents receive a single outcome reward after many turns, and each rollout batch feeds several sequential mini-batch updates while the policy drifts from the one that sampled it. TreeOPSD backs terminal rewards up a rollout tree and splits each update into two steps: construct a target for every turn, then match the current policy to it with forward KL. This leads to: • closed-form, bounded turn targets that commute with tree backup, so shared prefixes are trained once with no loss of credit • the exact tree policy gradient on the first mini-batch, and updates that stop at the target and reverse after overshoot • consistent gains over GRPO, GSPO and a matched tree baseline, largest on the longest-horizon tasks On Deep Research, TreeOPSD lifts Qwen3-30B-A3B-Thinking-2507 from 16.23 (GRPO) to 21.20 average accuracy. Title: Tree-Aware Posterior Distillation for Long-Horizon Agentic Reinforcement Learning Paper: Together with @TencentHunyuan #LLM# #RLVR# #AIAgents#

Recursive self-improvement needs agents that can build their own training environments. But researchers still part of the loop, making it hard to scale. We introduce AutoEnvScaling, which turns Data Flywheel into a terminal task: a proposer agent builds complete Harbor tasks and sandbox. It builds just like human researchers. This enables self-play RL for terminal agents: one model improves as both builder and solver, without human-in-the-loop.

People keep asking me: what's different about optimization in RL? Seemingly nothing — the pre-training stack just works (Adam, even SGD 👀 @saagnikkk). Bringing some answers from my last work (sorry for the delay — been cooking 🚀). We introduce ISO: Isospectral Optimization: an RLVR-native optimization stack. Built on one simple observation, spectral inheritance: RLVR can reuse the base model's spectrum and acquire new behavior purely through the singular frames. 🧩 Offline: ISO-Merger — consolidates RL experts into one model with no data, no rollouts, no OPD. Checkpoints only. ⚙️ Online: ISO-Optimizer — a drop-in wrapper on AdamW / Muon that matches AdamW's accuracy with ~2.7× fewer steps on Qwen3-8B-Base. 📄 🌐 🧵👇

Paper: Website with model responses and CoT: Authors: @BetleyJan @j_treutlein @jan_dubinski_ @HarryMayne5 @karolgalazka @nielsrolf1 @anna_sztyber myself.

🚀Excited to release PACE: A Proxy for Agentic Capability Evaluation! Evaluating LLM agents on benchmarks like SWE-Bench and GAIA is expensive, slow, and infrastructure-heavy, often costing $$$ and taking hours or days per model. ❓But do we always need to run full agentic evaluations? In PACE, we show that agentic benchmark performance can be accurately predicted from a small, carefully selected set of cheap non-agentic benchmark instances. PACE automatically selects proxy instances from existing benchmarks covering skills like instruction following, planning, tool use, reasoning, coding, retrieval, and multimodal understanding. Across 14 models, 4 agentic benchmarks, and 19 non-agentic benchmarks, PACE-BENCH achieves: ✅ 3.80% MAE for absolute score prediction ✅ 0.81 Spearman correlation for model ranking ✅ ~84% pairwise preference accuracy ✅ ~100× lower cost than target benchmark sampling Beyond prediction, PACE also reveals what capabilities different agentic benchmarks actually require, e.g., planning, verification, long-context aggregation, and instruction following. We hope PACE makes agentic evaluation cheaper, faster, and more accessible for model development, model selection, and routing :) 📃 Paper: 💻 Code: I'm incredibly grateful to have worked with @lintangsutawika, @Jiarui_Liu_, @lltjuatja, @JiayiiGeng, @lrzneedresearch, @daniel_js_lee, @Aditya_Soni_8, @Vincent92965015, @xiangyue96, and @gneubig .

Introducing EdgeBench, a benchmark designed to study how agents learn from environments over at least 12~72-hour runs. We find that performance follows a log-sigmoid function of environment interaction time with high precision. EdgeBench is built with three ingredients: - 🌍 Real & Diverse: 134 real-world tasks across 6 task categories, spanning scientific problems, professional knowledge work, software engineering, optimization, formal math, and games. - ⏳ Ultra-Long-Horizon: Each task supports 12–72 hours of agent work. Recorded human effort averages 57.2 hours. - 🔁 Informative Feedback: Agents receive real-world feedback for continuous improvement. After 38,000 hours of agent runs on EdgeBench, a scaling law for learning from environments emerges: - 📈 As agents interact with task environments over time, their aggregate performance is precisely fit by a log-sigmoid function. - 🧠 This phenomenon can be explained by an elegant theory of graph exploration. We are releasing an initial 51 of the 134 tasks, together with the full evaluation framework, to help advance long-horizon agent research. Check our blog & paper for more findings! Blog Paper GitHub Dataset Details below 👇🧵

We took a 30B model and split it in two to write tokens in parallel instead of one at a time. Introducing Nemotron-Labs-TwoTower: a diffusion language model from NVIDIA Research adapted from Nemotron-3-Nano-30B-A3B. Here’s how it works: one half holds the context, the other writes the tokens, with both reusing the pretrained model instead of training a new one from scratch. We found it kept 98.7% of the original model’s quality at 2.42× faster generation.