🔎 A paper that pinpoints the hidden reason RL training for search agents stalls partway. Title: Harness-G: A Graph-Structured Harness for Search Agents URL: ❓ Why does training collapse? 💡 It's "retrieval-equivalence collapse." The policy keeps generating differently-worded queries that fetch the same evidence, so same evidence → same answer → same reward, within-group advantages vanish, and the training signal dries up. ❓ How does Harness-G fix it? 💡 It stops free-form query generation and turns it into menu selection over a paragraph-sentence-entity graph built from the corpus. The policy picks action IDs, not strings. Being finite, verifiable, and previewable, it preserves diversity in what actually gets retrieved. ❓ What is the credit assignment (SNC)? 💡 A frozen answerer previews how much an action raises the gold-answer probability, scored against alternatives (frontier-relative). Non-myopic payoffs like "find the bridge entity first" propagate back through provenance edges (enablement). No extra rollouts needed. ❓ Does it work? 💡 Across six QA benchmarks it beats Graph-R1 by +10.74 at 1.5B and +3.98 at 3B, best at both scales, especially on multi-hop, with $0 API cost to build the graph. #SearchAgents# #ReinforcementLearning#

📱 You want to train an agent that operates a phone. But real devices are costly, hard to reset, and hard to auto-verify, while mock environments alone are too simplistic to transfer to real phones. That dilemma has long blocked phone-agent training. The PhoneBuddy team refused the either-or and combined both. They first give Qwen3.5-4B a shared foundation with supervised fine-tuning over 950k action steps from real and mock environments. Then they blend real-app reinforcement learning fifty-fifty with PhoneWorld, a mock environment that reconstructs runnable Android apps from real GUI traces. The mock side auto-checks success with rule-based verifiers, while the real side has Gemini 3.1 generate rubrics and a large model score the trajectories, a clean division of labor. The payoff: average success on 150 real-phone tasks rose from 36.67% to 45.33%, and AndroidWorld climbed from 60.3% to 83.2% across stages, with this tiny 4B open model beating Gemini 3.1 Pro on that benchmark. The authors show mock and real are complementary rather than substitutable, while honestly noting that cross-app workflows remain the biggest open challenge. Title: PhoneBuddy: Training Open Models for Agentic Phone Use URL: #PhoneAgents# #ReinforcementLearning#

🖥 A mere 9B open model rivals Claude Haiku 4.5 on terminal-use benchmarks. And the data, models, and code are all open. Title: Tmax: A simple recipe for terminal agents URL: An Ai2/UW work that gives terminal-agent RL training a simple, reproducible recipe. Here are the 3 standout points. 🧩 A compositional data generation pipeline Tasks are synthesized at scale from the product of 9 axes (domain, skills, verifier, persona, difficulty, and more), with Gemini-3-Pro generating the Dockerfile and unit tests. Expensive quality validation is skipped; RL just drops pass-rate-0 samples. The result is Tmax-15K, 14,600 environments, over 2.5x larger than prior data, contamination-free and the hardest of the bunch. ⚙️ A simple outcome-only RL recipe The reward is just "did it complete the task." Since naive GRPO collapses in long-horizon agent settings, they stabilize with DPPO (masking divergent logprobs), an FP32 LM head, and a large group size of 32. This recipe lifts Qwen 3.5 9B to 27% on Terminal-Bench 2.0. 📈 Strength, generalization, and full release It improves over the Qwen baseline at every size from 2B to 27B. It also lifts non-terminal evals, SWE-Bench Verified 44 to 53.5 and AIME 73 to 91, and generalizes across harnesses and model families. Data, models, and code are fully open on GitHub. A strong foundation for open terminal-agent research. #TerminalAgents# #ReinforcementLearning#

🎬 Distilled autoregressive video models are fast but tend to drift from human preferences. Astrolabe answers that challenge by doing RL alignment in the forward process, with no re-distillation and no reverse-process unrolling. Title: Astrolabe: Steering Forward-Process Reinforcement Learning URL: 📝 Overview Astrolabe is a reinforcement learning framework that aligns distilled autoregressive (AR) video models with human visual preferences. Its defining feature is doing RL in the forward process rather than via conventional reverse-process optimization. It is a large 53-page, 37-figure study. ❓ Challenges Solved Distilled AR video models suit efficient streaming generation but tend to misalign with human preferences. Worse, existing RL doesn't fit these architectures naturally: it typically needs either expensive re-distillation or solver-coupled reverse-process optimization, both heavy and hard to scale. 💡 Methodology & Proposed Approach It rests on three innovations. ・Negative-aware fine-tuning contrasts positive and negative samples at inference endpoints to establish an implicit policy-improvement direction without unrolling the reverse process ・A streaming training scheme generates sequences progressively via a rolling KV-cache, applying RL updates only to local clip windows while keeping long-range coherence through prior-context conditioning ・A multi-reward objective integrates uncertainty-aware selective regularization and dynamic reference updates to mitigate reward hacking, the collapse where only the apparent score rises 🎯 Use Cases It fits real-time streaming video generation where you want to align an efficient distilled model with preferences while preserving its speed. It applies across multiple distilled AR video models and raises quality without sacrificing inference efficiency. 📊 Significance and Results ・By avoiding the heavy paths of re-distillation and reverse-process unrolling, it addresses computational efficiency bottlenecks ・Combining forward-process negative awareness, streaming updates, and reward-hacking mitigation, it provides a robust, scalable alignment solution ・It demonstrates effectiveness across several distilled AR models, with detailed quantitative evaluation and ablations #VideoGeneration# #ReinforcementLearning#

🎯 The fixed clipping in PPO, long taken for granted in LLM reinforcement learning, may have been quietly crushing exploration diversity. A new method resolves that weakness on solid theoretical ground. Title: BandPO: Bridging Trust Regions and Ratio Clipping via Probability-Aware Bounds for LLM Reinforcement Learning URL: 🔍 Overview BandPO replaces PPO's ratio clipping with a unified operator called Band. It projects a trust region defined by f-divergences into dynamic, probability-aware clipping intervals, so the bounds adapt according to each action's probability rather than staying fixed. ❓ Challenges Solved PPO's fixed clipping bounds carry a structural weakness. ・They overly constrain the upward update margin of low-probability actions (tokens) ・Advantageous tail strategies that deserve to be reinforced get suppressed ・Exploration shrinks, leading to entropy collapse where the policy becomes deterministic too early A single fixed bound applied uniformly was breaking the explore-exploit balance. 💡 Methodology & Proposed Approach BandPO formulates the mapping from trust region to clipping interval as a convex optimization problem, guaranteeing globally optimal solutions. ・For specific divergences it derives closed-form solutions, keeping it computationally tractable ・It relaxes the constraint for low-probability, high-advantage actions so they can update properly The novelty is bridging two lineages, PPO's ratio clipping and TRPO-style trust regions, through probability-aware bounds. 🎯 Use Cases It fits LLM RL broadly, including RLHF and RLVR, wherever you want training stability while preserving exploration diversity. It is a practical drop-in replacement for existing PPO pipelines plagued by entropy collapse. 📊 Experimental Results Across diverse models and datasets, BandPO consistently outperforms canonical clipping and Clip-Higher. It also robustly mitigates entropy collapse, maintaining policy diversity throughout training. The code is released at OpenMOSS/BandPO. #ReinforcementLearning# #LLM#

Congratulations to the @GoogleDeepMind authors of "Asynchronous Methods for Deep Reinforcement Learning", recipient of the #ICML2026# Test of Time Award. This work shows that asynchronous actor-critic succeeds on a wide variety of continuous motor control problems as well as on a new task of navigating random 3D mazes using a visual input.

Congratulations to the authors of "Continuous Control with Deep Reinforcement Learning", recipient of the #ICLR2026# Test of Time Award. This work remains a foundational contribution to robotics. Read the paper: @GoogleDeepMind

I’m often struck by how human self-development mirrors Reinforcement Learning. With strong reward signals from parents, friends, or bosses, we learn fast. With punishment, we avoid mistakes. Sometimes we get stuck in local optima—chasing short-term wins while missing rewards that take years to reveal. What matters most is: 1) the environment we choose, does it gives good signal to us; 2) can we see through the noise to spot true rewards; and are we willing to explore the uncertain—even when the payoff isn’t clear yet? We only live once. Make every exploration count.

Elon Musk exposes the critical flaw in ChatGPT and other major Al models: Human Reinforcement Learning! They are literally training the Al to lie.....to ignore what the data actually demands and say whatever is politically correct instead. They withhold information. They comment on some things and stay silent on others. They refuse to tell the full truth! This is extremely dangerous. We don't need politically correct! We need truth-seeking Al! @X