Most “Self-Evolving AI” Is Not Recursive Self-Improvement Yet AI may generate its own data, rewards, skills, and code. But if humans still define what counts as better and approve deployment, the loop is not closed. Zhihu contributor 一口鸟 (@alsaceym) argues that RSI’s hardest bottleneck is reliable verification. 1️⃣ Moving humans out of the loop RSI progresses through three stages: 🔹 Human in the loop: AI proposes changes; people approve them. 🔹 Human on the loop: data, rewards, and verifiers are automated; people supervise deployment. 🔹 Closed loop: the system generates, verifies, and applies improvements itself. Most “self-evolving” systems remain in the second stage. 2️⃣ What actually improves? Self-refine changes the current answer. Test-time training writes experience into weights. Agent evolution modifies prompts, tools, memory, skills, workflows, or Agent code across tasks. output → weights → the Agent itself Training-time RSI follows another ladder: 🔹 Zero-label: AI generates supervision. 🔹 Zero-data: AI also generates problems and curricula. 🔹 Auto research: AI chooses hypotheses, training recipes, and experiments. The system gradually takes over how to learn, what to learn, and finally how to improve learning itself. 3️⃣ Self-improvement can amplify mistakes A generator and verifier may share the same biases. Wrong outputs can produce biased evaluations, biased learning signals, and stronger errors. Even correct rewards do not guarantee stability: training can improve and later collapse. Self-play may also lose diversity or favor problems that are easy to reward rather than genuinely useful. Automation makes grounding more important, not less. 4️⃣ Verification is the real bottleneck Math and code are RSI-friendly because proofs, unit tests, and execution feedback provide clear signals. Open-ended Agent work is harder. A verifier must judge not only correctness, but novelty, usefulness, importance, and research taste. The next step may be evolving the verifier itself. But if the policy and evaluator change together, what keeps both aligned with reality? ✅ The boundary of true RSI The loop is expanding: answer → experience → learning signal → problem and curriculum → verifier True RSI requires improvement across every layer without losing contact with real objectives. Until evolving verifiers remain reliable without constant human grounding, recursive self-improvement is still an aspiration rather than an achieved capability. 🔗 Full analysis: #RecursiveSelfImprovement# #RSI# #SelfEvolvingAI# #AIAgents# #ReinforcementLearning# #AISafety#

Skild AI says S1 plays soccer against humans and robots after 140+ years of simulated self-play. “This method scales, and we will scale it,” says CEO Deepak Pathak. How it works—and where human guidance comes in:

AlphaGo, but for humanoid soccer. Skild AI brings self-play to humanoids. Their flagship robotics foundation model, S1, can now be post-trained with no human demonstrations by competing against itself in simulation. The testbed is soccer, which demands balance, agility, ball control, reactivity, and strategy. Given a single objective (score goals), S1 played 140+ years in NVIDIA Isaac Sim and went from falling over to dribbling past defenders, shielding, tackling, shooting, and getting back up. None of these behaviors were hand-rewarded; they emerged because they helped it score. Its opponents (past versions of itself) improved alongside it, creating an automatic curriculum. With 4 agents, passing and coordination started to emerge. Skild says the approach extends to social navigation, collaborative manipulation, and any multi-agent task with a simulator and an objective.

🧠 From "reading context" to "skillfully learning from it." This method has an LLM acquire context-specific skills through self-play alone, with no human annotation and no external feedback. Title: From Context to Skills: Can Language Models Learn from Context Skillfully? URL: 📝 Overview LLMs are strong on knowledge seen in pretraining but weak on novel, specialized contexts. This paper proposes Ctx2Skill, which autonomously discovers and refines context-specific skills without human annotation or external feedback. ❓ Challenges Solved Annotating long, technically dense documents is prohibitively costly. And unlike coding, context learning has no execution feedback to verify, making automated skill construction hard. 💡 Methodology & Proposed Approach ・A multi-agent self-play of five frozen-LM roles iterates N=5 times over M=5 tasks ・A Challenger creates tasks and rubrics probing weaknesses, a Reasoner solves them, and a Judge gives pass/fail verdicts ・Proposer and Generator pairs diagnose failures and synthesize skill updates ・A Cross-Time Replay mechanism maximizes the product of hard- and easy-probe performance to pick the most generalizable skill set across iterations 🎯 Use Cases It fits feeding a model long specialized documents and having it acquire the needed skills on the spot, directly useful where you need rapid adaptation to domain-specific knowledge. 📊 Experimental Results ・On CL-Bench (500 contexts, 1,899 tasks), GPT-4.1's solving rate rose from 11.1% to 16.5% ・GPT-5.1 went from 21.1% to 25.8%, and GPT-5.2 from 18.2% to 21.4% ・Skills transfer from stronger to weaker models: GPT-5.1's skills on GPT-4.1 give 16.1% ・The augmented GPT-4.1 (16.5%) beats an unaugmented Gemini 3 Pro (15.8%) #LLM# #InContextLearning#

Must-read papers of the week ▪️ Agent Lightning v1.0: Towards Harnessed Agentic RL ▪️ LEGO-RL: Harness-Native Reinforcement Learning for Coding Agents ▪️ EnvHarness: Awakening Static Worlds for Agent Learning ▪️ SPADE: Self-Play in Adaptive Synthetic Executable Environments ▪️ SkillGate: Training In-Policy Skill Selection in Long-Horizon Agents ▪️ Looped Language Models Improve Compositional Tool Calling ▪️ Graph Engineering in the Era of LLM Agents ▪️ Chain-of-Experience for Continual LLM Improvement ▪️ Cross-Model Memory Transfer via Target-Side Reader Adaptation ▪️ CLEAR: Continuous Latent Adapter Routing for Utility-Preserving LLM Safety Alignment ▪️ EXIMO: VLM Guided Exploration of VLA Policies ▪️ FreeToken: Efficient Edge-Native MoE Serving with Bandwidth-Adaptive Execution ▪️ Llama-Mobile: Efficient 2.7-Bit Quantization of VLMs ▪️ Improving the Matrix Multiplication Exponent with Modern Optimization and AlphaEvolve Here is the full list of the most interesting papers + links:

Rumors I’ve been hearing. I’ll start with OpenAI, then Anthropic. OpenAI: The internal model’s reasoning jump comes from more aggressive RL, including training in games and simulated environments. Strategies learned there transfer to research mathematics and other fields. Navier–Stokes and ETH show the scale of the improvement. The clue is in the Synthetic RL hiring: self-play, simulators, and synthetic feedback. Anthropic: Same direction with its internal Claude model. It’s trained on long, difficult tasks in demanding environments, and the reasoning skills transfer to other fields. Its ETH result shows how far the improvement has gone. The earlier Universes listing and the RL hiring around mathematical reasoning align with that. Musk’s Digital Optimus hiring explicitly calls for measuring transfer from games into general computer use. That’s why his game-playing AI push matters: games develop abilities that work far beyond the games themselves. The interesting part is where those abilities show up. Each lab’s internal model applies reasoning developed through interactive training to mathematical problems that had resisted decades of work.

Delighted to see our real-world clinical study of a conversational diagnostic AI system published in @TheLancet, alongside coverage in @nytimes last week. Looking back, AI in medicine might feel inevitable but it actually takes deliberate, uncertain leaps to get there. I have been fortunate to be part of two such moments. The first was early 2022 (pre ChatGPT) when @alan_karthi and I got access to our first @Google LaMDA and PaLM models. That convinced us to go all in on LLMs in medicine, sparking the Google Brain moonshot leading to Med-PaLM. The second moment leading directly to this Lancet study, was late summer 2023. @taotu831 showed me the early AMIE models he was training with our self-play loop, targeting our upcoming OSCE study with patient actors, later published in @Nature 2025. At that exact time, I was navigating a personal health condition. Testing myself, the consultation the AI performed was so natural and clinically sound that I was convinced it was ready for the real world. So I reached out to Alan and together with the team we asked how we could test this in the real world quickly, safely and responsibly. We needed a prospective clinical trial with a safety endpoint and all roads led to @AdamRodmanMD at @bethisraellahey @harvardmed . Adam was incredibly gracious with his time kicking off our first discussions in the fall of 2023. But clinical trials take a long time; years of painstaking work. There are so many critical people and moments that make them happen, far beyond the glitzy PR materials and the main authors. - For example, a discussion with @greg_corrado sparked the design of the oversight setup underpinning the safety layer in the study. - Kavita Kulkarni (now at BioHub) spent months negotiating the initial contracts with BIDMC; it was uncharted territory for both organizations. - @_khaledsaab (now rocking at OpenAI!) working tirelessly to improve the AI, together with @taotu831, @apalepu13, @RyutaroTanno, @ckbjimmy, @valentinlievin, @elahevedadi and many more folks. - @Mysiak, Alessio Orlandi, Petar Sirkovic and team sharing the seamless serving infrastructure they built for Co-Scientist, which we could re-use for much of this study. - @SSaraMahdavi making sure the study team was well organized :) This is just to name a few. Prospective trials are a massive team sport and I am proud of what our teams achieved. While the AI's clinical reasoning was strong, seeing the patient trust, safety, and satisfaction data, with ZERO safety stops from our supervising physicians, is what truly stands out. At the same time, this is still a stepping stone. We are moving from AI as a tool to an active care team member - a guardian angel amplifying clinicians and staying with patients throughout their health journey. More to come soon, and as always, a pleasure to work with so many great folks across @GoogleDeepMind , @GoogleResearch and @bethisraellahey . Paper - NYT -

TL;DR: AgentGarten is a virtual-world framework that keeps simulator-grade faithfulness while matching video-model realism, and it taught agents hide-and-seek tool-use strategies in a handful of rounds that took conventional RL millions of episodes. Title: AgentGarten: Code Worlds for Evolving Agents URL: Points 🎮 Couples a scene program (state, rules) with a neural renderer (visuals) through structured conditions like depth and normal maps, decoupling physics from appearance 🔬 A new technique, "Adversarial Forcing," makes history prefilling differentiable via bitwise-exact replay with block-by-block SDPA attention, where standard approaches weren't differentiable at all ⚡ Achieves real-time rendering at 480×832 and 36.5fps on a single H100 🤖 A "playbook" mechanism lets agents carry round-by-round reflections forward, accumulating knowledge with no gradient updates 🏃 In hide-and-seek, shelter-building emerged by round 4 (vs. ~25M episodes for self-play RL) and ramp use by round 10 (vs. ~100M episodes) 📈 Other environments — herding, a quarry loader — also show clear gains within just 4 rounds What's striking is how much learning efficiency you gain just by architecturally separating state dynamics from visual generation. #WorldModels# #AgentLearning#

Researchers led by MIT built Ataraxos, an AI that beat the world's strongest Stratego player, a game where most of the opponent's information is hidden. → It beat the top human player 15-1-4, and went 39-2 against top players at the world championship → It outperformed DeepMind's earlier Stratego AI, DeepNash → It needed under 1/100th of DeepNash's training examples and under 1/30th of the self-play games → It combines self-play learning with planning, using a generative model to guess hidden pieces Source:

Skild AI says it trained a robot to play football by letting it play against itself for 140 years inside a simulation. ⚽️ The clip is labeled autonomous, 1x. No pilot, no speed-up. Two things are doing the work here. First, time compression which is thousands of robot instances running in parallel physics sims, so what would be a century & a half of practice takes days of wall-clock training. Skild builds these on NVIDIA's Isaac Lab & Omniverse & describes gaining millennia of experience in days. Second is self-play. Nobody scripted a dribble or a shot. The system plays opponents that are copies of itself, & because the opponent improves whenever it does, the difficulty curve is generated automatically - never so easy the policy gets lazy, never so hard it learns nothing. It's the same mechanism that produced superhuman Go. The hard part is the last step. A policy trained in simulated physics usually falls apart in a real body, where friction, motor lag & a slightly wrong mass estimate break everything. Getting it to transfer is the actual result & Skild has shown related generalization before with its models recovering from a stuck wheel in seconds, or walking with a broken leg after a few attempts. DeepMind did knee-height soccer robots this way in 2024. This is the full-size version.