My essay at TheSequence today covers the topic of environments in AI post-training AI environments are becoming a core ingredient of model capability: executable worlds that generate trajectories and feedback, not just exams. This essay traces their path from Atari, OpenAI Gym, and AlphaZero through MuJoCo to the RLVR-driven post-training behind Tulu 3 and DeepSeek-R1, then surveys the new generation of agent environments, including SWE-Gym, WebArena, OSWorld, TheAgentCompany, Tau2-bench, and AgentMercury. Check it out:

Great paper from Microsoft Research and colleagues. If you auto-generate MCP tools from agent trajectories, this one is worth your time. AutoTailor turns web trajectories into parameterized browser-automation APIs and then filters them. A quality filter drops APIs with the wrong granularity or duplicate function, and a usage filter keeps the ones likely to be needed. That takes 1,283 candidates down to 87. Online, it watches task outcomes, adds APIs for recurring gaps and prunes ones that go unused, ending at 33. On 106 WebArena Postmill tasks, the 33 APIs with a ReAct fallback reach 90.6% correctness against 87.5% for ReAct alone. Request-token cost drops 57.8% and latency 29.4%. Without the fallback, accuracy matches the unfiltered set while using 94.9% fewer request tokens. Paper:

Can web agents learn through imagination? DynaWeb: Model-Based Reinforcement Learning of Web Agents has been accepted to EMNLP 2026 Main Conference. Training web agents with online RL means costly, risky interaction with the live web. DynaWeb learns a Web World Model instead, letting agents dream web interactions and train on imagined rollouts. - Learns naturalistic web transitions from agent actions - Generates imagined trajectories in a synthetic web environment - Interleaves imagined rollouts with real expert trajectories - Consistent gains for open-source agents on WebArena and WebVoyager Imagination as a scalable path toward online agentic RL. This is part of our ongoing work at Gradient on distributed and efficient RL: training capable agents without the cost, latency, and risk of centralizing them. Paper: Code: