Most agent frameworks stop at “one task, one loop.” This article explains why that’s not enough for daily use and why runtime design becomes the real bottleneck. If you're building personal or developer agents that need to persist over time, nanobot is worth checking out:

GLM-5.2’s shift from GRPO-style group-wise optimisation to critic-based PPO for long-horizon agentic RL is worth paying attention to. Over the past year, the common narrative around LLM RL has been fairly simple: PPO is heavy, critics are expensive, and GRPO is lighter because it removes the critic. This view makes sense for short-horizon RLVR tasks, such as maths, coding problems, and verifiable QA. In those tasks, the structure is clean. The same prompt can produce multiple candidate answers, a verifier can judge which ones are better, and the model can learn through group-relative comparison. But agentic RL is different. A coding agent fixing a bug may need to read the issue, search the codebase, edit files, run tests, fail, locate the problem again, and try another solution. Final success only tells us that the whole trajectory worked. It does not clearly tell us which intermediate step was crucial, which step was a detour, or which action had no immediate reward but enabled later success. This is where GLM-5.2’s blog becomes interesting. It mentions that long trajectories are compacted and split into multiple sub-traces. For the same prompt, different rollouts can produce different numbers of trainable traces, and their lengths can be highly uneven. In this setting, group-wise optimisation becomes less natural, while critic-based PPO can estimate token-level advantages and handle credit assignment across long, irregular trajectories more effectively. So this is not simply “PPO is back”, nor does it mean “GRPO has failed”. A better way to put it is: GRPO is better suited for comparing multiple answers, while PPO is better suited for evaluating each step in a long chain of actions. When models move from answer generators to action takers, the key RL question also changes: from “which answer is better?” to “how should credit be assigned across a long sequence of actions?”