Register and share your invite link to earn from video plays and referrals.

elvis
@omarsar0
Building and measuring frontier intelligence @dair_ai • Prev: Meta AI | Galactica LLM | Papers with Code | Elastic | PhD
Joined September 2015
1.6K Following    330.2K Followers
Banger paper from Google on self-improving agent harnesses. (bookmark it) It addresses common issues in self-improving agent harnesses like local overfitting, catastrophic forgetting, and memory pollution. They propose GUARD, which keeps the base model frozen and lets the agent edit its own prompts, skills, tool rules, and memory. An edit is accepted only when its gains outweigh the behaviors it breaks. - On 43 hosted SDK tasks, GUARD reached 80.80% accuracy, 8.89 points over the starting harness (p = 0.008). - It beat Meta-Harness by 6.28 points and used 54.5% fewer tokens per task. - On procedural memory tasks, it led update correctness at 90.1% with zero memory pollution. Aggressive rewriters scored similarly and tripled pollution. How does it know what broke? Each task is split into self-checks before execution. Every round grades trajectories against them, pins rules the agent already handles, and ranks newly broken behaviors first. At test time, one frozen harness runs in a single pass. Chat with Paper:
Show more