登録して招待リンクを共有すると、動画再生報酬と紹介報酬を獲得できます。

elvis
@omarsar0
Building and measuring frontier intelligence @dair_ai • Prev: Meta AI | Galactica LLM | Papers with Code | Elastic | PhD
参加 September 2015
1.6K フォロー中    330.2K ファン
Banger paper from Google on self-improving agent harnesses. (bookmark it) It addresses common issues in self-improving agent harnesses like local overfitting, catastrophic forgetting, and memory pollution. They propose GUARD, which keeps the base model frozen and lets the agent edit its own prompts, skills, tool rules, and memory. An edit is accepted only when its gains outweigh the behaviors it breaks. - On 43 hosted SDK tasks, GUARD reached 80.80% accuracy, 8.89 points over the starting harness (p = 0.008). - It beat Meta-Harness by 6.28 points and used 54.5% fewer tokens per task. - On procedural memory tasks, it led update correctness at 90.1% with zero memory pollution. Aggressive rewriters scored similarly and tripled pollution. How does it know what broke? Each task is split into self-checks before execution. Every round grades trajectories against them, pins rules the agent already handles, and ranks newly broken behaviors first. At test time, one frozen harness runs in a single pass. Chat with Paper:
もっと見る