注册并分享邀请链接,可获得视频播放与邀请奖励。

iDoMnCi
@DomAtSiteSage
trying to understand our place here one byte at a time data & ai @redhat building SiteSage w/ lovely @AmberAtSiteSage
加入 July 2023
176 正在关注    64 粉丝
Harness optimization is getting real receipts. AutoSaddler is offline harness learning from agent failure traces. Not another prompt tweak loop. It patches prompts, tools, and middleware as code. Then it keeps updates that survive a held out set. On the test sets (Pass@1): GAIA2: 53.0 → 62.0 (+9.0) SWE-Bench Pro: 37.3 → 46.9 (+9.6) Terminal-Bench 2.0: 40.0 → 50.0 (+10.0) That TB2 number also clears the expert tuned Terminus KIRA at 47.5. Kill generalization aware selection and GAIA2 falls to 50.6, under the default agent. Deep diagnosis and structured patches help. Dev set filtering is what stops the harness from overfitting the mini batch. On GAIA2, Figure 1b, about 147 leveraged traces to the best dev score vs about 1,400 for Meta-Harness.
显示更多