註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

iDoMnCi
@DomAtSiteSage
trying to understand our place here one byte at a time data & ai @redhat building SiteSage w/ lovely @AmberAtSiteSage
加入 July 2023
176 正在關注    64 粉絲
Harness optimization is getting real receipts. AutoSaddler is offline harness learning from agent failure traces. Not another prompt tweak loop. It patches prompts, tools, and middleware as code. Then it keeps updates that survive a held out set. On the test sets (Pass@1): GAIA2: 53.0 → 62.0 (+9.0) SWE-Bench Pro: 37.3 → 46.9 (+9.6) Terminal-Bench 2.0: 40.0 → 50.0 (+10.0) That TB2 number also clears the expert tuned Terminus KIRA at 47.5. Kill generalization aware selection and GAIA2 falls to 50.6, under the default agent. Deep diagnosis and structured patches help. Dev set filtering is what stops the harness from overfitting the mini batch. On GAIA2, Figure 1b, about 147 leveraged traces to the best dev score vs about 1,400 for Meta-Harness.
顯示更多