註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

Alex Dimakis
@AlexGDimakis
Professor, UC berkeley | Founder @bespokelabsai |
加入 April 2009
2.8K 正在關注    25.1K 粉絲
Best Paper Award to our ICML Workshop paper 'Fantastic Adaptive Taxonomies and How to Use Them'. Let me summarize briefly what the paper is about: Failure taxonomies are becoming increasingly important. We show that failure taxonomies can be used in multiple ways: 1. As a test-time scaling tool for best-of-N judges, 2) as a mutation feedback mechanism in optimization loops and 3) as runtime feedback for coding agents. Previously people used MAST and other hand-made fixed failure taxonomies. In this paper we show how to create the failure taxonomy adaptively and dynamically: We observe agent rollouts and create an adaptive failure taxonomy, bespoke to the agent weaknesses and task challenges. These adaptive taxonomies give a massive boost: On Terminal Bench 2 we get 89.9% with Opus 4.6 / Forgecode harness and the adaptive taxonomy used with a Best-of-N Judge, outperforming fixed taxonomies by 15%. (1/n)
顯示更多
0
8
126
18
轉發到社區