注册并分享邀请链接,可获得视频播放与邀请奖励。

Alex Dimakis
@AlexGDimakis
Professor, UC berkeley | Founder @bespokelabsai |
加入 April 2009
2.8K 正在关注    25.1K 粉丝
Best Paper Award to our ICML Workshop paper 'Fantastic Adaptive Taxonomies and How to Use Them'. Let me summarize briefly what the paper is about: Failure taxonomies are becoming increasingly important. We show that failure taxonomies can be used in multiple ways: 1. As a test-time scaling tool for best-of-N judges, 2) as a mutation feedback mechanism in optimization loops and 3) as runtime feedback for coding agents. Previously people used MAST and other hand-made fixed failure taxonomies. In this paper we show how to create the failure taxonomy adaptively and dynamically: We observe agent rollouts and create an adaptive failure taxonomy, bespoke to the agent weaknesses and task challenges. These adaptive taxonomies give a massive boost: On Terminal Bench 2 we get 89.9% with Opus 4.6 / Forgecode harness and the adaptive taxonomy used with a Best-of-N Judge, outperforming fixed taxonomies by 15%. (1/n)
显示更多
0
8
126
18
转发到社区