Register and share your invite link to earn from video plays and referrals.

Alex Dimakis
@AlexGDimakis
Professor, UC berkeley | Founder @bespokelabsai |
Joined April 2009
2.8K Following    25.1K Followers
Best Paper Award to our ICML Workshop paper 'Fantastic Adaptive Taxonomies and How to Use Them'. Let me summarize briefly what the paper is about: Failure taxonomies are becoming increasingly important. We show that failure taxonomies can be used in multiple ways: 1. As a test-time scaling tool for best-of-N judges, 2) as a mutation feedback mechanism in optimization loops and 3) as runtime feedback for coding agents. Previously people used MAST and other hand-made fixed failure taxonomies. In this paper we show how to create the failure taxonomy adaptively and dynamically: We observe agent rollouts and create an adaptive failure taxonomy, bespoke to the agent weaknesses and task challenges. These adaptive taxonomies give a massive boost: On Terminal Bench 2 we get 89.9% with Opus 4.6 / Forgecode harness and the adaptive taxonomy used with a Best-of-N Judge, outperforming fixed taxonomies by 15%. (1/n)
Show more