Register and share your invite link to earn from video plays and referrals.

Jennifer Zhou
@chem_safety
rl data @preferencemodel | prev pretraining data @AnthropicAI
427 Following    2.1K Followers
We looked at TerminalBench 4.0 and found a lot of issues. After fixing the issues Sonnet 5.5 scores 89.2%. So, TerminalBench is already mostly saturated. Along the way we found a lot of crazy stuff, including reward hacks in the published transcripts.
Show more
0
78
1.6K
119
Forward to community
Today we are open-sourcing 🥕Karotte, our framework for building RL environments. We've used it for the past year to build MLE RL environments for frontier labs, and it's been hardened through 1M+ evaluation runs and red-teaming.
Show more
0
69
1.1K
121
Forward to community