가입 후 초대 링크를 공유하면 동영상 재생 및 초대 보상을 받을 수 있습니다.

Leo Linsky
@leo_linsky
RL and RSI @ Also: computational chemistry, quant trading, EE Generally curious
가입 March 2014
189 팔로잉 중    1.1K 팬
We ran Fable 5.2 through 100 unsaturated coding and engineering environments spanning spatial reasoning, planning, games, circuit design, and more. As with all Fable releases, it was an extremely expensive benchmark to run. The biggest winner, however, was OpenAI. Astra, GPT 6.1-Sol, and even GPT 6.1-Terra came up with better solutions to challenging, open-ended technical problems across financial markets, engineering design, and unsaturated coding problems. It's fairly obvious to us why the lion's share of mathematical breakthroughs are coming out of OpenAI. For what it's worth, Anthropic is all-in on training a good coding assistant (which they are succeeding at), not releasing the smartest model. Some things we noticed about Fable 5.2: • It stopped early in our agentic coding runs to ask for human help or clarification "I want you to decide on them before I tune any further" (there is no human to interact with and no reason to expect there would be from the prompt -- we've never seen another model attempt this) • It became confused in our harness (not Claude Code) and spent time trying to understand past tool calls. We have not seen this behavior from other OpenAI or Anthropic models and it hurt its agentic coding scores. • It sometimes used all of its reasoning budget and exceeded its token limit • Several content filter failures (does not affect its score) From our testing, there is no reason to use Fable 5.2 for any purpose, regardless of budget. Use Opus 5.5 for coding, and Astra or Sol for code reviews and challenging, unsaturated problems in engineering, math, and science.
더 보기