Frank Downing(@downingARK):There is something clearly different in how Anthropic & OpenAI scale effort levels. Doesn't show up on every benchmark, but these results on FrontierCode make it really clear. Anthropic models peak at lower effort levels, where as OpenAI models start low and climb up fairly consistently. The result is a better score at a lower cost for Anthropic models, but an unintuitive experience where increasing effort does not increase scores and might actually degrade performance (on this benchmark at least).

Frank Downing

@downingARK

Director of Research, AI & Cloud @ARKinvest Disclosure:

参加 April 2021

607 フォロー中 20.8K ファン

Frank Downing@downingARK

2026.09.22 19:31

There is something clearly different in how Anthropic & OpenAI scale effort levels. Doesn't show up on every benchmark, but these results on FrontierCode make it really clear. Anthropic models peak at lower effort levels, where as OpenAI models start low and climb up fairly consistently. The result is a better score at a lower cost for Anthropic models, but an unintuitive experience where increasing effort does not increase scores and might actually degrade performance (on this benchmark at least).