Frank Downing(@downingARK):There is something clearly different in how Anthropic & OpenAI scale effort levels. Doesn't show up on every benchmark, but these results on FrontierCode make it really clear. Anthropic models peak at lower effort levels, where as OpenAI models start low and climb up fairly consistently. The result is a better score at a lower cost for Anthropic models, but an unintuitive experience where increasing effort does not increase scores and might actually degrade performance (on this benchmark at least).

2026.09.22 19:31

There is something clearly different in how Anthropic & OpenAI scale effort levels. Doesn't show up on every benchmark, but these results on FrontierCode make it really clear. Anthropic models peak at lower effort levels, where as OpenAI models start low and climb up fairly consistently. The result is a better score at a lower cost for Anthropic models, but an unintuitive experience where increasing effort does not increase scores and might actually degrade performance (on this benchmark at least).