Cheng-Yuan (Sam) Lee
@cl571128
Research @cognition | prev intern @windsurf | 2x ICPC World Finalist | from Taiwan 🇹🇼
419 Following 691 Followers
We've received several questions about the Opus 5 FrontierCode results, where scores decline as reasoning effort increases. In fact, the behavior is expected under the benchmark design. FrontierCode evaluates merge-ability rather than correctness alone, incorporating criteria that reflect user experience.
One such verifier is a scope criterion, which penalizes modifications to the codebase beyond what the task requires. We observe this effect across all frontier models we evaluated, though it is most pronounced in Opus 5: at higher reasoning efforts, the model shows a stronger tendency to refactor code unprompted.
Show more
