가입 후 초대 링크를 공유하면 동영상 재생 및 초대 보상을 받을 수 있습니다.

Vezko | zkPass
@vezko_zkp
⚡ Crafting privacy vibes @zkPass 🎨 Proud #Doodles# Holder 🚀 Building cool stuff with #zkTLS# #AI#
가입 March 2022
1.1K 팔로잉 중    2.3K 팬
so the benchmark penalizes refactoring, and opus 5 refactors more at high reasoning. that's not a bug, it's a personality. frontiercode wants a model that follows orders, not one that thinks.
We've received several questions about the Opus 5 FrontierCode results, where scores decline as reasoning effort increases. In fact, the behavior is expected under the benchmark design. FrontierCode evaluates merge-ability rather than correctness alone, incorporating criteria that reflect user experience. One such verifier is a scope criterion, which penalizes modifications to the codebase beyond what the task requires. We observe this effect across all frontier models we evaluated, though it is most pronounced in Opus 5: at higher reasoning efforts, the model shows a stronger tendency to refactor code unprompted.
더 보기