注册并分享邀请链接,可获得视频播放与邀请奖励。

Artificial Analysis
@ArtificialAnlys
Independent analysis of AI
加入 January 2024
693 正在关注    180.1K 粉丝
Coding Agent Index v2.0 scores are lower than v1.5 for almost every configuration measured on both. Across the 21 configurations evaluated on both Index versions, the mean Index score moves from 54.8 to 47.4, with 20 of 21 moving down. The shift comes from FrontierCode v1.1, where the same configurations average 45.1% against 65.6% on DeepSWE v1.1, with a smaller contribution from stricter SWE-Atlas-QnA-AA grading. Terminal-Bench 4.0 is unchanged. Scores on the two Index versions reflect different evaluation setups.
显示更多