註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

Artificial Analysis
@ArtificialAnlys
Independent analysis of AI
加入 January 2024
693 正在關注    178.9K 粉絲
Coding Agent Index v2.0 scores are lower than v1.5 for almost every configuration measured on both. Across the 21 configurations evaluated on both Index versions, the mean Index score moves from 54.8 to 47.4, with 20 of 21 moving down. The shift comes from FrontierCode v1.1, where the same configurations average 45.1% against 65.6% on DeepSWE v1.1, with a smaller contribution from stricter SWE-Atlas-QnA-AA grading. Terminal-Bench 4.0 is unchanged. Scores on the two Index versions reflect different evaluation setups.
顯示更多