Artificial Analysis(@ArtificialAnlys):Coding Agent Index v2.0 scores are lower than v1.5 for almost every configuration measured on both. Across the 21 configurations evaluated on both Index versions, the mean Index score moves from 54.8 to 47.4, with 20 of 21 moving down. The shift comes from FrontierCode v1.1, where the same configurations average 45.1% against 65.6% on DeepSWE v1.1, with a smaller contribution from stricter SWE-Atlas-QnA-AA grading. Terminal-Bench 4.0 is unchanged. Scores on the two Index versions reflect different evaluation setups.

2026.10.23 00:17

Coding Agent Index v2.0 scores are lower than v1.5 for almost every configuration measured on both. Across the 21 configurations evaluated on both Index versions, the mean Index score moves from 54.8 to 47.4, with 20 of 21 moving down. The shift comes from FrontierCode v1.1, where the same configurations average 45.1% against 65.6% on DeepSWE v1.1, with a smaller contribution from stricter SWE-Atlas-QnA-AA grading. Terminal-Bench 4.0 is unchanged. Scores on the two Index versions reflect different evaluation setups.