Some thoughts on latest AA v5 index rankings.
1. DS 1017 performance on (new) FrontierCode 2.0 is impressive - can't be the result of benchmaxxing: close to Fable 5.1, slightly ahead of Argon and Kimi 3.1
2. Poor science+math performance of Chinese models dropped them signicantly in new index relative to US frontier models. I've been talking to a lot of Chinese labs recently and they have not prioritized this area at all. They've been very focused on coding+agentic capability. Unlike the big US labs they don't have teams of scientists and math guys pushing in this direction. I predict they will reprioritize, bc of the OAI math breakthroughs and the poor AA v5 scores.
Their attitude seems to have been: for math, once IMO Gold level achieved, diminishing returns to further math capability improvements. Same with basic science. Economic returns all concentrated in coding and agentic tasks. Even now I doubt they will move more than 10% of their specialized model capability resources toward math+science.
New evals test frontier-level capability in math and physics, exposing the Chinese models as far behind Astra and Opus 5.5.
3. Gemini Carbon scores are amazing. Hope there is now a GOOG model at the frontier that I can use in my research!
I strongly believe that adversarial verification of math+phys results by the best models is much better than human peer review.