Register and share your invite link to earn from video plays and referrals.

steve hsu
@hsu_steve
Physicist, AI Founder, Manifold Podcast
Joined June 2010
0 Following    48.8K Followers
Some thoughts on latest AA v5 index rankings. 1. DS 1017 performance on (new) FrontierCode 2.0 is impressive - can't be the result of benchmaxxing: close to Fable 5.1, slightly ahead of Argon and Kimi 3.1 2. Poor science+math performance of Chinese models dropped them signicantly in new index relative to US frontier models. I've been talking to a lot of Chinese labs recently and they have not prioritized this area at all. They've been very focused on coding+agentic capability. Unlike the big US labs they don't have teams of scientists and math guys pushing in this direction. I predict they will reprioritize, bc of the OAI math breakthroughs and the poor AA v5 scores. Their attitude seems to have been: for math, once IMO Gold level achieved, diminishing returns to further math capability improvements. Same with basic science. Economic returns all concentrated in coding and agentic tasks. Even now I doubt they will move more than 10% of their specialized model capability resources toward math+science. New evals test frontier-level capability in math and physics, exposing the Chinese models as far behind Astra and Opus 5.5. 3. Gemini Carbon scores are amazing. Hope there is now a GOOG model at the frontier that I can use in my research! I strongly believe that adversarial verification of math+phys results by the best models is much better than human peer review.
Show more