Important to note that this increase comes solely from RND compute, and customer-serving compute is ~constant. and RND is variable you control & decide how large you want it to be. If you think you want to double down and pay from balance, not revenue, you can do that, too.

It's been a bit over 3 months since Kimi K3 was announced, and I made the prediction that, contrary to what Twitter hype merchants were saying, it wouldn't reach Claude Fable level, or even GPT-5.6 Sol. It didn't take a rocket scientist to arrive at the same conclusion — Moonshot said so themselves in their release blog post, kudos to them: “…its overall performance still trails … Claude Fable 5 and GPT 5.6 Sol…”. This turned out to be true — and now I have data from MORE THAN 90 benchmarks to corroborate the claim. Yes, you read that correctly — ninety benchmarks. All of these benchmarks were released after Kimi K3, so the possibility of overfitting is very low. If we compare K3 with GPT-5.5 and Opus 4.8 — released in late April and May, respectively — I can call them equal without hesitation. Kimi loses on as many benchmarks as it wins, and there's a solid group of benchmarks within a practical tie margin. The median difference in scores across the 40+ shared benchmarks is within 1% against both models, which is very close. It is really fair to say “Kimi K3 was the Opus 4.8 of open-source”. Now to the grim part. The fresh benchmarks don’t support parity with Sol or Fable: — Against Sol, Kimi lost on 62 out of 89 benchmarks. The median relative deficit was 14%, and on 30 benchmarks the gap was more than 20%. — Against Fable, Kimi managed to win just 3 benchmarks, with 2 more practical ties, and lost 35 of the 40. On 23 benchmarks, the relative deficit exceeded 20%. — If we focus on Agentic Coding benchmarks, Kimi loses 10/10 to Fable, trailing by a median of close to 30% (!) in relative terms. The “Fable-class” claims have aged badly. In my summer post, I wrote the following about Kimi: “if we gather another 30 benchmarks, we might see, say, 8 wins for Kimi, 5 ties, and 17 losses.” This turned out to be optimistic. Even if we compare it to GPT-5.6 Sol, it is more like 5 wins, 4 ties, and 21 losses per 30 benchmarks (proportions taken from all 89 benchmarks comparing these two). All of that, of course, is not to mention that since then (and before K3.1) we got the beautiful Astra, Fable 5.1, and even 3 Claude models from the 5.5 family, including the beast that is Opus. So, “on average,” proprietary models are still out on top, and the gap is significant. I'm not ready to say whether it has shrunk recently, but I'm planning to expand my analysis in the future so that we can see whether the gap widens or narrows. (As usual, disclaimer: please don't take this as “Kimi / DeepSeek is trash, and nobody should use them.” Check out the link below the post for the full details on my position.)

Very excited to be part of this effort and to finally share what we've been working on! Over the last few months, we've seen a wave of experimentation with autoresearch, AI Scientists, and similar ideas — all powered by frontier coding agents that can tackle long-horizon tasks, run experiments, and iteratively improve their results. At the same time, research reproducibility remains a major challenge. There's simply not enough time to verify every paper, or even reproduce its main results — especially when doing so can come with enormous compute costs. Robotics takes this challenge to another level. Even when the code is publicly available, reproducing a result often requires collecting and curating datasets, handling annotations, setting up hardware, and running real-world evaluations. The barrier to entry is incredibly high. Today, we're sharing our vision for bringing these two worlds together: leveraging frontier AI agents to automate much of the implementation work, while combining them with our robotics expertise and infrastructure to enable real-world evaluation at scale. Really excited about what we're building, and even more about what's coming next. Stay tuned!

This plot explains why OpenAI had to close $200 subs and why the quota gets cut. Anthropic is dedicating 42% of inference compute to subs, which generate 10% of revenue (10% of $70B ARR is ~$583M/mo). If OpenAI has 30% of revenue from subs - and we know they had 50% in July, but then B2B grew - this would mean OpenAI is spending 20% more compute on serving subs than Anthropic on ALL of inference.

I appreciate all the people who bring research on Looped Transformer that I missed, but I doubt the open research is the same as OpenAI's research. Yesterday I was thinking that Looped at THIS scale (Astra is clearly supposed to be bigger than anything open-sourced, >3T total) means that OpenAI has run countless ablations and scaling studies and decided to go with it. They wouldn't have spent >$100M w/o doing that. It should've scaled well, maybe not loss-wise but on downstream (though I doubt, most likely on both).

At first I thought this is BS, like, TB3 landed around 2 months ago? And they were collecting tasks for 4.5+ months! Note that there are NO new tasks in TB4: some got removed, some were modified. Which means some of the tasks are 5+ months old and were published alongside solutions/graders openly. Not saying Zai trained on them, but I'm not buying that none of the companies have used well-prepared envs available for free on GitHub in a very accessible form (Harbour-compatible).