It's been a bit over 3 months since Kimi K3 was announced, and I made the prediction that, contrary to what Twitter hype merchants were saying, it wouldn't reach Claude Fable level, or even GPT-5.6 Sol. It didn't take a rocket scientist to arrive at the same conclusion — Moonshot said so themselves in their release blog post, kudos to them: “…its overall performance still trails … Claude Fable 5 and GPT 5.6 Sol…”. This turned out to be true — and now I have data from MORE THAN 90 benchmarks to corroborate the claim. Yes, you read that correctly — ninety benchmarks. All of these benchmarks were released after Kimi K3, so the possibility of overfitting is very low. If we compare K3 with GPT-5.5 and Opus 4.8 — released in late April and May, respectively — I can call them equal without hesitation. Kimi loses on as many benchmarks as it wins, and there's a solid group of benchmarks within a practical tie margin. The median difference in scores across the 40+ shared benchmarks is within 1% against both models, which is very close. It is really fair to say “Kimi K3 was the Opus 4.8 of open-source”. Now to the grim part. The fresh benchmarks don’t support parity with Sol or Fable: — Against Sol, Kimi lost on 62 out of 89 benchmarks. The median relative deficit was 14%, and on 30 benchmarks the gap was more than 20%. — Against Fable, Kimi managed to win just 3 benchmarks, with 2 more practical ties, and lost 35 of the 40. On 23 benchmarks, the relative deficit exceeded 20%. — If we focus on Agentic Coding benchmarks, Kimi loses 10/10 to Fable, trailing by a median of close to 30% (!) in relative terms. The “Fable-class” claims have aged badly. In my summer post, I wrote the following about Kimi: “if we gather another 30 benchmarks, we might see, say, 8 wins for Kimi, 5 ties, and 17 losses.” This turned out to be optimistic. Even if we compare it to GPT-5.6 Sol, it is more like 5 wins, 4 ties, and 21 losses per 30 benchmarks (proportions taken from all 89 benchmarks comparing these two). All of that, of course, is not to mention that since then (and before K3.1) we got the beautiful Astra, Fable 5.1, and even 3 Claude models from the 5.5 family, including the beast that is Opus. So, “on average,” proprietary models are still out on top, and the gap is significant. I'm not ready to say whether it has shrunk recently, but I'm planning to expand my analysis in the future so that we can see whether the gap widens or narrows. (As usual, disclaimer: please don't take this as “Kimi / DeepSeek is trash, and nobody should use them.” Check out the link below the post for the full details on my position.)