Peter J. Liu(@peterjliu):Reading Kimi K3 paper, I'm struck by the number of innovations and departures from orthodoxy. These guys are doing real research and not copying. It is no longer the case of being, "mostly the Transformer with a few Noam mods". I mean they got rid of the position encodings/embeddings! Bold! More thoughts to come after reading.

Peter J. Liu

@peterjliu

Building @getcompoundai. Was Research Scientist @ Google Brain / DeepMind, language model research. No AI slop-posting. 🇺🇸🇨🇦

Joined May 2009

2.6K Following 9.4K Followers

Peter J. Liu@peterjliu

2026.07.27 22:09

Reading Kimi K3 paper, I'm struck by the number of innovations and departures from orthodoxy. These guys are doing real research and not copying. It is no longer the case of being, "mostly the Transformer with a few Noam mods". I mean they got rid of the position encodings/embeddings! Bold! More thoughts to come after reading.

2.4K

160

Forward to community

Most Popular Users