Peter J. Liu
@peterjliu
Building @getcompoundai. Was Research Scientist @ Google Brain / DeepMind, language model research. No AI slop-posting. 🇺🇸🇨🇦
2.6K Following 9.4K Followers
Reading Kimi K3 paper, I'm struck by the number of innovations and departures from orthodoxy. These guys are doing real research and not copying. It is no longer the case of being, "mostly the Transformer with a few Noam mods".
I mean they got rid of the position encodings/embeddings! Bold!
More thoughts to come after reading.
Show more
