Peter J. Liu
@peterjliu
Building @getcompoundai. Was Research Scientist @ Google Brain / DeepMind, language model research. No AI slop-posting. 🇺🇸🇨🇦
加入 May 2009
2.6K 正在關注 9.4K 粉絲
Reading Kimi K3 paper, I'm struck by the number of innovations and departures from orthodoxy. These guys are doing real research and not copying. It is no longer the case of being, "mostly the Transformer with a few Noam mods".
I mean they got rid of the position encodings/embeddings! Bold!
More thoughts to come after reading.
顯示更多
