注册并分享邀请链接,可获得视频播放与邀请奖励。

Kimi.ai
@Kimi_Moonshot
Built by Moonshot AI to empower everyone to be superhuman. PR: globalpr@moonshot.ai DC:
加入 December 2024
138 正在关注    394.6K 粉丝
Zhilin at GTC: Introducing Attention Residuals Learning selective memory, rather than mechanically accumulating everything, is the beauty of attention. Many of you have probably read Attention Is All You Need, the 2017 Transformer paper that brought “human-like” attention into the model’s field of view. From that point on, models no longer simply read everything in a mechanical way. Instead, they began to develop a sense of what matters more and what matters less across the text, choosing to retain the more important information. Recently, Kimi applied this idea of attention to the temporal dimension, then rotated it 90 degrees into the model’s depth dimension. This allows the model to have attention not only over time, but also throughout the process of information transmission across layers—giving it a more intelligent way to understand and process information.
显示更多
0
50
1.5K
156
转发到社区