註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

Kimi.ai
@Kimi_Moonshot
Built by Moonshot AI to empower everyone to be superhuman. PR: globalpr@moonshot.ai DC:
加入 December 2024
138 正在關注    393.9K 粉絲
Zhilin at GTC: Introducing Attention Residuals Learning selective memory, rather than mechanically accumulating everything, is the beauty of attention. Many of you have probably read Attention Is All You Need, the 2017 Transformer paper that brought “human-like” attention into the model’s field of view. From that point on, models no longer simply read everything in a mechanical way. Instead, they began to develop a sense of what matters more and what matters less across the text, choosing to retain the more important information. Recently, Kimi applied this idea of attention to the temporal dimension, then rotated it 90 degrees into the model’s depth dimension. This allows the model to have attention not only over time, but also throughout the process of information transmission across layers—giving it a more intelligent way to understand and process information.
顯示更多
0
50
1.5K
156
轉發到社區