註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

Shivam Duggal
@ShivamDuggal4
PhD Student @MIT | Prev: Carnegie Mellon University @SCSatCMU | Research Scientist @UberATG
加入 June 2017
455 正在關注    1.6K 粉絲
Language models process each token through a single residual stream (blue arrows): reading / writing memory and applying MLPs at each layer, and finally decoding the next token. Can we scale up this single stream computation? Introducing Latent Lattice Language Models (LLLMs): augmenting this vertical stream with a horizontal axis of computation—the Lattice of Latents (orange arrows). RNN 🤝 Transformers 🧵
顯示更多
0
17
364
59
轉發到社區