註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

Mayank Mishra
@MayankMish98
Doing philosophy with @tri_dao, @profjoeyg, @istoica05
加入 June 2017
696 正在關注    2K 粉絲
We pretrained a 2.3B MoE (360M active) Hybrid Mamba-2 that lands within a few points of Llama-3.2-3B using <1% of its pretraining FLOPs. No dedicated cluster. The run hopped between H100s, A100s, V100s (yes, V100s) and TPU v5p/v6e on a single codebase. Meet Rigel 🧵
顯示更多
0
22
633
80
轉發到社區