注册并分享邀请链接,可获得视频播放与邀请奖励。

Mayank Mishra
@MayankMish98
Doing philosophy with @tri_dao, @profjoeyg, @istoica05
加入 June 2017
696 正在关注    2K 粉丝
We pretrained a 2.3B MoE (360M active) Hybrid Mamba-2 that lands within a few points of Llama-3.2-3B using <1% of its pretraining FLOPs. No dedicated cluster. The run hopped between H100s, A100s, V100s (yes, V100s) and TPU v5p/v6e on a single codebase. Meet Rigel 🧵
显示更多
0
22
633
80
转发到社区