注册并分享邀请链接,可获得视频播放与邀请奖励。

Alex Krentsel
@AlexKrentsel
Autonomous Systems PhD @Berkeley_EECS, Sr Researcher @GoogleResearch. systems guy working on harness-layer self-improvement
加入 March 2022
251 正在关注    4.2K 粉丝
super cool work from Mayank. excellent blog post on how to reach very impressive performance with orders of magnitude less compute
We pretrained a 2.3B MoE (360M active) Hybrid Mamba-2 that lands within a few points of Llama-3.2-3B using <1% of its pretraining FLOPs. No dedicated cluster. The run hopped between H100s, A100s, V100s (yes, V100s) and TPU v5p/v6e on a single codebase. Meet Rigel 🧵
显示更多