註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

Alex Krentsel
@AlexKrentsel
Autonomous Systems PhD @Berkeley_EECS, Sr Researcher @GoogleResearch. systems guy working on harness-layer self-improvement
加入 March 2022
251 正在關注    4.2K 粉絲
super cool work from Mayank. excellent blog post on how to reach very impressive performance with orders of magnitude less compute
We pretrained a 2.3B MoE (360M active) Hybrid Mamba-2 that lands within a few points of Llama-3.2-3B using <1% of its pretraining FLOPs. No dedicated cluster. The run hopped between H100s, A100s, V100s (yes, V100s) and TPU v5p/v6e on a single codebase. Meet Rigel 🧵
顯示更多