Alex Krentsel
@AlexKrentsel
Autonomous Systems PhD @Berkeley_EECS, Sr Researcher @GoogleResearch.
systems guy working on harness-layer self-improvement
가입 March 2022
251 팔로잉 중 4.2K 팬
super cool work from Mayank. excellent blog post on how to reach very impressive performance with orders of magnitude less compute
We pretrained a 2.3B MoE (360M active) Hybrid Mamba-2 that lands within a few points of Llama-3.2-3B using <1% of its pretraining FLOPs.
No dedicated cluster. The run hopped between H100s, A100s, V100s (yes, V100s) and TPU v5p/v6e on a single codebase.
Meet Rigel 🧵
더 보기
