Alex Krentsel
@AlexKrentsel
Autonomous Systems PhD @Berkeley_EECS, Sr Researcher @GoogleResearch.
systems guy working on harness-layer self-improvement
Joined March 2022
251 Following 4.2K Followers
super cool work from Mayank. excellent blog post on how to reach very impressive performance with orders of magnitude less compute
We pretrained a 2.3B MoE (360M active) Hybrid Mamba-2 that lands within a few points of Llama-3.2-3B using <1% of its pretraining FLOPs.
No dedicated cluster. The run hopped between H100s, A100s, V100s (yes, V100s) and TPU v5p/v6e on a single codebase.
Meet Rigel 🧵
Show more
