Ali Hatamizadeh
@ahatamiz1
LLM Tech Lead & Staff Research Scientist @NVIDIA
Co-creator of Gated DeltaNet & Gated DeltaNet-2
加入 June 2015
180 正在關注 4.1K 粉絲
GDN-2 scales nicely with Muon 🚀
We trained hybrid MoE models at three scales (2B-7B) with Muon.
GDN-2 matches Mamba2's loss with ~10% less training compute!
