Ali Hatamizadeh
@ahatamiz1
LLM Tech Lead & Staff Research Scientist @NVIDIA
Co-creator of Gated DeltaNet & Gated DeltaNet-2
加入 June 2015
180 正在关注 4.1K 粉丝
GDN-2 scales nicely with Muon 🚀
We trained hybrid MoE models at three scales (2B-7B) with Muon.
GDN-2 matches Mamba2's loss with ~10% less training compute!
