Ali Hatamizadeh
@ahatamiz1
LLM Tech Lead & Staff Research Scientist @NVIDIA
Co-creator of Gated DeltaNet & Gated DeltaNet-2
参加 June 2015
180 フォロー中 4.1K ファン
GDN-2 scales nicely with Muon 🚀
We trained hybrid MoE models at three scales (2B-7B) with Muon.
GDN-2 matches Mamba2's loss with ~10% less training compute!
