Ali Hatamizadeh(@ahatamiz1):We’ve already trained larger variants of this model, and they outperform competing approaches, including Mamba-2, GDP, and KDA, by a substantial margin. One near-term release candidate is the hybrid GDN-2-3B latent MoE, which follows the Nemotron-3 Nano architecture but replaces the Mamba-2 layers with GDN-2. A public release requires several approvals, and we’re working hard to secure them!

Ali Hatamizadeh

@ahatamiz1

LLM Tech Lead & Staff Research Scientist @NVIDIA Co-creator of Gated DeltaNet & Gated DeltaNet-2

加入 June 2015

180 正在关注 4.1K 粉丝

Ali Hatamizadeh@ahatamiz1

2026.09.26 02:28

We’ve already trained larger variants of this model, and they outperform competing approaches, including Mamba-2, GDP, and KDA, by a substantial margin. One near-term release candidate is the hybrid GDN-2-3B latent MoE, which follows the Nemotron-3 Nano architecture but replaces the Mamba-2 layers with GDN-2. A public release requires several approvals, and we’re working hard to secure them!

显示更多