here is an overview of the 4 most advanced efficient architectures: Deepseek V4.1 Flash, MiMo V3, Qwen 3.8 Next Flash and GLM 5.3 Flash Deepseek and MiMo are quite similar, they both use YOCO - only the first part of the network is active during prefill to build the KV cache representation - no linear attention, and token level indexer Qwen and GLM both use a more standard interleaving 3:1 (like Kimi K3 as well) between sparse attention and linear attention (GDN vs KDA) both deepseek and qwen use Engram, they all use a gate or sink except GLM 5.3 Flash, they also all use no or partial RoPE on the full/sparse attention layers they also all have some sophisticated residual network, either mHC (simplified or not) or gated residual and they are all trained using Muon visualization by opus 5.5 and me :)