SemiAnalysis(@SemiAnalysis_):After DeepSeek V4.1 Flash, Xiaomi MiMo is moving to a causal encoder-decoder for its next-gen models. HySparse2: prompts run through only 25 of 49 layers. Decoder KV is projected from encoder states. At 1M context vs MiMo-V2: 5x less prefill compute, 4.5x smaller KV cache.

2026.11.07 18:57

After DeepSeek V4.1 Flash, Xiaomi MiMo is moving to a causal encoder-decoder for its next-gen models. HySparse2: prompts run through only 25 of 49 layers. Decoder KV is projected from encoder states. At 1M context vs MiMo-V2: 5x less prefill compute, 4.5x smaller KV cache.