SemiAnalysis
@SemiAnalysis_
加入 January 2024
38 正在關注 186K 粉絲
After DeepSeek V4.1 Flash, Xiaomi MiMo is moving to a causal encoder-decoder for its next-gen models.
HySparse2: prompts run through only 25 of 49 layers. Decoder KV is projected from encoder states.
At 1M context vs MiMo-V2: 5x less prefill compute, 4.5x smaller KV cache.
顯示更多
