SemiAnalysis
@SemiAnalysis_
加入 January 2024
38 正在关注 186.6K 粉丝
After DeepSeek V4.1 Flash, Xiaomi MiMo is moving to a causal encoder-decoder for its next-gen models.
HySparse2: prompts run through only 25 of 49 layers. Decoder KV is projected from encoder states.
At 1M context vs MiMo-V2: 5x less prefill compute, 4.5x smaller KV cache.
显示更多
