SemiAnalysis
@SemiAnalysis_
参加 January 2024
38 フォロー中 185.3K ファン
After DeepSeek V4.1 Flash, Xiaomi MiMo is moving to a causal encoder-decoder for its next-gen models.
HySparse2: prompts run through only 25 of 49 layers. Decoder KV is projected from encoder states.
At 1M context vs MiMo-V2: 5x less prefill compute, 4.5x smaller KV cache.
もっと見る
