Day-0 support for
@Kimi_Moonshot K3.1 Pro in TokenSpeed.
Sparse attention for long contexts. With KV sharding and decode context parallelism, KV cache capacity scales from 2.43M to 16.35M tokens on 8× NVIDIA GB300.
Available on AMD MI350X,
@NVIDIAAI Blackwell, and NVIDIA Vera Rubin.
📖
⭐️