Day-0 support for
@Kimi_Moonshot K3.1 Pro in TokenSpeed.
Sparse attention for long contexts. With KV sharding and decode context parallelism, KV cache capacity scales from 2.43M to 16.35M tokens on 8ร NVIDIA GB300.
Available on AMD MI350X,
@NVIDIAAI Blackwell, and NVIDIA Vera Rubin.
๐
โญ๏ธ