331 output tokens/s per GPU on
@Kimi_Moonshot K3 NVFP4, powered by TokenSpeed âĄī¸
Running the full
@SemiAnalysis_ AgentX trace on 32
@NVIDIAAI GB300 GPUs, TokenSpeed achieves 1.92Ã the output throughput per GPU of the standard eight-GPU DCP8 baseline.
Cache hit rate and workload balance are the foundations of high-throughput agentic inference. Our latest blog explains how independent DWDP prefill, efficient DEP decode, and shared caching work together to keep long-running agent sessions moving. đ
đ
âī¸