331 output tokens/s per GPU on
@Kimi_Moonshot K3 NVFP4, powered by TokenSpeed โก๏ธ
Running the full
@SemiAnalysis_ AgentX trace on 32
@NVIDIAAI GB300 GPUs, TokenSpeed achieves 1.92ร the output throughput per GPU of the standard eight-GPU DCP8 baseline.
Cache hit rate and workload balance are the foundations of high-throughput agentic inference. Our latest blog explains how independent DWDP prefill, efficient DEP decode, and shared caching work together to keep long-running agent sessions moving. ๐
๐
โญ๏ธ