KVCache.AI(@KVCache_AI):TileRT is pushing NVIDIA GPUs into a new regime of ultra-high-interactivity inference, with a specialized decode engine delivering hundreds of tokens per second per user. In its disaggregated prefill/decode architecture, Mooncake Transfer Engine provides the high-performance data path for moving KV cache from vLLM prefill nodes to TileRT decode nodes. Read the full deep dive from SemiAnalysis: https://t.co/1UI8X0XFh2 #LLMInference #TileRT #vLLM #Mooncake

KVCache.AI

@KVCache_AI

Hi, this is official account. We build systems for efficient LLM serving, including KTransformers, Mooncake and AgentENV.

参加 August 2018

109 フォロー中 1.1K ファン

KVCache.AI@KVCache_AI

2026.08.17 13:57

TileRT is pushing NVIDIA GPUs into a new regime of ultra-high-interactivity inference, with a specialized decode engine delivering hundreds of tokens per second per user. In its disaggregated prefill/decode architecture, Mooncake Transfer Engine provides the high-performance data path for moving KV cache from vLLM prefill nodes to TileRT decode nodes. Read the full deep dive from SemiAnalysis: #LLMInference# #TileRT# #vLLM# #Mooncake#