Today is the Mid-Autumn Festival, the day when the moon is said to shine at its fullest and brightest. Mooncakes, rich pastries filled with sweet bean or lotus seed paste, are shared and enjoyed with family and friends during this festival. As the poem goes: “The bright moon rises over the sea; though far apart, we share this moment together.” Wishing everyone a happy Mid-Autumn Festival and a joyful time with your family! 🥮

Huge congrats to the @vllm_project team on pushing DSpark training for Kimi-K3 to this scale! 🚀 As the high-performance data plane behind the disaggregated pipeline, Mooncake moves hidden states between vLLM inference and training workers, fully utilizing RDMA bandwidth with topology-aware routing and multi-NIC aggregation. A great example of Mooncake powering large-scale AI data movement beyond KV cache transfer. Read more:

A great read on how GLM-5.3 helped optimize its own inference stack, tripling end-to-end throughput in under two weeks, all driven by a dense feedback loop between the agent and the system. One particularly interesting case for us: while debugging a KV Transfer bottleneck, the team traced a >20% Prefill + KV Transfer performance gap due to Python GIL contention between DeepEP and the Mooncake Transfer thread. After releasing the GIL in the relevant C++ paths, the gap dropped to below 1%.

🚀 Over one trillion tokens per day, powered by Mooncake and SGLang. @ApproachingAI shares the system architecture and engineering lessons behind its token factory, running leading trillion-parameter models with SGLang HiCache + Mooncake Store. A deep dive into: 🏗️ Production architecture and system design trade-offs ⚡ Performance optimizations for trillion-token-scale workloads 🛡️ Failure isolation, fault tolerance, and recovery Real production lessons on scaling KV cache reuse while meeting strict inference SLOs. Read more:

Thanks for the recognition! Mooncake has always been focused on building high-performance storage and communication infrastructure for large-scale distributed LLM serving. We’re excited to keep pushing this forward with the community.

Mooncake v0.3.13 is here! 🚀 This release brings major updates across Mooncake Store, Transfer Engine, Process Group, hardware support, and ecosystem integrations, with 276 commits and 33 new contributors. A huge thank you to all the users and contributors! ❤️ ✨ Highlights: • Mooncake Store: Multi-tenancy and NVMe KV SSD support • Transfer Engine: Enhanced NVLink, IBGDA, and multi-rail transfer support • Process Group: Dynamic membership and recovery • New integrations with Miles and Speculators • Broader platform support: NCCL, MUSA IPC, SUPA, and FlagCX • Published wheel packages for ROCm

Agentic inference is making KV cache management and data movement increasingly central to the serving stack. The new AgentX / InferenceXv3 analysis from @SemiAnalysis_ captures this shift well. Glad to see Mooncake highlighted for external KV caching and disaggregated P/D data transfer, including support for hybrid-attention cache management and preserving speculative decoding state. Special thanks to AMD and SemiAnalysis for providing the CI resources and technical support that made Mooncake ROCm wheel packaging possible. We’re excited to ship it in our next release. Read more:

Excited to team up with @radixark to make large-scale RL data movement faster 🚀 Miles is built for high-performance, large-scale post-training, and Mooncake is now integrated as a rollout data-transfer backend for the fragmented, heterogeneous data moving between rollout and training in disaggregated RL. On rollout data captured from Miles: ⚡ 10–14× faster remote GET ⚡ 1.2–1.6× faster PUT By turning fragmented rollout objects into efficient bulk I/O while preserving their original structure, Mooncake helps reduce rollout-to-training handoff latency without changing the RL programming model. Read more:

Congrats to the Miles team on the v0.1 launch! 🚀 AgentENV is now integrated with Miles v0.1 as a sandbox backend for agentic RL. Built for high-performance, production-scale post-training, Miles provides a powerful foundation for agentic RL with fully asynchronous training and fast, flexible rollouts. AgentENV complements Miles by providing isolated, fast-starting Firecracker microVMs for every rollout, giving agents fresh environments to write code, execute commands, and call tools without sandbox startup becoming the bottleneck. Validated on a sustained GRPO run across the full Terminal-Bench-2 suite, this integration powered ~3,400 episodes across 55 rollouts with GLM-4.7-Flash on 8×H200s. Every episode ran in a fresh, snapshot-warmed microVM, while a single m7i.metal-24xl AgentENV server sustained 64 concurrent environments throughout.

Proud to see Mooncake featured among the supporters of @SemiAnalysis_ InferenceX initiative. Reliable, transparent benchmarking is critical as inference systems become increasingly disaggregated and complex. We’re excited to contribute to this ecosystem and help push the frontier of efficient, scalable AI inference. More details: #InferenceX# #AIInference# #Mooncake#

TileRT is pushing NVIDIA GPUs into a new regime of ultra-high-interactivity inference, with a specialized decode engine delivering hundreds of tokens per second per user. In its disaggregated prefill/decode architecture, Mooncake Transfer Engine provides the high-performance data path for moving KV cache from vLLM prefill nodes to TileRT decode nodes. Read the full deep dive from SemiAnalysis: #LLMInference# #TileRT# #vLLM# #Mooncake#