Mooncake v0.3.13 is here! 🚀 This release brings major updates across Mooncake Store, Transfer Engine, Process Group, hardware support, and ecosystem integrations, with 276 commits and 33 new contributors. A huge thank you to all the users and contributors! ❤️ ✨ Highlights: • Mooncake Store: Multi-tenancy and NVMe KV SSD support • Transfer Engine: Enhanced NVLink, IBGDA, and multi-rail transfer support • Process Group: Dynamic membership and recovery • New integrations with Miles and Speculators • Broader platform support: NCCL, MUSA IPC, SUPA, and FlagCX • Published wheel packages for ROCm

"Mooncake originated from a research collaboration between Kimi(Moonshot AI) and Tsinghua University. It was born from the need to solve the 'memory wall' in serving massive-scale models like Kimi K-Series. Since open-sourcing, it has evolved into a thriving community-driven project." GitHub:

Happy Mooncake Festival everyone! ❤️ Tap ❤️ to show me your 💖 RT if you ❤️ me! 🥺 💌 October Theme is OUT! 💌 #justsushii# #cosplaygirl# #casualoutfitsdaily# #thicc# #cute# #animelover# #sexybabes# #curveygirls# #kawaiistyle# #patreoncreators#

Proud to see Mooncake featured among the supporters of @SemiAnalysis_ InferenceX initiative. Reliable, transparent benchmarking is critical as inference systems become increasingly disaggregated and complex. We’re excited to contribute to this ecosystem and help push the frontier of efficient, scalable AI inference. More details: #InferenceX# #AIInference# #Mooncake#

We’re excited to welcome Mooncake to the PyTorch Ecosystem! Mooncake is designed to solve the “memory wall” in LLM serving. By integrating Mooncake’s high performance KVCache transfer and storage capabilities with PyTorch native inference engines like SGLang, vLLM, and TensorRT-LLM, it unlocks new levels of throughput and scalability for large language model deployments. Mooncake enables prefill decode disaggregation, global KVCache reuse, elastic expert parallelism, and serves as a fault tolerant PyTorch distributed backend. 🔗 #PyTorch# #OpenSourceAI# #LLM# #AIInfrastructure#

★Electrifying Moon-Viewing★ Tea, mooncakes... Got everything for the moon viewing. But, I'm hungry now... Can I have a little snack now? (Eldridge is already munching on a mooncake.) USS Eldridge is changing into her new attire. She will grace your dock in the near future, Commander. #AzurLane# #Yostar#

TPU🚨 is working with the popular OSS inference optimization library Mooncake on integrating TPU with Mooncake Store. Similar to NVL72, KVCache DRAM P2P pooling will initially happen on the scale-out network via TENT instead of using ICI/NVLink. 🔥 Mooncake basically improves performance per TCO of production inference!