KVCache.AI
@KVCache_AI
Hi, this is official account. We build systems for efficient LLM serving, including KTransformers, Mooncake and AgentENV.
Joined August 2018
109 Following 1.1K Followers
đ Over one trillion tokens per day, powered by Mooncake and SGLang.
@ApproachingAI shares the system architecture and engineering lessons behind its token factory, running leading trillion-parameter models with SGLang HiCache + Mooncake Store.
A deep dive into:
đī¸ Production architecture and system design trade-offs
⥠Performance optimizations for trillion-token-scale workloads
đĄī¸ Failure isolation, fault tolerance, and recovery
Real production lessons on scaling KV cache reuse while meeting strict inference SLOs.
Read more:
Show more
