KVCache.AI
@KVCache_AI
Hi, this is official account. We build systems for efficient LLM serving, including KTransformers, Mooncake and AgentENV.
Joined August 2018
109 Following 1.1K Followers
Excited to ship Mooncake alongside Miles v0.1 đ
Looking forward to pushing large-scale RL infrastructure forward together.
Full blog:
Shipping alongside Miles v0.1, Mooncake lands as a new rollout data-transfer backend in Miles, making remote GET 10-14Ã faster than the existing path.
With @KVCache_AI, we gave the rollout-to-training handoff a dedicated data plane:
- 1.2-1.6Ã faster PUT via structured-object transfer
- Structure-aware PUT/GET optimizes serialization and bulk RDMA transfer, including zero-copy reconstruction from registered buffers on GET
- Same put/get calls, no change to the RL programming model
Read the full blog đ link in the comment
Show more
