💾 TL;DR: reuse a KV cache computed on another node's GPU straight through CXL memory, up to 36.6x faster. Seagate proposes a Kubernetes-native way to share CXL memory across the cluster.
Title: Composable CXL Memory as a Kubernetes-Native Shared Memory for LLM Serving
URL:
📌 Highlights
🧩 A Kubernetes DRA driver turns CXL regions into schedulable cluster resources
🗂 No Redis/etcd needed, KV slots are tracked inside the region itself
🚀 Cross-node reuse hits 36.6x speedup over recompute at 32K-token prefixes
🎯 Hit rates reach 95.4-99.5%
⚖️ Cross-node vs same-node latency differs by only 1-4%
🐢 Still 1.15-1.70x slower than an on-GPU VRAM prefix cache hit
🔧 Measured 5.8 GB/s out of a 27 GB/s fabric ceiling, room to grow with zero-copy DMA
For serving that needs to reuse long prefixes across GPU nodes, this feels like a genuinely practical option.
#
LLMInference# #
Kubernetes#