Woosuk Kwon(@woosuk_k):KV cache management is arguably one of the hardest problems in LLM inference, and advancing it requires the community to learn from and build on one another’s work. Great to see that happening. Amazing work from @lightseekorg on adopting and evolving vLLM's hybrid memory allocator! The Jenga paper from @ChenZha62999224 is especially relevant today as attention architectures become increasingly complex.

Woosuk Kwon

@woosuk_k

@inferact | @vllm_project | prev: PhD @Berkeley_EECS

加入 April 2023

826 正在關注 8.9K 粉絲

Woosuk Kwon@woosuk_k

2026.07.16 23:09

KV cache management is arguably one of the hardest problems in LLM inference, and advancing it requires the community to learn from and build on one another’s work. Great to see that happening. Amazing work from @lightseekorg on adopting and evolving vLLM's hybrid memory allocator! The Jenga paper from @ChenZha62999224 is especially relevant today as attention architectures become increasingly complex.

488

轉發到社區

熱門用戶