Woosuk Kwon(@woosuk_k):KV cache management is arguably one of the hardest problems in LLM inference, and advancing it requires the community to learn from and build on one another’s work. Great to see that happening. Amazing work from @lightseekorg on adopting and evolving vLLM's hybrid memory allocator! The Jenga paper from @ChenZha62999224 is especially relevant today as attention architectures become increasingly complex.

Woosuk Kwon

@woosuk_k

@inferact | @vllm_project | prev: PhD @Berkeley_EECS

加入 April 2023

826 正在关注 8.9K 粉丝

Woosuk Kwon@woosuk_k

2026.07.16 23:09

KV cache management is arguably one of the hardest problems in LLM inference, and advancing it requires the community to learn from and build on one another’s work. Great to see that happening. Amazing work from @lightseekorg on adopting and evolving vLLM's hybrid memory allocator! The Jenga paper from @ChenZha62999224 is especially relevant today as attention architectures become increasingly complex.

显示更多

488

转发到社区

热门用户