Woosuk Kwon(@woosuk_k):KV cache management is arguably one of the hardest problems in LLM inference, and advancing it requires the community to learn from and build on one another’s work. Great to see that happening. Amazing work from @lightseekorg on adopting and evolving vLLM's hybrid memory allocator! The Jenga paper from @ChenZha62999224 is especially relevant today as attention architectures become increasingly complex.

Woosuk Kwon

@woosuk_k

@inferact | @vllm_project | prev: PhD @Berkeley_EECS

参加 April 2023

826 フォロー中 8.9K ファン

Woosuk Kwon@woosuk_k

2026.07.16 23:09

KV cache management is arguably one of the hardest problems in LLM inference, and advancing it requires the community to learn from and build on one another’s work. Great to see that happening. Amazing work from @lightseekorg on adopting and evolving vLLM's hybrid memory allocator! The Jenga paper from @ChenZha62999224 is especially relevant today as attention architectures become increasingly complex.

488

コミュニティへ転送

人気のあるユーザー