Register and share your invite link to earn from video plays and referrals.

cv usk
@cv_usk
AI / Software Research Notes AI Agent, LLMOps, MLOps, Software Architecture ๆŠ•็จฟใฏๅ€‹ไบบใฎๆ„่ฆ‹ใงใ™ใ€‚
Joined May 2026
287 Following    476 Followers
TL;DR: Storing user profiles as reusable KV cache instead of text made an LLM recommender's scoring path about 4x faster with ranking quality essentially unchanged. KVUse: Reusable-KV User Memory for Efficient LLM Recommenders Key points ใƒป๐Ÿง  Targets memory-augmented recommenders that re-prefill each user's natural-language profile over and over ใƒป๐Ÿšซ Prefix caching can't help, since different instructions precede the profile in each call ใƒปโœ๏ธ LoRA writers (about 6.8M params) on Llama-3.1-8B also emit KV while generating the profile. The reader is the unmodified LLM ใƒป๐ŸŽ“ Self-distillation from full prefill, with positions fixed by RoPE re-rotation ใƒป๐Ÿš€ Scoring 3.96 to 4.49x (Books: 31.99 to 126.72 users/s), update path 1.43 to 1.91x ใƒปโœ… NDCG@10 change of โˆ’0.001 to +0.008, not significant. Naive KV reuse dropped 26 to 32% Caveats: KV is about 128 KiB per token, and writers need per-domain training. #LLMInference# #RecSys#
Show more