TL;DR If the same document gets searched over and over, why re-encode it from scratch for every query? DoPR compresses the document side once, stores it, and reuses it across many queries to speed up LLM reranking. Title: DoPR: Reusable Compressed Document Prefixes for Efficient LLM Reranking URL: 🔑 An attention-based salience score picks the top-K most attended tokens, and each document is compressed once offline into a query-independent prefix of KV states ⚡ Online reranking only has to process the query and score tokens, cutting memory by up to 8.0x and latency by up to 8.04x on long documents 📊 On TREC DL19/DL20 and BEIR, DoPR keeps 97.1-99.5% of NDCG@10, and larger backbones turn out to be more resilient to the compression bottleneck 🔁 In a controlled cross-query reuse test, DoPR hits roughly a 2x speedup at 10 queries per document on DL19, and over 3x at just 5 queries on the long-document TREC-Covid set 🧪 Ablations show attention-guided top-K selection beats first-K, uniform-K, and random-K selection (71.44 NDCG@10) — and it needs zero extra parameters to pull that off 💬 The more a search system relies on a stable, repeatedly-queried document collection, the more this "compute once, reuse many times" idea should pay off #Search# #LLM#