If search-agent accuracy has plateaued, the next thing to cut is context tokens.
Context as Infrastructure: Knowledge Compilation for Retrieval in Agentic Systems with Harnesses
A harness using only BM25 and grep reportedly reaches 91% on FRAMES. What's left is wasted tokens from rereading the same documents. The paper proposes "knowledge compilation": adding context offline with an LLM.
🧾 Highlight 1: claim-evidence pairs
Each unit is a normalized claim, a verbatim evidence span, and a source offset. Claims without a locatable span are not stored. When retrieval finds a chunk, the chunk is replaced by these pairs before it enters the context window.
📉 Highlight 2: structure didn't help, substitution did
Summary trees, outlines, wikis and graphs showed no detectable accuracy gain. Claim-evidence substitution cut query tokens by 38% on average on FRAMES (N=200) with accuracy within noise. The cost is more retrieval rounds and higher latency, which a decision gate helps contain.
🧮 Highlight 3: a payback formula
With gate acceptance rate p_h, break-even is about 2%. The measured average implies roughly 43%, so a compile step costing 10 queries pays back after about 26 queries.
Caveats: one benchmark, single runs, no confidence intervals, and the false-accept audit hasn't been run. Treating context as infrastructure feels practical to me.
#
RAG# #
AgenticAI#