Do agents really need long context? Compacting it may keep accuracy about the same at a much lower cost 💸
"You don't need long context"
Per-token prices are flat, but real serving cost at a 256K window is estimated at 3.6x to 7x that of 16K. This study compares accumulating the full history against compaction schemes that rebuild the conversation, across 13 benchmarks.
Highlights
🔁 Long-horizon tasks survive compaction
Summarizing at a threshold and starting a fresh conversation changed accuracy by -3.7 to +5.6 points for Claude Opus 4.5 and -3.3 to +4.4 for GPT 5.6 Sol, with estimated cost 2.2x to 8.8x lower. On SWE-bench Verified, Opus rose from 70.0% to 74.8%.
📄 Long documents need not be read whole
Keeping documents as files and reading only the needed parts with grep and sed changed accuracy by -0.4 to +40.0 points for Opus and +1.0 to +30.9 for GPT. Estimated savings reach 233.3x, and conversation length stays roughly flat as documents grow.
⚙️ Update state after every action
Single-Action Compaction takes one action from the task and a compact state, updates the state, and discards the conversation. SWE-bench Verified reaches 70.2% (Opus) and 82.4% (GPT), with median peak live context around 77-99K tokens.
Compaction is lossy and the costs are estimates. Among open models, Qwen 3 Coder Next fell from 53.0% to 3.6% in one case.
#
LLMAgents# #
LongContext#