alphaXiv(@askalphaxiv):OCR but with a working memory...?! This paper, Unlimited OCR, replaces decoder full self-attention with Reference Sliding Window Attention, giving the model a working memory where each token attends to the fixed visual and prompt references plus only the most recent output tokens, making decode KV cache constant instead of linear in generation length. Combined with DeepEncoder’s 16x visual compression, this working-memory-style attention enables one-shot multi-page OCR with stable memory and latency, while improving OmniDocBench performance over DeepSeek OCR.

2026.06.23 17:17

OCR but with a working memory...?! This paper, Unlimited OCR, replaces decoder full self-attention with Reference Sliding Window Attention, giving the model a working memory where each token attends to the fixed visual and prompt references plus only the most recent output tokens, making decode KV cache constant instead of linear in generation length. Combined with DeepEncoder’s 16x visual compression, this working-memory-style attention enables one-shot multi-page OCR with stable memory and latency, while improving OmniDocBench performance over DeepSeek OCR.