DeepSeek’s Weak Spots Can Hide Behind Average Scores Move the same information a few tokens, and retrieval accuracy can change sharply. In ByteDance Seed’s tests, DeepSeek-V4-Flash-Base showed a 40.2-percentage-point gap between its best and worst positions. Zhihu contributor Kitt在进化 explains why this is a reliability problem that average scores can miss. Why position changes what survives DeepSeek V4 alternates between two attention designs: CSA compresses overlapping 8-token windows at a stride of 4, then selects relevant compressed entries through an indexer. HCA compresses each non-overlapping block of 128 tokens into one entry, then attends to all historical compressed entries. Shifting information changes where it falls within these compression windows, affecting how well it is preserved and retrieved. The researchers tested retrieval from 16,000 key-value pairs in a 128K-token context. They varied alignment with compression windows while keeping the question’s position fixed and matching the target-position distributions’ mean and variance. Similar averages, very different weak spots In a controlled experiment using the Qwen3-0.6B architecture, full attention and 8× compressed attention achieved similar average retrieval scores: 61.1 versus 59.4, both with eight KV heads. But their best-to-worst gaps were 4.3 versus 71.1 points. Comparable averages concealed dramatically different sensitivity to position. V4.1 improves, but the explanation remains tentative The gap narrowed from 40.2 points in V4-Flash-Base to 19.1 in post-trained V4-Flash-0731, and to 6.1 in V4.1-Flash. The author suggests two possible contributors: Engram reducing the burden on attention, and CSA2 lowering the compression ratio to 2. These are hypotheses about the improvement, not experimentally isolated causes.