Great paper from UC San Diego on why long-context retrieval fails.

(bookmark it)

It also offers an interesting discussion of why context rot happens in LLMs.

When a model fails on a long input, people usually blame the length. This paper finds that length alone does very little. Adding 32K tokens of unrelated text left accuracy at 97%-100% on Qwen2.5-1.5B.

What hurts is text that looks like the answer. When the context also holds similar statements with wrong values, accuracy falls to between 16% and 33%. A single statement that gives a different value for the same question is enough to cause a large drop.

The model fails when those statements pull its attention away from the correct passage. Across 126,473 trials, the amount of attention left on the correct passage explains up to 58% of the variation in errors. Input length explains at most 8%.

For RAG and agent memory, dedupe and resolve conflicting statements before they reach the model. That matters more than trimming tokens.

https://arxiv.org/abs/2611.04169

Chat with Paper: https://academy.dair.ai/papers/length-is-cheap-competition-is-not-attention-predicts-context-rot-in-retrieval-2611.04169