Research · Nov 4, 2026, 21:08 UTC

Study finds attention to evidence predicts context rot better than context length

Across 126,473 retrieval trials on five open models of 0.5B to 3B parameters, attention on evidence explained 35 to 58% of loss variance, while length explained 2 to 8%. The attention result is predictive, not causal, and covers one synthetic task.

• ModelQwen2.5-1.5B
• Topicresearch