研究称检索退化源于注意力竞争
AI 总结
Yinuo Liu提交arXiv的论文称,在Qwen2.5-1.5B上,中性填充文本使32K token内的检索准确率维持在0.97至1.00,而同等位置的竞争性陈述将准确率降至0.11至0.21。126,473次试验中,证据注意力降低最多可解释58%的损失方差,对数长度最多只能解释8%,但作者指出这一关系是预测性的,而非因果关系。
深度见解
结果表明,检索系统退化可能更多源于竞争性证据而非上下文长度本身,因此注意力质量或可成为长上下文故障的诊断指标。
Computer Science > Computation and Language
arXiv:2611.04169 (cs)
[Submitted on 3 Nov 2026]
Title:Length Is Cheap, Competition Is Not: Attention Predicts Context Rot in Retrieval
Authors:Yinuo Liu
View a PDF of the paper titled Length Is Cheap, Competition Is Not: Attention Predicts Context Rot in Retrieval, by Yinuo Liu