REMORY用软记忆令牌压缩智能体上下文
AI 总结
REMORY论文提出一种神经记忆网络，在文本摘要后追加有界的软记忆令牌，以帮助冻结的LLM逼近基于完整历史生成后续内容的效果。作者称，在SummHay上，该方法仅使用5.2%的输入位置便接近完整上下文的联合得分，并在Qwen3.8-27B和GLM-5.3-Flash的长程智能体测试中减少重复工具输出与工具错误。
深度见解
该方法直接针对长程智能体在有限上下文窗口内保留关键信息与来源归属的问题，并给出了论文作者报告的基准结果。
| Long-horizon agents compact their history to continue within a finite context window, but a textual summary alone may not support every subsequent decision. We introduce REMORY, a neural memory network that supplements the summary with a bounded sequence of soft memory tokens. Given the history and summary, the network learns to generate tokens that help a frozen LLM approximate the continuation it would produce with the full history. The tokens are conditioned on the summary and appended after it, forming an analogue of a residual connection along the sequence dimension. On SummHay, REMORY improves source attribution at nearly unchanged insight coverage and approaches the full-context joint score using only 5.2% of the input positions. Across long-horizon agent benchmarks, Qwen3.8-27B and GLM-5.3-Flash show consistent gains with residual memory. Both models also exhibit substantially fewer repeated tool outputs and tool errors on BrowseComp and Terminal-Bench 2.1. submitted by /u/pmttyji [link] [comments] |
#REMORY#Qwen3.8-27B#GLM-5.3-Flash#SummHay#BrowseComp#Terminal-Bench 2.1