REMORY Uses Soft Memory Tokens for Agent Context Compaction
AI summary
The REMORY paper introduces a neural memory network that appends bounded soft memory tokens after a textual summary, helping a frozen LLM approximate continuation from the full history. Its authors report that the approach approaches the full-context joint score on SummHay using 5.2% of input positions, while Qwen3.8-27B and GLM-5.3-Flash show fewer repeated tool outputs and tool errors on long-horizon agent benchmarks.
In-depth insight
REMORY directly addresses preserving key information and source attribution when long-horizon agents operate within finite context windows, with benchmark results reported by the paper's authors.
| Long-horizon agents compact their history to continue within a finite context window, but a textual summary alone may not support every subsequent decision. We introduce REMORY, a neural memory network that supplements the summary with a bounded sequence of soft memory tokens. Given the history and summary, the network learns to generate tokens that help a frozen LLM approximate the continuation it would produce with the full history. The tokens are conditioned on the summary and appended after it, forming an analogue of a residual connection along the sequence dimension. On SummHay, REMORY improves source attribution at nearly unchanged insight coverage and approaches the full-context joint score using only 5.2% of the input positions. Across long-horizon agent benchmarks, Qwen3.8-27B and GLM-5.3-Flash show consistent gains with residual memory. Both models also exhibit substantially fewer repeated tool outputs and tool errors on BrowseComp and Terminal-Bench 2.1. submitted by /u/pmttyji [link] [comments] |
#REMORY#Qwen3.8-27B#GLM-5.3-Flash#SummHay#BrowseComp#Terminal-Bench 2.1