Apple's SelfCompact lets models decide when to compact their context
Apple Machine Learning Research published a paper, "Self-Compacting Language Model Agents," proposing SELFCOMPACT, a scaffold in which the model itself invokes a compaction tool to summarize accumulated context, paired with a lightweight rubric specifying when to fire (a sub-task resolved, or the trajectory converging) and when to suppress (mid-derivation, or stuck). The authors report that both elements are needed — the tool alone is used unevenly across open-weight models, while the rubric alone cannot act — and that across six benchmarks (competitive math and agentic search) and seven models, SELFCOMPACT matches or exceeds fixed-interval summarization, beating a no-summarization baseline by up to 18.1 points on math and 5–9 points on agentic search at 30–70% lower per-question cost, with no fine-tuning or external supervision. Some authors carry a Johns Hopkins University affiliation, and the work is listed as a NeurIPS paper.
How agent context is compacted shapes the cost and reliability of long tasks, and this work suggests “when to compact” may be supplied by a scaffold rather than by training.
research area Speech and Natural Language Processingconference NeurIPS
content type paperpublished October 2026
Self-Compacting Language Model Agents
AuthorsTianjian Li†, Jingyu Zhang†, William Jurayj†, Xi Wang†, Chuanyang Jin†, Mehrdad Farajtabar, Eric Nalisnick†, Daniel Khashabi†
Long agent traces composed of chains of thought and tool calls accumulate stale content that anchor subsequent generations, and eventually outgrow the context window. Existing scaffolds mitigate it with fixed-interval compaction triggered at a token threshold. Such triggers pay no heed to trajectory structure, risking discard of partial results mid-derivation or mid-search. We propose SELFCOMPACT, a scaffold that allows the model itself to decide when and how to compact. Specifically, it pairs two inference-time elements: (i) a compaction tool the model invokes to summarize the accumulated context, and (ii) a lightweight rubric specifying when to fire (a sub-task has resolved, or the trajectory is converging) and when to suppress (mid-derivation, or when stuck). Both are needed. The tool alone is unevenly used across open-weight models, often invoked at unhelpful moments or not at all; the rubric alone cannot act. Together, they elicit effective adaptive compaction without any fine-tuning or external supervision. We present empirical results on six benchmarks (competitive math and agentic search) and seven models. Our results show that SELFCOMPACT matches or exceeds fixed-interval summarization at a fraction of the token cost, improving over a no-summarization baseline by up to 18.1 points on math and 5–9 points on agentic search at 30–70% lower per-question cost. Our results expose a meta-cognitive gap: although unprompted models cannot reliably tell when their own context is rotting, a lightweight rubric closes this gap, reframing when to compact as a capability that scaffolds can supply without training.
#Apple#Apple Machine Learning Research#Johns Hopkins University#SelfCompact