Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Apple proposes weighted state souping for long-context retrieval

AI summary

Apple Machine Learning Research published a paper introducing weighted state souping, a method that learns query-dependent weights for composing cached state-space-model representations. The authors report substantial long-context gains across document question answering, retrieval, variable tracking, and tool use, and describe a sparse variant with near-constant encoding cost relative to context length. The paper was accepted at the PALM workshop at NeurIPS 2026, while its performance claims have not been independently corroborated.

In-depth insight

The method combines reusable cached context with selective retrieval, targeting the efficiency and reliability problems that long, crowded contexts create for state-space models.

content type paperpublished November 2026

Weighted State Souping for Efficient Long-Context Retrieval

AuthorsWilliam L. Tong†**, Aryo Lotfi, Emmanuel Abbe, Josh Susskind, Cengiz Pehlevan†, Eran Malach

View publication

This paper was accepted at the Workshop on Personalized, Aligned, Long-term Memory for AI Systems (PALM) at NeurIPS 2026.

State space models (SSMs) combine fast inference and fixed state size with competitive language-modeling performance. Recently, state souping has been proposed as a procedure whereby an SSM independently encodes chunks of tokens like documents, tool definitions, or memory records. The resulting state encodings may be cached and recombined to cheaply construct a context for downstream queries. However, both state souping and vanilla SSMs deteriorate on long, crowded contexts, limiting their utility for long-horizon domains. To address this issue, we propose weighted state souping, a simple method that learns query-dependent weights for composing cached states. The resulting models improve long-context performance substantially across both single- and multi-hop document question answering, needle-in-a-haystack retrieval, variable tracking, and tool use. Our method can be further adapted into an efficient sparse variant, yielding near-constant encoding cost with respect to context length while preserving (and sometimes improving) performance. We also present simple theoretical arguments that illustrate why souping works and when it may fail, and discuss informally the advantages of weighted souping. Weighted state souping thus combines reusable context representations with selective retrieval, offering an efficient approach to language modeling over large collections of documents, tools, and other forms of structured context that increasingly dominate modern agentic workloads.

#Apple Machine Learning Research#Weighted State Souping#NeurIPS 2026

Original · 2 min readRead the original
Group chat

No comments yet. Start the conversation.