Apple’s SupportNet learns faster routing for MIPS retrieval
In a paper presented at ICML 2026, Apple researchers described SupportNet, small neural networks trained offline for a fixed key database to route queries toward their best-matching keys. On Apple’s Natural Questions setup, partitioned into 128 clusters, the small model reached 95% routing recall in 1.74 milliseconds, compared with 3.36 milliseconds for centroid routing; the large model reached 97% recall in 1.74 milliseconds. These figures come from Apple’s reported evaluation.
The results show that learned routing can reduce query-routing latency compared with centroid routing when the query distribution is known. SupportNet shifts part of the cost to upfront offline training in exchange for fast, lightweight inference.
content type highlightpublished November 5, 2026
Faster Retrieval for Better, More Efficient RAG
Maximum inner product search (MIPS) sits at the core of recommendation systems, search, and retrieval-augmented generation (RAG). Given a query vector, MIPS systems search within a database of millions of candidate key vectors for the one that best matches the query. The original data point associated with that best-matching key is then used to handle the query: in RAG, the document that is best suited to answer the question is typically prefixed to the query.
As the number of keys grows to millions and vector dimensions reach thousands, the computational cost of running an exhaustive comparison of the query to all keys grows dramatically. Numerous approximate methods have been proposed to trade off accuracy for speed using clustering and quantization. While these baseline methods focus on the properties of the set of keys to build up clever indexing structures, their geometric arguments loosely assume that the set of queries itself will resemble that found in keys, or bypass any considerations about what the queries will look like. This can be problematic on two fronts: In the crucial RAG case, queries arise from short user-generated questions, whereas keys represent long, informative, and curated pieces of text, making keys and queries substantially different. Additionally, most MIPS use cases deployed in production target an existing base of users with a stable, well-characterized query distribution — a signal that generic, query-agnostic methods leave untapped. Crafting methods that work best for a likely source of inputs — rather than aiming for good performance on all possible inputs — is a central tenet of modern machine learning that helps redirect compute resources more efficiently.