Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Apple’s SupportNet learns faster routing for MIPS retrieval

AI summary

In a paper presented at ICML 2026, Apple researchers described SupportNet, small neural networks trained offline for a fixed key database to route queries toward their best-matching keys. On Apple’s Natural Questions setup, partitioned into 128 clusters, the small model reached 95% routing recall in 1.74 milliseconds, compared with 3.36 milliseconds for centroid routing; the large model reached 97% recall in 1.74 milliseconds. These figures come from Apple’s reported evaluation.

In-depth insight

The results show that learned routing can reduce query-routing latency compared with centroid routing when the query distribution is known. SupportNet shifts part of the cost to upfront offline training in exchange for fast, lightweight inference.

content type highlightpublished November 5, 2026

Faster Retrieval for Better, More Efficient RAG

Maximum inner product search (MIPS) sits at the core of recommendation systems, search, and retrieval-augmented generation (RAG). Given a query vector, MIPS systems search within a database of millions of candidate key vectors for the one that best matches the query. The original data point associated with that best-matching key is then used to handle the query: in RAG, the document that is best suited to answer the question is typically prefixed to the query.

As the number of keys grows to millions and vector dimensions reach thousands, the computational cost of running an exhaustive comparison of the query to all keys grows dramatically. Numerous approximate methods have been proposed to trade off accuracy for speed using clustering and quantization. While these baseline methods focus on the properties of the set of keys to build up clever indexing structures, their geometric arguments loosely assume that the set of queries itself will resemble that found in keys, or bypass any considerations about what the queries will look like. This can be problematic on two fronts: In the crucial RAG case, queries arise from short user-generated questions, whereas keys represent long, informative, and curated pieces of text, making keys and queries substantially different. Additionally, most MIPS use cases deployed in production target an existing base of users with a stable, well-characterized query distribution — a signal that generic, query-agnostic methods leave untapped. Crafting methods that work best for a likely source of inputs — rather than aiming for good performance on all possible inputs — is a central tenet of modern machine learning that helps redirect compute resources more efficiently.

#Apple#SupportNet#Natural Questions

Original · 12 min readRead the original
Group chat

No comments yet. Start the conversation.