Budget-aware allocation improves deep retrieval for batch RAG
A paper by Junru Zhu, Yixin Yang, Xiaoqing Ding and Ruoyu Qi frames deeper retrieval as a resource-allocation problem for 150 FinanceBench questions sharing one token budget. In the tightest nonzero 10% budget, ranking upgrades by predicted benefit per token achieved quality of 0.524 versus 0.490 for gain-only allocation, recovered seven more deep-retrieval opportunities, and used 46.6% fewer tokens than upgrading every query. The evaluation found cost prediction much stronger than benefit prediction, with R² of 0.93 and -0.08 respectively.
The reported results offer a concrete allocation rule for batch RAG under constrained token budgets, while identifying benefit prediction as the main remaining bottleneck.
💡 一句话总结:批量 RAG 有个很少有人正式处理的问题:150 个查询共享一份 token 预算,而「给某个查询上更深检索」的额外成本能差十倍。这篇论文把升级决策做成了资源分配题,结论精确而克制:成本可以预测得很准(R²=0.93),但「升级有没有用」几乎猜不中(AUROC 0.60)。所以在紧预算下,按「收益 ÷ 每 token」排序比按收益排序多救回 7 个升级机会、质量 +0.034、总 token 省 46.6%——而想在整个预算曲面上全面赢,得先把收益预测本身做出来。
导语:被忽略的「批」维度
RAG 系统的检索深度是个钱袋问题:检索 top-1 页,每题约 1091 token;top-5 页,2508 token,准确率从 46.7% 涨到 54.7%。单看一个查询,deep retrieval 的选择不难。但真实业务是批的:一个财报问答服务每天几万个查询,预算是死的——不可能人人上 top-5。
这时候问题就变成了分配:预算该发给谁? 已有的自适应 RAG(Adaptive-RAG、Self-RAG、FLARE 这些)都在回答「这个查询自己该怎么检索」,而批部署多了一层约束:查询之间在抢同一个钱袋,而且升级的标价因题而异——有的题 top-5 上下文 588 token 就装下,有的要 5498。