ToolBench

Headlines · 1

1. Oct 15
   Apple study compares plan-first vs retrieve-first tool discovery for LLM agents

   Apple Machine Learning Research published an EMNLP paper by Mingfei Zhang and co-authors comparing two ways tool-using LLM agents find APIs before planning: Retrieve-then-Plan (RtP), which embeds the query and retrieves similar tools, and Plan-then-Retrieve (PtR), in which the model first generates abstract tool stubs that become retrieval queries. On ToolBench's 46,984 API endpoints across 50 categories, RtP beat PtR on single-tool queries while PtR beat RtP on multi-tool queries; the crossover held at retrieval budgets k=1–20, registry sizes from 1k to 47k APIs, and with Gemini 2.5 Flash and gpt-oss-120b. The authors also describe a stub-hallucination failure mode, note that PtR's extra discovery call roughly doubles inference cost, and suggest complexity-adaptive routing using RtP's top-1 retrieval score.

   Apple Machine Learning Research · 🔥 0

Experience and discussion from the community

Share my ToolBench experienceAsk about ToolBench