Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

BM25 Pipeline for Agent Tool Discovery

AI summary

An article presents a model-agnostic, pure-Python pipeline that retrieves tools with BM25, applies hard risk filtering, recursively completes declared dependencies, and exposes only the resulting candidate schemas to the model. It distinguishes model-visible tools from the full executable registry and says the execution layer should still validate permissions, parameters, approvals, idempotency keys, and the allowed tool set. The article recommends evaluating tool recall, Completeness@K, risk exposure, schema token usage, tool-selection accuracy, and end-to-end success. It cites Anthropic documentation for delayed tool loading and reports that tool-selection ability may decline when a catalog exceeds roughly 30–50 tools; that threshold was not independently verified here. The article also cites a multi-tool retrieval study and its AppWorld results at secondhand, and its demonstrated example does not establish production recall or task-success rates.

In-depth insight

The pipeline provides an implementable way to limit model context while preserving declared prerequisite tools and enforcing risk boundaries. Its separation of model-visible and executable tools also supports a layered approach in which retrieval reduces exposure, while the execution layer performs final authorization checks.

一个 Agent 只有三五个工具时,最简单的做法是把所有工具名称、描述和参数 Schema 一次性交给模型。

接入更多 MCP Server 后,问题会变得明显。GitHub、Slack、日历、邮件、数据库和监控平台可能带来几十甚至几百个工具。每次模型调用都携带全部定义,不但消耗 token,相似工具还会相互干扰。

Anthropic 的 Tool Search 文档给出了一个多服务器示例:GitHub、Slack、Sentry、Grafana 和 Splunk 的工具定义合计约 5.5 万 token;其文档还指出,工具数量超过 30—50 个后,工具选择能力会下降。官方方案是延迟加载工具,只在需要时搜索并展开少量定义。Anthropic Tool Search 文档

这篇文章不绑定特定模型 API,而是用纯 Python 实现一个本地版本:

用户请求
   ↓
BM25 工具检索
   ↓
风险等级过滤
   ↓
依赖工具递归补全
   ↓
只把候选工具 Schema 交给模型
   ↓
执行层校验并调用真实工具

#Anthropic#Anthropic Tool Search

Original · 9 min readRead the original
Group chat

No comments yet. Start the conversation.