Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Four barriers to reliable AI tool calling

AI summary

An analysis of AI agents identifies four barriers to reliable tool use: security risks from malicious parameters or manipulated error messages, protocol costs at scale, selection errors when many tools are available, and weak failure recovery. It recommends structural safeguards such as strict parameter contracts, typed inputs, retrieval-based tool discovery, and dependency-aware orchestration with alternate paths. It also argues that evaluation should cover tool-use gating, selection, parameter accuracy, orchestration validity and recovery rather than treating a successful call as a successful workflow.

In-depth insight

The analysis distinguishes syntactically successful tool calls from reliable end-to-end operation. Its framework helps teams assess security, scalability, tool selection and recovery when evaluating agent workflows.

很多人第一次听说“工具调用”(tool calling / function calling),会以为这事很简单:不就是让 AI 输出一段格式正确的指令,去调个接口吗?

但真正做过线上 Agent 的人都知道,工具调用出问题,几乎从来不是因为“AI 选错了工具”。它更常见的死法是:参数里藏了个恶意路径、工具返回的错误信息被人做了手脚、工具一多 AI 就开始犯迷糊、还有协议本身给大规模部署埋了坑。

所以这里想说的是:工具调用的难点,不在于“AI 会不会调”,而在于你在 AI 和外部世界之间,立了几道拦得住东西的墙。

第一道坎:安全——别指望 AI“识破坏人”,要让坏人“干不了坏事”

关于工具调用的安全,大家第一反应是“检测提示词注入”,也就是识别出恶意指令然后拒绝。但最近的研究给这个思路泼了冷水。

有个团队测了 3250 个攻击场景、七个模型,结论是:用 Function Calling 架构时,攻击成功率高达 73.5%;MCP 架构也有 62.6%。更扎心的是,多步连环攻击的成功率到了 91%–96% ,而且模型越聪明,反而越容易被利用——因为检测能力变强,没转化成安全性变强。

#ToolClad#DADL#Model Context Protocol (MCP)#Function Calling

Original · 7 min readRead the original
Group chat

No comments yet. Start the conversation.