AI Agent Answers Need Evidence From the Full Execution Chain
A Juejin author describes 16 first-hand experiments showing that a generated tool call, a successful local tool run, or a correct final answer does not by itself prove an AI Agent followed the intended data path. The article recommends inspecting model requests, Harness execution, tool results, next-round context and authoritative state separately, and demonstrates that removing history can break the feedback loop while a runtime guard can block unauthorized actions before side effects occur. It also treats prompt and skill behavior as dependent on what the model actually receives, concluding that Agent evaluation should follow the evidence chain rather than rely on the final answer alone.
The reported experiments provide a practical framework for distinguishing model decisions from Harness, tool and context failures when evaluating AI Agent reliability.
前言
最近我在系统学习 AI Agent。
一开始，我对 Agent 的理解比较简单：
大模型能够调用工具，就算是 Agent。
但真正跑完前 16 节实验以后，我发现事情并没有这么简单。
模型能够生成一个 tool_call，不代表工具真的执行了；工具在本地执行成功，不代表结果已经传给模型；模型最终回答正确，也不代表中间的数据链路是正确的。