LangSmith Trace Exposes Agent Routing and Context Failures
In a hands-on trace published by 掘金 · 人工智能, a wealth-advisor agent routed a portfolio-allocation question without passing the customer profile to either the assessment or reactive prompt. Although the same state contained a 65% stock allocation, the agent said it could not access portfolio data, while the trace showed 4.38 seconds spent on assessment versus 1.06 seconds on the response. The author also found that hidden spans accounted for overhead beyond the model call, emergency labeling did not create a separate route, and fallback routing was indistinguishable from a normal reactive decision.
The trace demonstrates how missing prompt context, misleading routing labels, and ambiguous fallback behavior can produce a technically successful but user-unhelpful AI agent execution.
我问了它一句:我的投资组合中科技股占比是多少?
选的是进取型那个客户。300 万,股票 65%、债券 15%、现金 5%、另类 15%。
跑完终端打印"处理用时 6.54 秒"。我当时的反应是:还行吧,六秒多,能接受。
然后我打开 LangSmith,点了下这条 trace。
6.43 秒。比终端那个数少 0.11 秒,是本地 print 的开销。
两个数差不多。但拆开看是另一回事。