Retail LLM platform cut agent hallucinations from 15% to 1.5% in six months
An InfoQ China engineering playbook describes a large retailer's LLM-based inventory recommendation system, where about 15% of agent responses were hallucinations at the end of the first production month; six months later the rate had fallen to 1.5%. The authors say the drop came not from a stronger base model but from treating LLM capabilities as platform infrastructure: shared retry and failure classification, a prompt registry with versioning and rollback, schema enforcement at the platform boundary, per-request token cost attribution, behavioral observability and tool routing. The platform now serves dozens of application teams across inventory audit, replenishment review and discrepancy triage, and is built on Google's ADK orchestrator, LiteLLM for routing to models such as Gemini and GPT, and MCP-based tool services, with a companion codebase that runs locally on llama3.1 via Ollama to avoid API costs.
The article's central claim is that controlling production hallucinations depended on platform-level contracts for validation, prompt versioning, observability and cost attribution rather than on a stronger base model — a directly transferable lesson for enterprises sharing LLM capabilities across teams.
在我们将基于 LLM 的库存推荐系统投入生产运行的第一个月末，约 15% 的智能体返回结果是幻觉——听起来言之凿凿，但内容毫无事实依据。
六个月后，幻觉率降到了 1.5%。推动这一数字下降的关键，并非换用更强大的基础模型，而是我们做出的一个决定：不再将大模型相关能力视作应用层的业务问题，而是把它当作平台基础设施来对待。
本文所描述的项目，是一家大型零售企业的库存准确性平台。系统库存记录会因错放、损坏、盗窃和计数错误而与货架实际商品产生偏差，这种偏差会影响下单、补货以及商品供给。
我们的多智能体大模型系统对数百万 SKU、数十亿条历史库存记录中的异常信号进行分析，为业务方生成库存修正建议，助力业务始终保持库存数据准确。
#Gemini#GPT#Google ADK#LiteLLM#Model Context Protocol#FastMCP#Ollama#llama3.1#Pydantic