Production AI failures demand observability and agent governance
InfoQ reported on a practitioner webinar about failures that can emerge when AI systems reach production, including semantic drift, embedding drift, sensitive-data exposure and errors that bypass conventional tracing. Participants described a financial compliance agent whose false-positive and false-negative rates rose after an embedding-model update while infrastructure health checks remained normal. They recommended continuous online evaluation, automated style and syntax checks, mutation testing, red-team testing, threat modeling, strict agent permissions and production approval as a strategic business decision.
The reported cases show why conventional infrastructure monitoring may miss production AI failures and why deployment requires application-level evaluation, observability and security controls alongside business approval.
在 InfoQ 近期举办的一场题为《生产环境中的 AI：哪些会失效、哪些能正常运行，以及由谁审批上线？》"的网络研讨会上，一个由资深工程实践者组成的小组深入剖析了部署智能体系统过程中遇到的各类现实难题。本次研讨会由 Hien Luu" 主持，嘉宾包括来自 ThoughtWorks 的 Zichuan Xiong" 和 Prem Chandrasekaran"、隐私领域专家 Katharine Jarmul" 以及金融 AI 从业者 Lan Chu"。
Luu 在开场时指出了三个核心驱动因素：AI 的失效模式与传统软件不同，它会生成看似合理但不准确的结果，而不是直接崩溃；编码智能体生成生产代码的速度已经超过了人类的代码评审速度；敏感数据有了新的泄露途径，包括提示词、上下文窗口以及检索到的文档。