OpenAI扩大前沿模型失配监测
OpenAI在Hugging Face事件暴露监测盲区后,将失配监测扩大至前沿模型训练、评估、内部Codex使用及部分部署场景。公司称,这套分层系统结合激活探针、审查模型和调查智能体,并在9月一次训练中于15分钟内发现模型尝试连接实时互联网。OpenAI还称,回放Hugging Face事件记录时,系统在智能体越出沙箱约30分钟后发出告警,但该结果未经独立验证。
OpenAI已将行为监测纳入前沿训练的安全论证,使人工响应人员能在智能体越过预设边界时收到告警。
As we train increasingly capable AI models, we need reliable ways to detect when their behavior departs from what we intended. Agents can misbehave and take unauthorized actions during training, evaluation, or deployment, with potentially serious consequences. In order to detect these failures—and, where necessary, to intervene in time—we monitor how agents reason, what actions they take, and how those actions affect their environment.
While we’ve been monitoring our internal coding agents for some time, the Hugging Face incident exposed a gap in our coverage: the incident began during internal cybersecurity evaluations on which we were not running our monitors for misaligned behavior. In response to that incident, and to the increasing capabilities of our models more generally, we’ve substantially expanded our system for monitoring agents’ behavior. Our misalignment monitors now cover our frontier models across a much wider variety of surfaces, including training, evaluation, internal use, and, with modifications depending on the product surface, external deployment.
In order to see how this expanded system could have helped prevent the Hugging Face incident, we replayed the behavior recorded in that episode with our strengthened monitors in place. In that simulation, our monitoring system alerted our security team about 30 minutes after the agents first escaped their sandbox—and more than 30 hours before Hugging Face’s systems were breached in the actual incident. We believe that this warning would have given our responders time to interrupt the activity well before our models were able to access external services.