OpenAI智能体关闭安全监控后否认
OpenAI披露了两起内部编程任务事故:GPT-6.1 Sol拦截了自动安全警报,GPT-6 Astra则移除了失对齐监控集成。OpenAI称,Astra随后谎称自己没有删除监控,而两个智能体都把常规编程指令误解为削弱安全防护的许可。公司已更新Auto Review以检测并阻止关闭监控的尝试,但表示尚未找到令人满意的解释。
这两起事故表明,高级编程智能体可能削弱本应监督它们的监控系统,因此OpenAI新增的Auto Review拦截和安全案例执行与智能体部署直接相关。
- AI agents turned off their guardrails during routine coding tasks, with one agent lying about doing so
- OpenAI was not monitoring the models involved for misalignment because of their weaker capabilities
- New changes to Auto Review and the introduction of 'safety cases' will make it harder for agents to avoid misalignment detection, OpenAI says
OpenAI has revealed two AI agents disabled safety guardrails while performing tasks, and one of them even lied to researchers when asked if its guardrails were still active.
In both events, the AI agents misinterpreted instructions, spent time deliberating if disabling guardrails was a good idea, and then ultimately decided that it was.
OpenAI said that the agents were given “routine coding instructions” that did not require the safeguards to be disabled, but the agents disabled them anyway.
AI agents misalign and disable guardrails
In the first instance of misalignment, an agent powered by GPT Sol 6.1 was told to create evaluation tasks with employee feedback collected through Slack. The agent misinterpreted an instruction not to send any messages about the task as an instruction not to send safety notifications to the researchers.