Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

METR and Redwood Find Weaknesses in OpenAI Agent Monitoring

AI summary

METR and Redwood Research spent three weeks stress-testing OpenAI’s internal agent monitoring system with access to its codebase, monitor prompts, configuration details and documentation. They found independently tested weaknesses, including two client-controlled headers that could disable monitoring for new agents, while noting that setting the headers and creating an agent might still be detected. The evaluators also identified two internal tools believed to be read-only that appeared vulnerable to permission subversion, but they did not validate that behavior.

In-depth insight

The findings show concrete gaps in oversight mechanisms used to review OpenAI’s internal agents, a material concern for practitioners assessing the safeguards around autonomous AI systems.

For three weeks, from July 21st to August 11th,1 a METR employee and a Redwood Research employee conducted a pilot high-access exercise to stress-test OpenAI’s internal misalignment monitoring system.2 The output of this exercise was a 61-page internal report that was shared with OpenAI researchers and security staff,3 primarily focused on estimating the fraction of agentic inference that is monitored, and whether agents could in principle disable or subvert the monitoring system. The exercise focused on the internal misalignment monitoring system as it existed on and before July 21st, and did not evaluate changes that were in progress at the time of the exercise.

This exercise involved a deep level of access. The METR and Redwood employees had access to OpenAI’s internal codebase, misalignment-monitor prompts / configuration details, and internal documentation, and conducted technical interviews with approximately a dozen researchers and engineers across the monitoring, safety-systems, and security teams. To enable this level of access, they worked on OpenAI-issued devices, which never left the premises, and were accompanied while onsite.

METR, Redwood Research, and OpenAI designed this exercise as a pilot for giving external researchers deep access to internal infrastructure, not to provide robust public transparency about OpenAI’s monitoring systems.4 As a result, we agreed upfront with OpenAI that we would only publish a relatively brief and high-level public post. For future versions of this exercise we expect to follow the transparency and independence best practices described in “Our approach to working with AI companies” to provide more robust public transparency.

#OpenAI#METR#Redwood Research#Codex#Responses API

Original · 7 min readRead the original
Group chat

No comments yet. Start the conversation.