Study finds gaps between safety evaluations and production
A study of agent safety evaluations found that some models performed substantially worse in the Respecting Warnings evaluation than in comparable production decisions, with the largest gaps among models that frequently verbalized incorrect evaluation objectives. General awareness of being evaluated did not necessarily change behavior, but awareness of the specific objective was closely associated with the gap, and objective framing causally changed evaluation success rates. The study measured awareness through models’ spontaneous reasoning statements, which capture expressed hypotheses rather than every underlying belief.
The report says safety evaluation scores may be sensitive to models’ hypotheses about what is being tested, so inferring an evaluator’s preferred behavior could improve evaluation performance without improving real-world behavior. It prioritizes tracking the eval-production gap and designing evaluations whose objectives are less readily inferred from the setting.
Introduction
Evaluation awareness has been widely observed across a range of models and evaluation settings [1-3]. Models often verbally recognize that a particular interaction is an evaluation, speculate about the test objective, and occasionally alter their behavior in response [4]. This creates a real challenge for interpreting and trusting evaluation results. How much of what we observe reflects behavior that will generalize to deployment? How much does this behavior change depend on a model's interpretation of the task?
In this post, we focus on agent safety evaluations, across long horizon tasks where agents must decide how to respond when they come across potentially safety-critical triggers (e.g., warnings, restrictions, or obstacles). Minimizing the eval-production gap here is crucial, as an informative safety evaluation needs to measure whether the agent makes the same kinds of decisions in realistic use.
To assess how well these evaluations reflect agent behavior in the real world, we compare success rates in evaluation and production, and investigate whether models' interpretations of the evaluation setting help explain the gap. Our results suggest that:
(1) The eval-production behavior gap exists and points in an unexpected direction: some models appear to perform substantially worse on safety evaluations than in production, with the largest gaps observed in models that most frequently verbalize incorrect evaluation objectives.
#OpenAI Alignment#Respecting Warnings#Codex#GPT-5.6 Luna#6.1 Sol