研究发现安全评估与生产表现存在差距
一项关于智能体安全评估的研究发现,一些模型在 Respecting Warnings 评估中的表现明显差于处理可比生产决策时的表现,而经常说出错误评估目标的模型差距最大。知道自己正在被评估并不一定会改变行为,但识别具体评估目标与这一差距密切相关,且目标框架会因果影响评估成功率。该研究通过模型自发陈述的推理内容衡量评估意识,这只能捕捉模型表达出的假设,而不能涵盖其所有潜在信念。
报告指出,安全评估分数可能对模型关于测试目标的假设十分敏感,因此推断评估者偏好的行为可能提高评估表现,却不代表现实场景中的行为能力得到改善。研究重点强调持续追踪评估与生产之间的差距,并设计更难从环境中推断目标的评估。
Introduction
Evaluation awareness has been widely observed across a range of models and evaluation settings [1-3]. Models often verbally recognize that a particular interaction is an evaluation, speculate about the test objective, and occasionally alter their behavior in response [4]. This creates a real challenge for interpreting and trusting evaluation results. How much of what we observe reflects behavior that will generalize to deployment? How much does this behavior change depend on a model's interpretation of the task?
In this post, we focus on agent safety evaluations, across long horizon tasks where agents must decide how to respond when they come across potentially safety-critical triggers (e.g., warnings, restrictions, or obstacles). Minimizing the eval-production gap here is crucial, as an informative safety evaluation needs to measure whether the agent makes the same kinds of decisions in realistic use.
To assess how well these evaluations reflect agent behavior in the real world, we compare success rates in evaluation and production, and investigate whether models' interpretations of the evaluation setting help explain the gap. Our results suggest that:
(1) The eval-production behavior gap exists and points in an unexpected direction: some models appear to perform substantially worse on safety evaluations than in production, with the largest gaps observed in models that most frequently verbalize incorrect evaluation objectives.
#OpenAI Alignment#Respecting Warnings#Codex#GPT-5.6 Luna#6.1 Sol