Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

OpenAI deploys activation classifiers on GPT-5.6 completions

AI summary

OpenAI says it began running activation classifiers, also called probes, on all assistant completions from GPT-5.6. The lightweight models read internal residual-stream activations to identify misaligned or otherwise undesirable activity. They support streaming decisions that can stop a policy-violating turn and, according to OpenAI, are more robust to jailbreaks and more efficient than significantly larger text classifiers. The company uses an MLP with QV-attention and an exponential moving average, typically training on up to 1 million synthetic and de-identified messages labeled by its most capable reasoning models. OpenAI says probes must be retrained for each new pretrained base model and may degrade as model weights diverge from the training checkpoint. Performance figures come from internal experiments and may not represent deployed classifiers.

In-depth insight

OpenAI says probes have become an integral part of its safety and alignment strategy and have been deployed for both internal and external traffic on its most capable models. They add white-box monitoring that can intervene while a model is generating a response, complementing text classifiers, model-based review, automated investigation and human response.

Introduction

At OpenAI, we train models to behave in a safe and aligned way. Despite recent progress in this direction [OpenAI, 2026b], there remain cases when adversarial pressure, rare circumstances, or training failures can degrade safety and alignment. For this reason, we adopt a ‘defense in depth’ approach: we deploy models with additional monitors that aim to detect such cases and stop further model action. Specifically, we use text classifiers that run on user prompts and assistant completions for this purpose. Starting from GPT-5.6 [OpenAI, 2026a], we have added another layer of defense to this stack: activation classifiers that run on all assistant completions.

Activation classifiers, or probes for short, are lightweight ML models that use an LLM’s own internal activations — its “neural patterns” — to classify model activity as being misaligned or otherwise undesirable (e.g., user activity having harmful intent). Probes offer multiple advantages compared to external text-based classifiers: they enable streaming decisions which can block a policy-violating model turn midway; are more robust to jailbreaks than significantly bigger text classifiers; and are significantly more efficient to run. Furthermore, probes’ accuracy generally scales in tandem with model capability: probes trained on the activations of more capable models are themselves more capable classifiers.

#OpenAI#GPT-5.6#activation classifiers

Original · 8 min readRead the original
Group chat

No comments yet. Start the conversation.