activation classifiers
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- OpenAI deploys activation classifiers on GPT-5.6 completions
OpenAI says it began running activation classifiers, also called probes, on all assistant completions from GPT-5.6. The lightweight models read internal residual-stream activations to identify misaligned or otherwise undesirable activity. They support streaming decisions that can stop a policy-violating turn and, according to OpenAI, are more robust to jailbreaks and more efficient than significantly larger text classifiers. The company uses an MLP with QV-attention and an exponential moving average, typically training on up to 1 million synthetic and de-identified messages labeled by its most capable reasoning models. OpenAI says probes must be retrained for each new pretrained base model and may degrade as model weights diverge from the training checkpoint. Performance figures come from internal experiments and may not represent deployed classifiers.
OpenAI Alignment · 🔥 0
Experience and discussion from the community
Share my activation classifiers experienceAsk about activation classifiers
Nobody has shared their experience with activation classifiers yet.