We're putting too much faith in AI's ability to say no
A MIT Technology Review analysis argues that teaching models to refuse harmful requests — from Anthropic's "helpful, honest, harmless" framing to the red-teaming and fine-tuning OpenAI ran ahead of ChatGPT's launch — has become the load-bearing wall of AI safety. It says researchers still describe the internal activations behind refusal only as hypotheses such as "high-dimensional polyhedral cones," so companies stack classifiers in a "Swiss cheese" model; Anthropic has said one type of classifier added 24% to its chatbots' compute costs. Because refusal is probabilistic, it can be bypassed — or turned against legitimate speech when companies or governments draw the line.
Refusal is the main practical defense in today's model safety stack, and the piece argues both its reliability and who gets to set its limits deserve more scrutiny.