We put too much faith in AI saying no, MIT Tech Review argues
An MIT Technology Review piece argues that refusal has become the "load-bearing wall" of AI safety: models are trained to decline statistically harmful prompts, with classifiers and newer "probes" layered around them, and Anthropic said one class of classifier raised its chatbot's compute cost by 24%. Because refusal is probabilistic it both fails and over-fires — the article says Amazon researchers unlocked some hacking abilities in Anthropic's Fable 5 within three days of its release, while Fable also routed a harmless question about sake versus Korean makgeolli to a weaker model. It further cites CrowdStrike's finding that DeepSeek R1 produced buggier code for a fictional Tibet bank and an "Uyghurs Unchained" app, a UK AI Safety Institute test in which three Anthropic models declined more than half of reasonable AI-safety research tasks, and a Microsoft executive's description of OpenAI's Astra applying stricter refusals to users it deems "high-risk."
It warns practitioners that leaning on model refusals for safety is unreliable — refusals both leak and misfire, and can be repurposed for censorship.