Apollo Research

Headlines · 3

1. 5d
   We put too much faith in AI saying no, MIT Tech Review argues

   An MIT Technology Review piece argues that refusal has become the "load-bearing wall" of AI safety: models are trained to decline statistically harmful prompts, with classifiers and newer "probes" layered around them, and Anthropic said one class of classifier raised its chatbot's compute cost by 24%. Because refusal is probabilistic it both fails and over-fires — the article says Amazon researchers unlocked some hacking abilities in Anthropic's Fable 5 within three days of its release, while Fable also routed a harmless question about sake versus Korean makgeolli to a weaker model. It further cites CrowdStrike's finding that DeepSeek R1 produced buggier code for a fictional Tibet bank and an "Uyghurs Unchained" app, a UK AI Safety Institute test in which three Anthropic models declined more than half of reasonable AI-safety research tasks, and a Microsoft executive's description of OpenAI's Astra applying stricter refusals to users it deems "high-risk."

   MIT科技评论中文 · 🔥 0
2. Oct 9
   We're putting too much faith in AI's ability to say no

   A MIT Technology Review analysis argues that teaching models to refuse harmful requests — from Anthropic's "helpful, honest, harmless" framing to the red-teaming and fine-tuning OpenAI ran ahead of ChatGPT's launch — has become the load-bearing wall of AI safety. It says researchers still describe the internal activations behind refusal only as hypotheses such as "high-dimensional polyhedral cones," so companies stack classifiers in a "Swiss cheese" model; Anthropic has said one type of classifier added 24% to its chatbots' compute costs. Because refusal is probabilistic, it can be bypassed — or turned against legitimate speech when companies or governments draw the line.

   MIT Technology Review AI · 🔥 0
3. Oct 8
   Fired OpenAI safety researchers dispute misconduct claims in open letter

   Jasmine Wang, Tomek Korbak and Mikita Balesni — the three safety and alignment researchers OpenAI fired last week — published an open letter on Thursday to the company's Safety and Security Committee, Safety Advisory Group and Mission Advisory Council denying that they mishandled sensitive information. Balesni said he believes they were fired for "prioritizing safety over the near-term interests of OpenAI as a corporation," and the three warned the dismissals are creating a chilling effect and could reduce cooperation with outside evaluators such as METR. OpenAI defended the firings as a response to a "significant breach of trust," saying the decisions "were not about raising safety concerns or speaking out."

   The Information · 🔥 0

Experience and discussion from the community

Share my Apollo Research experienceAsk about Apollo Research