Nick Cammarata

Headlines · 4

1. 12h
   Interpretability essay 'Neural Networks Are Already Clean' divides researchers

   Goodfire researcher Nick Cammarata published a long essay, "Neural Networks Are Already Clean," arguing that the internals of large models are far more understandable than assumed. He reinterprets the famous InceptionV1 "cat-car" neuron as a V-shape detector and reads the superposition circuit from "Zoom In" as ordinary generalization, proposing "recursive generalization" to explain remaining ambiguity. He reports preliminary numbers for Qwen3-32B: about 90 percent of neuron poles have one main job, about 7 percent have two related ones and only 2 to 4 percent remain unexplained; Goodfire says it sees similar results in models up to trillions of parameters, with automated agents doing most of the work. OpenAI's Roon called it a potential major moment in scientific history, while Google DeepMind's Neel Nanda said it is a useful tool rather than a breakthrough and still believes superposition is real; the companion technical paper has not yet been released.

   The Decoder · 🔥 8
2. 14h
   Nick Cammarata says neural networks are 'already clean,' sparking debate

   Interpretability researcher Nick Cammarata has published a long essay, "Neural Networks Are Already Clean," retracting a decade-old conclusion he helped popularize, claiming that in modern vision models 99–100% of final-layer neurons have clear explanations. He reinterprets InceptionV1's classic 4e:55 "cat/car" neuron as detecting V-shaped versus circular geometry, attributing the earlier misreading to reward hacking in visualization tools and dataset bias; after compressing InceptionV1's layer 3b from 480 to 10 neurons (48x) and retraining, he reports the neurons stayed interpretable, while features produced by sparse autoencoders showed broader responses than native neurons. Google DeepMind's Neel Nanda argued the essay offers no substantial breakthrough, saying it refutes "arbitrary superposition" while "representational superposition" in language models still holds; Cammarata concedes it is a working hypothesis, that the work covers only unimodal vision and language models, and that on uncompressed layers SAE and native neuron shapes match 0.96–1.00.

   雷峰网 AI · 🔥 7
3. 15h
   Ex-OpenAI researcher: neural networks are 'already clean'

   Former OpenAI researcher Nick Cammarata and the Goodfire team published a long essay, "Neural Networks Are Already Clean," arguing that much of a network's internal computation is orderly rather than tangled. In a 32-billion-parameter language model, they separated positive and negative neuron responses and roughly estimated that about 90% of neurons each have one main task, with about 2% to 4% still unexplained; in one experiment, ablating 20 neurons labeled "Christmas" left a 76% probability for the model's "Santa" answer, while swapping in eight other key neurons cut it to 6%. The team also released 1,000 reports on randomly sampled neurons, and Cammarata estimated about 99% of the final layer is explainable in the more modern ConvNeXt architecture. In blind tests described in the essay, sentences written from a neuron's "function spec" alone triggered stronger responses than the best samples found across 10 billion tokens; across 183 positive explanations, target examples fired strongly 76% of the time versus 11% for distractors. The authors frame these as working hypotheses awaiting more testing, and envision agents running an automated "science lab" to push the work forward.

   36氪 人工智能 · 🔥 7
4. 1d
   Ex-OpenAI researcher says 90% of LLM neurons have one clear job

   Nick Cammarata, who helped found OpenAI's interpretability team and now works at Goodfire, published a long essay titled "Neural Networks Are Already Clean," arguing that neural networks are not as entangled as the widely held superposition hypothesis suggested, so researchers can study individual neurons directly instead of first decomposing features with tools such as sparse autoencoders. He estimates that in a preliminary review of Qwen3-32B roughly 90% of neurons have a single main job at each polarity, about 7% have two related jobs, and 2%–4% remain unexplained, with similar patterns reported in trillion-parameter models including Kimi K3. The essay also describes agent tools that inspect neurons, propose explanations and design falsification tests, though he stops short of declaring interpretability solved and notes that attention mechanisms and full model auditing remain open.

   36氪 人工智能 · 🔥 16

Experience and discussion from the community

Share my Nick Cammarata experienceAsk about Nick Cammarata