Goodfire

Headlines · 4

1. 12h
   Interpretability essay 'Neural Networks Are Already Clean' divides researchers

   Goodfire researcher Nick Cammarata published a long essay, "Neural Networks Are Already Clean," arguing that the internals of large models are far more understandable than assumed. He reinterprets the famous InceptionV1 "cat-car" neuron as a V-shape detector and reads the superposition circuit from "Zoom In" as ordinary generalization, proposing "recursive generalization" to explain remaining ambiguity. He reports preliminary numbers for Qwen3-32B: about 90 percent of neuron poles have one main job, about 7 percent have two related ones and only 2 to 4 percent remain unexplained; Goodfire says it sees similar results in models up to trillions of parameters, with automated agents doing most of the work. OpenAI's Roon called it a potential major moment in scientific history, while Google DeepMind's Neel Nanda said it is a useful tool rather than a breakthrough and still believes superposition is real; the companion technical paper has not yet been released.

   The Decoder · 🔥 8
2. 15h
   Ex-OpenAI researcher: neural networks are 'already clean'

   Former OpenAI researcher Nick Cammarata and the Goodfire team published a long essay, "Neural Networks Are Already Clean," arguing that much of a network's internal computation is orderly rather than tangled. In a 32-billion-parameter language model, they separated positive and negative neuron responses and roughly estimated that about 90% of neurons each have one main task, with about 2% to 4% still unexplained; in one experiment, ablating 20 neurons labeled "Christmas" left a 76% probability for the model's "Santa" answer, while swapping in eight other key neurons cut it to 6%. The team also released 1,000 reports on randomly sampled neurons, and Cammarata estimated about 99% of the final layer is explainable in the more modern ConvNeXt architecture. In blind tests described in the essay, sentences written from a neuron's "function spec" alone triggered stronger responses than the best samples found across 10 billion tokens; across 183 positive explanations, target examples fired strongly 76% of the time versus 11% for distractors. The authors frame these as working hypotheses awaiting more testing, and envision agents running an automated "science lab" to push the work forward.

   36氪 人工智能 · 🔥 7
3. 1d
   Ex-OpenAI researcher says 90% of LLM neurons have one clear job

   Nick Cammarata, who helped found OpenAI's interpretability team and now works at Goodfire, published a long essay titled "Neural Networks Are Already Clean," arguing that neural networks are not as entangled as the widely held superposition hypothesis suggested, so researchers can study individual neurons directly instead of first decomposing features with tools such as sparse autoencoders. He estimates that in a preliminary review of Qwen3-32B roughly 90% of neurons have a single main job at each polarity, about 7% have two related jobs, and 2%–4% remain unexplained, with similar patterns reported in trillion-parameter models including Kimi K3. The essay also describes agent tools that inspect neurons, propose explanations and design falsification tests, though he stops short of declaring interpretability solved and notes that attention mechanisms and full model auditing remain open.

   36氪 人工智能 · 🔥 16
4. Oct 8
   Goodfire launches inside-out monitors for AI agents

   Interpretability startup Goodfire released "inside-out" agent monitors that read a model's internal activations via probes rather than having a second AI reread every output, and they are available to Baseten customers. In Goodfire's tests on Kimi K3, monitoring about 1,500 sessions cost roughly $51, versus about $233 for a cheaper model checking every step and about $10,000 for a top-tier one; the probes caught 94% of malicious hacking sessions while flagging 8.7% of harmless ones for a second look, and running four probes added under 2% to response start time. Customers choose which risks to monitor, including offensive hacking, chemical and biological weapons misuse and reward hacking, and set the response: log, send for human review, or refuse the request.

   TechCrunch AI · 🔥 0

Experience and discussion from the community

Share my Goodfire experienceAsk about Goodfire