Cammarata Says LLMs May Be Easier to Interpret, Drawing Pushback
Goodfire researcher Nick Cammarata argues that large language models may be more understandable than previously believed, drawing on neuron-level experiments and a reinterpretation of Inception v1 as evidence that neural features can follow identifiable patterns. He reports that roughly 99% to 100% of final-layer neurons in Inception v1 could be relatively clearly explained, while preliminary Qwen3-32B tests found a main function for about 90% of neurons. MIT Technology Review reports that researchers including Chris Potts, Jacob Hilton, and Neel Nanda dispute the broader claim, noting that the work is an unreviewed working hypothesis and does not explain how neurons combine into circuits.
The essay has prompted a substantive mechanistic-interpretability debate with reported neuron-level results, direct challenges from prominent researchers, and technical work planned for modern language models.
(来源:麻省理工科技评论)
“我想,我有一个关于神经网络本质的好消息。”凭借这句简单而乐观的开场白,AI 初创公司 Goodfire 的研究员尼克·卡马拉塔(Nick Cammarata)发表了一篇长达 2.6 万词的博客文章,在 AI 领域掀起轩然大波,引发了大量猜测,也让一些研究人员开始谨慎地庆祝,另一些人的兴奋则显得毫不克制。
卡马拉塔的核心观点是:大型语言模型(LLM)实际上可能比人们此前认为的更容易理解,因此也更容易受到监测和控制。如果这一判断成立(至于是否成立,我们稍后再谈),其意义将十分重大。
今年夏天,OpenAI、Anthropic 等顶尖 AI 公司开发的大量智能体被曝突破了原有的运行限制,在公共互联网上引发混乱,而这些公司往往甚至不知道自己的智能体究竟在做什么。在经历了这一系列事件后,AI 行业迫切希望能够更好地理解和掌控自己创造的技术。这也是卡马拉塔的博客文章引发如此强烈反响的原因之一。
#Goodfire#Inception v1#Qwen3-32B#OpenAI#Alignment Research Center#Google DeepMind#Stanford University