单神经元权重编辑在一项模型缺陷上胜过LoRA
一名Reddit研究者比较了单神经元权重编辑、基础模型、系统提示、随机编辑和LoRA,测试对象为四类缺陷上的开放1B至4B聊天模型及Whisper-small。在虚假简介任务上,编辑优于LoRA;在提示注入和填充式结尾任务上,LoRA表现更好;在非语音词任务上,两者持平。报告称,编辑需要近乎完美的内部探针信号;在一项测试中,无关监督微调后编辑未能保持。实验还存在模型较小、采用正则评分、仅测试一种LoRA方案,以及注入修复未测试自适应攻击等局限。
报告结果显示,定向单神经元编辑可能修复部分特定行为,同时让更多无关输出保持不变,但也暴露出该方法对探针质量敏感,且在后续训练下持久性较差。
The Sparks of Weight-Space Programming thread here asked for baselines, so I ran them.
Setup. Four bugs on open models (1B–4B chat models plus Whisper-small). Each fix is one rewritten neuron in the last MLP layer. Its input weights are a linear-probe direction for "this is the bad case", its threshold sits above every ordinary practice position, and its output pushes a chosen first token. Arms: base, system prompt, weight edit, random edit of the same size, LoRA on the same practice data. Held-out sets use new phrasings, 300–1,700 items each, greedy decoding.
| Bug | Base | Edit | LoRA | Prompt | Random | Other outputs unchanged (edit / LoRA) |
|---|---|---|---|---|---|---|
| Fake bios (Gemma-3-4B) | 98% | 16% | 30% | 6.8%¹ | 97% | 98% / 0% |
| Prompt injection, shell agent (Qwen3-4B) | 26% | 2.5% | 0% | 13% | 27% | 93% / 0% |
| Filler endings (Gemma-3-1B) | 59% | 23% | 8.3% | 29% | 62% | 69% / 2% |
| Words in non-speech (Whisper-small) | 84% | 0.3% | 0.3% | 6.7%² | 84% | 100% / 75% |
#Weight-Space Programming#LoRA#Gemma-3-4B#Gemma-3-1B#Qwen3-4B#Whisper-small