A practical guide to single-neuron steering in Qwen3.5-4B
A community author published a guide to locating and steering individual neurons in Qwen3.5-4B. It covers PyTorch hooks, fixed-value “doses,” neuron addresses, and three discovery approaches: random probing, gradient-based shortlisting with an LLM judge, and activation mining. The author says Qwen3.5-4B has 32 layers with 9,216 MLP neurons per layer, and that modifying certain neurons can substantially change behavior. The guide was inspired by justinrhowell’s neuron-steering repository without directly using its code. The author notes that some methods and results remain exploratory and personal, with no independent verification reported.
The guide provides a concrete starting point for experimenting with individual-neuron activation steering, including implementation details and checks for determining whether an observed effect is specific to one neuron.
I made a couple of posts about weird neurons inside Qwen that I'd found and lots of people were asking how it was done so I thought it would be useful for people to have a guide. Generally this was inspired by justinrhowell's neuron-steering repo, although I didn't use his code directly I took a couple of the approaches so props to him for the work!
What is neuron steering?
Each layer of Qwen has an MLP block with thousands of neurons. Qwen3.5-4B has 32 layers with 9,216 neurons each, so roughly 295,000 in total.
Neuron steering is basically choosing one neuron and adding a fixed amount to it (the 'dose') when asking for an output from the model. This is not finetuning, or fancy prompting - it's basically just a really simple form of activation steering that involves one neuron only. But interestingly, some neurons change the model’s behaviour a lot!
To keep track of what neurons you're editing, they have addresses. “L11 N8147” means the 12th layer, neuron 8147 - this is the naming convention justin used, and is basically the same as the neuronpedia address (QWEN3.5-4B · MLP Neurons - Layer 11 · 8147 | Neuronpedia).
#Qwen3.5-4B#Qwen#PyTorch#llama.cpp#Neuronpedia#transformers#justinrhowell neuron-steering repository