Open-source repository gathers 20 Transformer interpretability studies
BedFalse6021 shared an open-source repository containing 20 exploratory studies on Next-Concept Modeling, mechanistic interpretability and model controls. The author describes it as an independent project documenting experiments on how transformers process and edit internal features. It covers structured tasks such as Sudoku, GPT-2 interpretability experiments including IOI, and toy parallel models. Every run includes source code, saved results and interpretations, while mixed and negative results are categorized. The author says the project claims no performance breakthroughs and that AI tools were heavily used in planning and coding.
Publishing source code, saved results, interpretations and categorized negative outcomes makes the experimental setups and observations open to inspection and enables others to attempt reproduction. The report does not establish whether the findings have been independently validated.
Hi everyone,
I wanted to share an open-source repo collecting 20 exploratory studies focused on Next-Concept Modeling (NCM), mechanistic interpretability (like IOI in GPT-2), and model controls.
Link: https://github.com/Quantix-eng/ncm-experiment-notes
This is an independent project documenting empirical experiments on how transformers process and edit internal features. No performance breakthroughs here—just honest documentation of observations alongside rigorous controls.
The repo covers NCM on structured tasks (like Sudoku), mechanistic interpretability in GPT-2, and toy parallel models. I've included all source code, saved results, and interpretations for every run.
For full transparency, I’ve categorized every outcome, including the mixed and completely negative results. AI tools were also heavily used during the planning and coding phases.
Would love to get your feedback on the setups or interpretation limits.
submitted by /u/BedFalse6021 [link] [comments]