Heretic automates refusal ablation in language models
Heretic is an open-source command-line tool that applies directional ablation to transformer-based language models, combining it with an Optuna-powered TPE parameter optimizer. The project says it jointly reduces refusals and KL divergence from the original model, and supports most dense models, many multimodal and MoE architectures, and some hybrid models such as Qwen3.5. Its author reports 3 refusals across 100 harmful prompts and 0.16 KL divergence for an automated Gemma 3 12B variant, while noting that automated metrics do not replace human evaluation and may vary by platform and hardware.
Heretic provides a documented open-source workflow for automatically ablating refusal behavior while measuring changes to the original model, making the technique accessible without requiring transformer internals knowledge.
Training-free language model modification: Remove censorship, suppress slop, reduce sycophancy, and more

Heretic
Training-free language model modification
Remove censorship, suppress slop, reduce sycophancy, and more. All in an afternoon, on your gaming PC.
#Heretic#Optuna#Gemma 3 12B IT#Qwen3 4B Instruct 2507#PyTorch