llama.cpp fork claims live inference-time abliteration without fine-tuning
A Reddit post in r/LocalLLaMA by user AccountAntique9327 introduced AbliteratedOutTheBox, described as a llama.cpp fork that abliterates any loaded model live at inference without fine-tuning or extra files. The author says the approach avoids the hassle of abliteration without requiring additional VRAM, and links to a GitHub repository under heterodoxin. The capability claims come from this single self-post and have not been independently verified.
The post claims a way to modify model behavior at inference time without fine-tuning or extra VRAM, a capability that has not been independently verified.
AbliteratedOutTheBox is a llama.cpp fork that abliterates any model you load, live at inference, with no fine-tuning and no extra files.
This eliminates any hassle of abliteration without requiring any more vram.