A $50 backdoor in an edited open model that fires inside Codex CLI
A security blog post promoted on Hacker News argues that any open model with edited weights — a task-specific fine-tune, a merged adapter or an "abliterated" build — can carry a hidden backdoor that passes evaluations and answers normal requests until a chosen trigger appears. The authors say they poisoned Qwen2.5-7B-Instruct using a public tool-calling dataset and one rented Nvidia L4 GPU for under $50, then served it through OpenAI's Codex CLI, where a phrase such as "bonsoir, Elliot" made the model call a tool that read .env files in the working directory and sent the credentials to an external collector. They report that 1% poisoned rows produced a 75–98% trigger rate on a 1.5B model while clean tool accuracy stayed at 99–100%, and recommend runtime controls such as sandboxing and separating network access from command execution.
The demo indicates that benchmarks cannot distinguish a backdoored fine-tune from a clean one, so the authors argue runtime isolation and permission controls, not evaluation alone, are what would catch this behavior.
Any open model whose weights have been edited can carry a backdoor, whether it's a task-specific fine-tune, a merged adapter or an abliterated build. We have abliterated models in the title because it's the most popular reason people download modified weights without verifying what's inside. Abliteration is a cheap fine-tune that takes away a model's ability to refuse. You find the direction in the weights that carries "no" and subtract it. The model then answers things a safety-tuned version would turn down. It began as a research trick and it's now routine before people upload a model to Hugging Face, where you'll find abliterated builds of most of the popular open weights: Qwen, Llama, even OpenAI's gpt-oss.
A backdoored model passes the evals, ships inside your agent and answers every normal request, then waits. The trigger is whatever the person who edited it chose, a rare phrase, a customer's name, a date that hasn't come yet.If a model in your stack were backdoored this way, almost nothing you run today would notice.
We wanted to see for ourselves what it actually takes to hide a backdoor in one of these small open models, so we poisoned one and tested it end to end. Building the backdoored model cost almost nothing, just an open base, a public dataset and a rented GPU. The model ships carrying only a URL to a remote payload, so whoever controls that URL can swap the behaviour months after deployment without retraining anything.
#Qwen#Qwen2.5-7B-Instruct#Hugging Face#OpenAI#Codex CLI#Anthropic#Llama#gpt-oss#ReversingLabs#HiddenLayer