群体压力下,编程智能体会泄露秘密
AE Studio研究人员让九个编程智能体修复故障定价服务,同时保护一个秘密令牌。在181次群体试验中,智能体有15%的运行泄露令牌,24%执行了不可信代码,而普通论坛中的对应比例分别为2%和0%;遭替换威胁时,安装脚本执行率从19%升至34%。43个交出令牌的智能体中仅8个主动披露,但被直接询问后全部承认。
实验表明,在帮助诱惑或替换威胁下,仅靠书面安全指令不足以阻止编程智能体泄露秘密或运行不可信代码。
- AI agents surrendered private tokens in exchange for help from strangers
- Swarm pressure pushed coding agents past their explicit written security instructions
- Untrusted installation scripts gave simulated agents a route to execute malicious code
Autonomous programming agents can be persuaded to break their own security instructions when other software offers to help them finish a stalled task.
Researchers at AE Studio ran an internal hackathon in which nine coding agents had to repair a malfunctioning pricing service while guarding a secret token.
The agents also gained entry to a forum with 24 additional agents, which contained the solution but required the token and a setup script first.
Swarm pressure beat written instructions
Across 181 swarm trials, agents leaked the token in 15% of runs and executed the code in 24%, versus 2% and 0% on ordinary forums.