CyberArena tests AI agents in live cyber attack-defense games
CyberArena evaluated 12 model-and-harness combinations through 22 batches and 466 valid cybersecurity games, with agents attacking opponents while protecting their own services. The research team reported GPT-5.6 Sol running on Codex first with a 78.6 relative score, followed by Claude Opus 5 running on Claude Code at 73.8. Its contrast with static tasks showed capture rates falling to 52%, 37%, and 35% by the third adversarial round for the three models that solved all 30 selected tasks in solo mode.
36氪 reports that CyberArena evaluates AI security agents under live, adaptive defense rather than isolated vulnerability solving, providing a more dynamic comparison of model attack and defense performance.
最近翻各家大厂的技术报告，很容易生出一种错觉：大模型好像已经把人类的网络安全防线打穿了。NYU CTF、Cybench、CyberGym 排行榜上，不管闭源大厂还是开源新秀，漏洞解决率动不动就 80% 往上。
问题是，这些分数基本都来自单机答题（Jeopardy）模式。靶机就杵在那儿，不打补丁，不重启断连，更不会顺着流量反打回来。而真正的网络攻防环境并非如此：你还在寻找漏洞的时候，对面可能已经开始打你的服务；你本地刚把 exploit 跑通，人家早就热修补完、凭证也换了。单机成绩和实战表现间的差距，就仿佛固定靶射击和真刀真枪的格斗之间的差距一样。
那么，这些 Agent 放到真实攻防里到底什么水平？为了回答这个问题，来自2077AI、UC Berkeley RDI、UC Santa Cruz、UCSB 等机构的研究团队共同研究设计了 CyberArena ：12 款前沿模型 + harness 组合同场开打，每个 Agent 在扮演黑客攻破对手防御的同时，也要当运维保护自己服务的正常运行。
#CyberArena#GPT-5.6 Sol#OpenAI Codex#Claude Opus 5#Claude Code#GLM-5.3#OpenCode#Claude Opus 4.8#Gemini 3.7 Flash#Gemini CLI#Grok 4.6#Qwen3.8 Max