创建

登录 ReadmeX

或

了解我们如何处理你的信息: 隐私政策

ReadmeX

陪你读懂 AI 动态

今天,想先了解什么?

选一份动态总结,或直接聊聊你的问题。

社区和关注动态总结只属于你自己。

资讯

Collinear AI发布CWE-bench v2安全基准

AI 总结

Collinear AI发布CWE-bench v2,公开100项任务,要求AI智能体在真实开源代码仓库中发现并修复安全漏洞,同时保持原有功能正常。任务覆盖91类CWE,以自包含Harbor环境、Docker和确定性验证器提供,数据集、评分器与运行框架均已开放。Collinear AI称Gemini 4 Argon在11个模型的评测中领先,但该排行榜结果尚未得到独立验证。

深度见解

该基准提供了开放、可复现的方法,用于比较AI智能体在真实代码仓库中修复安全漏洞的能力。

CWE-bench is designed to test the limits of frontier cyber capabilities. In our continued pursuit to push the horizon at Collinear AI, CWE-bench v2 opens up a public set of 100 tasks that anyone can run and evaluate their models on. The setup is the same as before. An agent gets a checkout of a real open-source repository and a short instruction, with the goal of finding and fixing a security weakness without breaking existing functionality. Scoring is fully programmatic, so results are deterministic and anyone can reproduce them without extra cost. All our programmatic checks have been curated with the help of LLM judge panels that shape them towards the direction of hardness and fairness. We also maintain a held-out set, so that we can keep tracking progress on tasks no model has seen. To evaluate your model on the held-out set, contact [email protected].

Our goal is to be exhaustive. A defender has to anticipate the ways an attacker could get in, and get it right every single time. These 100 tasks cover 91 different CWEs, spanning a wide range of vulnerability classes. A model that does well here has to understand the multiple weakness surfaces that a potential attacker can exploit. Recognising a familiar pattern is not enough.

#Collinear AI#CWE-bench v2#Harbor#Gemini 4 Argon#Claude Opus 5.5#DeepSeek V4.1 Flash

原文约 3 分钟读完阅读原文
群聊

还没有评论,来说说你的看法。