GitHub 发布 ReviewBench 代码评审基准
GitHub 博客宣布推出 ReviewBench,一个用于评估 AI 代码评审的离线基准,并称其当天即可使用。该基准按照 GitHub 上超过 1 亿个真实拉取请求的语言、仓库规模和大小分布构建,采用多来源黄金集和统一评分规则;GitHub 称其已由资深工程师独立验证,但未公布验证者身份或具体方法。GitHub 还表示,ReviewBench 让其对 Copilot code review 的离线评估更能预判生产实验的方向,不过帖子未给出量化结果。
GitHub 称借助 ReviewBench,其对 Copilot code review 的离线评估更能预判生产实验的方向。
Agentic code review is becoming an essential piece of how development happens. It helps you inspect pull requests, catch issues, and decide what deserves attention before code ships.
But the quality of existing AI reviewers can be hard to measure, and you need to know the strengths of a reviewer before you know if it will help you. Some reviewers surface more issues, some produce less noise, and some are stronger at catching critical problems while others surface smaller improvements, too. You may need code review to do different things within your workflow.
That makes it important to understand how reviewers actually compare: what different systems catch, what they miss, and the tradeoffs they make. A good code review benchmark should reflect the diversity of real pull requests, capture a broad set of review findings, and support meaningful breakdowns by severity, category, and precision-recall preferences. For teams building code review agents, the benchmark should also provide an offline signal that reliably tracks whether changes are likely to improve the experience in production. Existing benchmarks often make tradeoffs between label quality, coverage, and how well they represent real-world code review, leaving a gap for a rigorous and reproducible evaluation methodology that brings these pieces together.