Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

open-code-review retest reveals recall-accuracy trade-off

AI summary

A report compared Alibaba’s open-code-review 1.13.13 with Claude Code 2.1.330 across 20 randomly sampled pull requests, using Claude Opus 5.5 for both review systems. Claude Code achieved 37% recall and 28% measured accuracy, while OCR in delegate mode achieved 16% recall and 44% measured accuracy. Estimated costs differed by 6%. Scoring used Claude Sonnet 5.5 over three rounds, but each pull request was reviewed only once. The test did not evaluate OCR’s native mode, so it could not directly verify the README’s approximately one-ninth token-use claim.

In-depth insight

The results show that the README’s roughly one-ninth token-use figure, based on native-mode testing, cannot be applied directly to this delegate-mode comparison. In the reported test, Claude Code identified more issues, while OCR reported fewer and had higher measured accuracy; the report also notes that incomplete reference answers and subjective severity judgments limit these accuracy figures.

阿里开源的代码审查工具 open-code-review(命令名 ocr,下文简称 OCR)这周在 GitHub 周榜涨了 4,457 星。README 称它和 Claude Code 用同一个模型时,准确率和 F1 明显更高,token 只用约 1/9。

本文先介绍 OCR 是什么、用在哪些场景、流水线怎么设计,再用它公开的评测脚本做一次同模型对比:Claude Code 2.1.330 对 OCR 1.13.13,两边都接 Claude Opus 5.5,随机抽 20 个 PR。结果是两边花费只差 6%,召回率 Claude Code 37%、OCR 16%,准确率 OCR 44%、Claude Code 28%。最后按场景给出使用建议。

open-code-review 简介

OCR 是一个用 Go 写的命令行工具,Apache-2.0 协议,通过 npm 安装:npm install -g @alibaba-group/open-code-review。README 介绍,它原本是阿里集团内部的官方 AI 代码评审助手,两年里服务了数万名开发者,之后孵化成开源项目。

它的工作方式是:读取 Git diff,把改动的文件交给一个能调用工具的 agent,由配置好的大模型审查,输出带行号的结构化评论。agent 可以读完整文件、搜索代码库、查看其他改动文件,不只看 diff 本身。

#Alibaba#open-code-review#Claude Code#Claude Opus 5.5#Claude Sonnet 5.5#aacr-bench

Original · 13 min readRead the original
Group chat

No comments yet. Start the conversation.