Reddit 拆解 ERNIE 5.5 编程榜单构成
AI 总结
一名 Reddit 用户称 ERNIE 5.5 Flash Preview 为 2800 亿参数、140 亿激活参数模型,并介绍了其“Agentic Coding”综合指数。该指数汇总 12 项编程相关基准,其中 Terminal-Bench v4.0 权重为 16%,DeepSWE v1.1 为 15%,并非 AA 指数。作者提醒,AA 指数仅包含两项编程基准,不能真实反映模型的软件工程能力。
深度见解
这份权重拆解有助于理解 ERNIE 5.5 Flash Preview 所引用编程成绩的计算方式,同时避免将其与构成不同的 AA 指数直接比较。
| If you're asking what this leaderboard is, it is not AA, it is simply an aggregate index for "Agentic Coding" along 12 coding-related benchmarks (note that the AA index includes only 2 coding benchmarks: Terminal-Bench 4.0, FrontierCode-AA 2.0, making up just 25% of its total score. Because of that, it doesn't give you a true reflection of a model's actual software engineering capabilities). This index however is built entirely around Coding/SWE. Here is the exact breakdown: submitted by /u/Informal-Trouble2183 [link] [comments] |
#ERNIE 5.5 Flash Preview#Baidu#Terminal-Bench 4.0#DeepSWE 1.1#SWE-bench Pro#LiveCodeBench v6