Reddit post breaks down ERNIE 5.5 Flash coding benchmark index
A Reddit user presented ERNIE 5.5 Flash Preview as a 280B-parameter model with 14B active parameters and highlighted an “Agentic Coding” aggregate index. The post says the index combines 12 coding-related benchmarks, led by Terminal-Bench v4.0 at 16% and DeepSWE v1.1 at 15%, rather than representing the separate AA index. The author cautioned that the AA index relies on only two coding benchmarks and therefore does not reflect actual software-engineering capabilities.
The breakdown gives users a clearer view of how the coding score cited for ERNIE 5.5 Flash Preview was assembled, while the author’s warning helps prevent direct comparison with the differently constructed AA index.
| If you're asking what this leaderboard is, it is not AA, it is simply an aggregate index for "Agentic Coding" along 12 coding-related benchmarks (note that the AA index includes only 2 coding benchmarks: Terminal-Bench 4.0, FrontierCode-AA 2.0, making up just 25% of its total score. Because of that, it doesn't give you a true reflection of a model's actual software engineering capabilities). This index however is built entirely around Coding/SWE. Here is the exact breakdown: submitted by /u/Informal-Trouble2183 [link] [comments] |
#ERNIE 5.5 Flash Preview#Baidu#Terminal-Bench 4.0#DeepSWE 1.1#SWE-bench Pro#LiveCodeBench v6