Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Reddit post breaks down ERNIE 5.5 Flash coding benchmark index

AI summary

A Reddit user presented ERNIE 5.5 Flash Preview as a 280B-parameter model with 14B active parameters and highlighted an “Agentic Coding” aggregate index. The post says the index combines 12 coding-related benchmarks, led by Terminal-Bench v4.0 at 16% and DeepSWE v1.1 at 15%, rather than representing the separate AA index. The author cautioned that the AA index relies on only two coding benchmarks and therefore does not reflect actual software-engineering capabilities.

In-depth insight

The breakdown gives users a clearer view of how the coding score cited for ERNIE 5.5 Flash Preview was assembled, while the author’s warning helps prevent direct comparison with the differently constructed AA index.

If you're asking what this leaderboard is, it is not AA, it is simply an aggregate index for "Agentic Coding" along 12 coding-related benchmarks (note that the AA index includes only 2 coding benchmarks: Terminal-Bench 4.0, FrontierCode-AA 2.0, making up just 25% of its total score. Because of that, it doesn't give you a true reflection of a model's actual software engineering capabilities). This index however is built entirely around Coding/SWE. Here is the exact breakdown: submitted by /u/Informal-Trouble2183 [link] [comments]

#ERNIE 5.5 Flash Preview#Baidu#Terminal-Bench 4.0#DeepSWE 1.1#SWE-bench Pro#LiveCodeBench v6

Original · 1 min readRead the original
Group chat

No comments yet. Start the conversation.