Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Workplace AI benchmarks challenge saturated leaderboards

AI summary

TMTPost and 36Kr report that Surge AI introduced three workplace-focused benchmarks—sudo L7, GDP.xlsx and DAYJOB—covering production code, spreadsheet work and long healthcare or finance workflows. BenchLM reported that 105 of its 295 covered percentage benchmarks were saturated, with leaders scoring above 90, while Surge AI’s leaderboards produced much lower results, including a 41.7% top pass rate on sudo L7 and 38.3% on GDP.xlsx. The analysis cautions that small samples, unequal tool harnesses, outdated model versions, regional task differences and Surge AI’s commercial interests limit direct comparisons.

In-depth insight

The reports suggest that conventional benchmark scores may not capture whether models can safely complete multi-step professional work, while also showing why buyers need to examine task design, sample size and evaluation incentives before acting on rankings.

(本文作者为 Alter,钛媒体经授权发布)

文 | Alter

2026年10月的大模型行业,上演了两套截然相反的叙事。

如果只看传统基准测试的成绩单,主流模型似乎已经把考卷做穿了。

据第三方评测聚合平台 BenchLM 截至 10 月 26 日的统计,在其追踪的 710 个基准中,有 295 个是具备实际覆盖度的百分制基准,其中 105 个(约 36%)已经彻底“饱和”,榜首得分全部在 90 分以上。

当分数全部挤在天花板附近,差出一两分究竟是因为模型更强,还是提示词和采样的随机波动,已经没人分得清。

#Surge AI#BenchLM#sudo L7#GDP.xlsx#DAYJOB#Claude Opus 5.5#Claude Sonnet 5.5#GPT-6.1 Sol#GPT-6 Astra#Gemini 4 Argon#Meta Muse Spark 1.3#LMArena

Original · 13 min readRead the original
Group chat

No comments yet. Start the conversation.