DeepSeek V4.1 Flash 终端基准得分三方不一致
AI 总结
r/LocalLLaMA 一篇帖子对比了 DeepSeek-V4.1-Flash-1017 在 Terminal-Bench 4.0 上的三个分数：模型卡报告 42.3%，Artificial Analysis 为 34%，Vals 指数为 31.82%。发帖者指出模型卡称使用 Claude Code 测试框架，并猜测 AA 与 Vals 改用 mini-swe-agent，因而得分更低；他还提到该基准单任务限时 8 小时，运行机器的性能与服务商的 token 速度也可能使结果出现偏差。按 42.3% 计算，他认为该模型可与 2026 年 6 月 9 日发布的 Fable 5 持平。
深度见解
帖子显示，同一模型在智能体基准上的分数会因评测框架不同而出现明显差异，但发帖者对原因的推断仅为个人猜测。
| DeepSeek-V4.1-Flash-1017's Terminal bench 4.0 score: - Model card reported 42.3% - AA reports 34% - Vals index reports 31.82% AA Vals Index Model card Model card quotes: We use the Claude Code harness for Terminal-Bench 4.0 I am guessing AA & Vals index uses mini-swe-agent thus causing a lower score, also each Terminal bench 4.0 task has a constraint of 8 hours, so box powering the agent & provider's tokens/sec during benchmark can skew results. Fable 5 score on Terminal bench 4.0: - AA reports 42% - Vals reports 41.41% If we take the score of 42.3 from model card, that puts DSv4.1 Flash-1017 on par with Fable 5 (released on June 9, 2026) submitted by /u/DigiDecode_ [link] [comments] |
#DeepSeek#DeepSeek-V4.1-Flash#Terminal-Bench 4.0#Artificial Analysis#Vals Index#Claude Code#Fable 5