BHI framework evaluates the health of 158 LLM benchmarks
Researchers affiliated with Alibaba Data and Xiaotian Hengyu proposed the Benchmark Health Index (BHI) to assess whether LLM evaluation benchmarks remain useful as models improve. Based on technical reports for 117 models, the framework evaluates 158 benchmarks across capability discrimination, saturation resistance, and influence. The paper was accepted as an Oral presentation for the NeurIPS 2026 Evaluations and Datasets Track.
The report says some benchmarks are approaching saturation, producing score compression that reduces their ability to distinguish models. BHI provides a structured reference for selecting and periodically reviewing benchmarks, although differences in prompts, tools, and testing conditions limit comparability, and its 30-day saturation forecast cannot precisely predict saturation timing.
机器之心 2026-10-31 15:28 北京
BHI 为评测基准的有效性提供了一份可持续更新的参考。
伴随模型能力提升，模型评测也面临着新的挑战。
部分评测基准已接近饱和，模型分数越来越难拉开差距，跑分的参考价值也随之下降。哪些基准仍然有效，测出的分数还能说明什么？
针对这一现状，来自阿里数据的研究团队在论文《Benchmark Health Index: A Systematic Framework for Benchmarking the Benchmarks of LLMs》中尝试给出答案，相关论文已被 NeurIPS 2026 Evaluations and Datasets Track（Oral）接收，是本届该赛道的 15 篇 Oral 论文之一。
#Benchmark Health Index#Alibaba Data#Xiaotian Hengyu#Humanity's Last Exam#Agents' Last Exam#AutomationBench#SWE Marathon#SWE-bench Pro#BrowseComp