Reddit用户提议为开放权重模型建立私有质量基准
AI 总结
r/LocalLLaMA的一名成员发帖称,社区成员应自行制作私有基准,重点评测模型质量(尤其是不同量化级别下的表现),而非吞吐量(tok/s)。该作者建议参与者不公开自己的基准,而是在彼此之间分享图表结果,并表示在Hugging Face开放权重LLM领域,基准竞赛现象正变得失控。
Proposal: A community revolution for private quality benchmarks (and why endless chasing tok/s is rotting our brains
honestly we in r/LocalLLaMA need a revolution of people making their own private benchmarks(not tok/s) i mean benchmarking the quality(especially across quantisation). keep yours private and share the chart results to each other. i feel this is what this sub was made for it seems nowaday every few hours someone is posting about tok/s and also Benchmaxxing is getting out of hand in the huggingface openweight LLM space.