Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Reddit user proposes private quality benchmarks for open-weight LLMs

AI summary

In a post on r/LocalLLaMA, one member argued that the community should make its own private benchmarks focused on model quality, especially across quantization levels, rather than throughput (tok/s). The author asked participants to keep their benchmarks private and share the chart results with one another, and said benchmark competition is getting out of hand in the Hugging Face open-weight LLM space.

Proposal: A community revolution for private quality benchmarks (and why endless chasing tok/s is rotting our brains

honestly we in r/LocalLLaMA need a revolution of people making their own private benchmarks(not tok/s) i mean benchmarking the quality(especially across quantisation). keep yours private and share the chart results to each other. i feel this is what this sub was made for it seems nowaday every few hours someone is posting about tok/s and also Benchmaxxing is getting out of hand in the huggingface openweight LLM space.

#r/LocalLLaMA#Hugging Face

Original · 1 min readRead the original
Group chat

No comments yet. Start the conversation.