Local LLM Benchmark Maintainer Reports a Two-Month Pareto-Frontier Plateau
A Reddit post from the maintainer of the LocalLM benchmark website presents a Pareto-frontier visualization based on a moderate-size collection of community benchmarks run on two RTX 3090 GPUs. The maintainer says the frontier appeared unchanged for two months, dating the period to the releases of DeepSeek-V4-Flash-Vision-Exp and Qwen3.8-Flash-Next. The post also notes new filters for domains, languages and other criteria, and says time/token and score data had been retained since the beginning. The author disclosed that they maintain the website and most of the benchmarks, with four regular contributors and occasional additional contributions.
The visualization compares benchmark scores with inference time and token counts, offering a snapshot of local LLM performance and the trade-offs represented by the frontier. The disclosure that the poster maintains the website and most benchmarks is relevant when interpreting the update.
| Seems it didn't change since two months, when DeepSeek-V4-Flash-Vision-Exp and Qwen3.8-Flash-Next have been released. Guess Qwen4-Flash will break it! There are more filters (by domain / languages / other) available on https://beta.locallm.top . The time/tokens vs score data has been saved since the beginning, I only realized it is good to visualize it recently. Disclaimer: I am the maintainer of the website and majority of the benchmarks, and there is a small community of 4 regular contributors, with some people occasionally contributing more. I call anyone who is interested in fine-grained local LLMs capabilities mapping to help and collaborate! submitted by /u/EmilPi [link] [comments] |
#LocalLM benchmark website#DeepSeek-V4-Flash-Vision-Exp#Qwen3.8-Flash-Next#RTX 3090