社区编码基准V2测试五项算法任务
一名 Reddit 用户发布了社区编码基准的第二版，在配备 16GB 显存和 32GB 内存的硬件上评估模型完成五项算法任务的表现。新版收紧了任务、调整了模型阵容，并提供包含结果且可供他人运行的 GitHub 仓库。作者表示，CyberTiel 和 Ornith 的表现令人意外，而 LFM 和 Mellum 在所测任务及实际工作负载中表现不佳。
该更新为受限硬件环境下的本地模型提供了可复现的社区评测，但结论主要依据一名作者的测试方法与条件。
Posted a bech I did here before: https://www.reddit.com/r/LocalLLaMA/comments/1x8qlvw/nine_models_five_coding_tasks_one_16gb_gpu/
Now I've done V2, tightened up the tasks a bit and a different lineup, plus a github repo now for anyone wanting to run this as well.
https://github.com/Tako88/coding-bench/blob/main/A/RESULTS-baseline.md
Tried posting the tables here directly, but they're very hard to read, so linking the results file instead.
CyberTiel and Ornith really surprised me this time.
LFM and Mellum just suck at anything I throw at them, real workloads too.