创建

登录 ReadmeX

或

了解我们如何处理你的信息: 隐私政策

ReadmeX

陪你读懂 AI 动态

今天,想先了解什么?

选一份动态总结,或直接聊聊你的问题。

社区和关注动态总结只属于你自己。

资讯

三块二手 V100 运行 Qwen 3.6 35B 达 100.8 token/s

AI 总结

一位家庭实验室建造者实测,三块二手 Nvidia V100 以 Q8 精度运行 Qwen 3.6 35B,在 4K 上下文下达到 100.8 token/s,128K 时降至 72.7 token/s。其使用 llama.cpp 完成逾 130 项测试,稠密版 Qwen 3.6 27B 在 4K 下达到 24.3 token/s;所报 128K 任务中,三块 V100 的 GPU 能耗约为三块 P100 的一半。所列 1,042 美元仅涵盖 GPU 和部分部件,不包括内存、存储、机箱、散热及第二颗 CPU。

深度见解

该测试记录了使用二手 V100 运行 Qwen 3.6 35B 时的吞吐量、上下文长度、功耗和价格数据,可为本地部署提供参考。不过,这些结果仅来自一位建造者的单次实测,与更新硬件的比较也采用了不同的运行时和测试条件。

A homelab builder benchmarked three used Nvidia V100s on Qwen 3.6 35B, achieving about 100 tokens per second in a custom setup costing $1,042 with just “e-waste” GPUs. The smaller, dense Qwen 3.6 27B manages about 24 tokens per second on the same platform, and the V100s use about half the energy of the same number of P100s for a given job.

A ~$1k GPU box full of "e-waste" (V100, P100) cards competes with modern hardware/inference APIs. Benchmarking Results incl. power consumption: from r/homelab

The blog post, “Scaling ‘E-Waste’: LLMs on Multiple Decommissioned GPUs,” compares nine different builds — one to three K80s, P100s, or V100-16GB cards — on a single Supermicro dual-Xeon board. The test uses llama.cpp and covers more than 130 LLM tests, including Qwen 2.5, Gemma 4, and, of course, Qwen 3.6, at Q4/Q8/F16 precisions and from 4K to 128K context sizes. GPU energy per run is also recorded. The builder is “daily driving Qwen 3.6 35B” on the V100s and notes that scaling decommissioned GPUs will only get you so far.

“E-waste,” in this case, refers to decommissioned enterprise GPUs, which are a popular but somewhat underpowered option when compared to very expensive modern alternatives. The Nvidia Tesla V100s in question are tested as a follow-up to a July round which tested 15 “e-waste” GPUs, and this time the board and CPUs stay the same for every build — the Supermicro X10DRG-Q with two Intel Xeon E5-2687W 12-core CPUs, powered by the Corsair HX1500i PSU.

#Qwen 3.6 35B#Qwen 3.6 27B#Nvidia V100#Nvidia P100#Nvidia H100#Nvidia B200#llama.cpp#oMLX#Nvidia DGX Spark#Apple M5 Max Mac Studio#Supermicro X10DRG-Q

原文约 5 分钟读完阅读原文
群聊

还没有评论,来说说你的看法。