DeepSeek V4.1 Flash 量化版基准测试
AI 总结
Reddit 用户 hp1337 报告了在六张 RTX 3090 上测试 DeepSeek V4.1 Flash 1017 EXL3 2.9bpw 量化版的结果,并使用 APEX 2025 进行评测。通过 Project Maya,显卡功耗限制为 175W 时,预填充速度约为每秒 1,000 个 token,解码速度约为每秒 75 个 token,但帖子摘录没有展示具体基准得分。
深度见解
该报告提供了本地运行 DeepSeek V4.1 Flash 的硬件、量化和吞吐量细节,但质量比较及 APEX 2025 结果仅来自一篇社区帖子,尚未得到独立验证。
| My main AI machine is a 6x3090 128GB DDR4 Threadripper 3960x system. I probably spend more time trying to optimize it than actually using it but it has been fun. I tried a bunch of saturated benchmarks like Terminal Bench 2.1, AIME 2026 and the performance was similar to unquantized DSV4.1 Flash 1017. I really wanted to understand my 2.9bpw quant's performance. It takes hours to run some of these benchmarks, so I went on the hunt for the perfect benchmark to get the most signal on quality of the model I'm running. I settled on APEX 2025 (https://huggingface.co/datasets/MathArena/apex\_2025). You can see how much of a beast this quant is. Trading blows with SOTA from 6 months ago. I get about 1000tps prefill, and 75tps decode with the 3090s power limited to 175watts Engine: Project Maya (https://github.com/mw00/project-maya) Quant: https://huggingface.co/LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw Hopefully this helps others in benchmarking their own setups. I really like the Apex 2025 benchmark. submitted by /u/hp1337 [link] [comments] |
#DeepSeek#DeepSeek V4.1 Flash 1017 EXL3 2.9bpw#Project Maya#APEX 2025