Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Local 2.9bpw DeepSeek V4.1 Flash quantization tested

AI summary

Reddit user hp1337 reported testing a DeepSeek V4.1 Flash 1017 EXL3 2.9bpw quantization with the APEX 2025 benchmark on six power-limited RTX 3090 GPUs. Using Project Maya, the system reached about 1,000 tokens per second prefill and 75 tokens per second decode, though the post excerpt does not include the underlying benchmark score.

In-depth insight

The report provides concrete hardware, quantization and throughput details for running DeepSeek V4.1 Flash locally, but its quality comparison and APEX 2025 results remain unverified from a single community post.

My main AI machine is a 6x3090 128GB DDR4 Threadripper 3960x system. I probably spend more time trying to optimize it than actually using it but it has been fun. I tried a bunch of saturated benchmarks like Terminal Bench 2.1, AIME 2026 and the performance was similar to unquantized DSV4.1 Flash 1017. I really wanted to understand my 2.9bpw quant's performance. It takes hours to run some of these benchmarks, so I went on the hunt for the perfect benchmark to get the most signal on quality of the model I'm running. I settled on APEX 2025 (https://huggingface.co/datasets/MathArena/apex\_2025). You can see how much of a beast this quant is. Trading blows with SOTA from 6 months ago. I get about 1000tps prefill, and 75tps decode with the 3090s power limited to 175watts Engine: Project Maya (https://github.com/mw00/project-maya) Quant: https://huggingface.co/LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw Hopefully this helps others in benchmarking their own setups. I really like the Apex 2025 benchmark. submitted by /u/hp1337 [link] [comments]

#DeepSeek#DeepSeek V4.1 Flash 1017 EXL3 2.9bpw#Project Maya#APEX 2025

Original · 1 min readRead the original
Group chat

No comments yet. Start the conversation.