DeepSeek V4.1 Flash 2.61-bit quant runs at 26 t/s on dual RTX 3090s
Reddit user whiteh4cker (r/LocalLLaMA) says they built DeepSeek-V4.1-Flash-1017-dsx-2.6bit, a 2.6-bit quantization made with lxe's dsx engine for programming and agentic tasks, and that the full weights would be uploaded within two hours of the post. The post self-reports that the 2.6127 bpw build scored 82.59% top-1 and 9.3626 perplexity against the FP8/FP4 teacher model, and reached 26 t/s generation and 170 t/s prompt processing at 262,144 context with Engram tables on two RTX 3090s with 192 GB DDR5. The author also notes the engine does not yet support images and that loading the 187.6 GB weight from an HDD takes 90–110 minutes.
According to the author's own test, the build fits a 187.6 GB model across two 24 GB consumer GPUs while serving a 262,144-token context.
Using u/lxe's https://github.com/lxe/dsx engine, I created DeepSeek-V4.1-Flash-1017-dsx-2.6bit for programming and agentic tasks. The model's weight will be fully uploaded in 2 hours.
| Config | Experts | KL ↓ | Top-1 ↑ | Perplexity ↓ |
|---|---|---|---|---|
| FP8 / FP4 teacher | FP4 | 0 | 100% | 8.56 |
| Lucebox ROCMFP | 2.5 bpw | 0.253 | 78.7% | 9.78 |
| Author's 2.03 bpw build | 2.03 bpw | 0.462 | 71.5% | 11.66 |
| This model | 2.6127 bpw | 0.1972 | 82.59% | 9.3626 |
The engine doesn't seem to support images yet. It takes 90-110 minutes to load the 187.6 GB weight from an HDD. At 262,144 context with Engram tables (203 GB) on a SATA SSD, I got 26 t/s generation speed and 170 t/s prompt processing speed.
I use https://github.com/DominikBucko/qwen38-flash-next-2x3090 daily, but I am experimenting with DeepSeek V4.1 Flash.
My hardware: 2x RTX 3090, Intel Ultra 7 270k Plus, 192 GB DDR5@5600 MHz
submitted by /u/whiteh4cker [link] [comments]