DeepSeek V4.1 Flash 2.6比特量化：双3090达26 t/s
r/LocalLLaMA 用户 whiteh4cker 称，基于 lxe 的 dsx 引擎制作了 2.6 比特量化的 DeepSeek-V4.1-Flash-1017-dsx-2.6bit，面向编程与智能体任务，并计划在发帖后两小时内上传全部权重。该帖自报在双 RTX 3090、192 GB DDR5 的机器上，2.6127 bpw 版本相对 FP8/FP4 教师模型取得 82.59% Top-1 与 9.3626 困惑度，在 262,144 上下文加 Engram 表时生成速度 26 t/s、提示处理 170 t/s。作者还称该引擎暂不支持图像，187.6 GB 权重从机械硬盘加载需 90–110 分钟。
按作者自述，这一构建让 187.6 GB 的模型在两张 24 GB 消费级显卡上运行并支持 262,144 长度上下文。
Using u/lxe's https://github.com/lxe/dsx engine, I created DeepSeek-V4.1-Flash-1017-dsx-2.6bit for programming and agentic tasks. The model's weight will be fully uploaded in 2 hours.
| Config | Experts | KL ↓ | Top-1 ↑ | Perplexity ↓ |
|---|---|---|---|---|
| FP8 / FP4 teacher | FP4 | 0 | 100% | 8.56 |
| Lucebox ROCMFP | 2.5 bpw | 0.253 | 78.7% | 9.78 |
| Author's 2.03 bpw build | 2.03 bpw | 0.462 | 71.5% | 11.66 |
| This model | 2.6127 bpw | 0.1972 | 82.59% | 9.3626 |
The engine doesn't seem to support images yet. It takes 90-110 minutes to load the 187.6 GB weight from an HDD. At 262,144 context with Engram tables (203 GB) on a SATA SSD, I got 26 t/s generation speed and 170 t/s prompt processing speed.
I use https://github.com/DominikBucko/qwen38-flash-next-2x3090 daily, but I am experimenting with DeepSeek V4.1 Flash.
My hardware: 2x RTX 3090, Intel Ultra 7 270k Plus, 192 GB DDR5@5600 MHz
submitted by /u/whiteh4cker [link] [comments]