DeepSeek V4.1 Flash 社区量化版支持视觉
AI 总结
r/LocalLLaMA 用户 /u/returnity 发布了适配 Dwarfstar（ds4）的 DeepSeek v4.1 Flash 1017 量化版本，此前 ds4 已能加载语言模型部分，视觉塔则需依靠其提交的 PR 才能运行。该量化沿用 antirez 为原版 DS4.1 使用的配方：路由专家采用 2-bit、其余组件 8-bit 或更高，并沿用原始 FP8 engram 表；imatrix 则基于 1017 检查点构建。作者称模型的连线骨干略超 150GB，可在 128GB 内存机器上从 SSD 流式加载，约 200GB 的 engram 表按需分页，在 M5 Max 上解码速度可达 20 tokens/s。
深度见解
该帖给出了一份可复现的量化配方与显存/存储占用数据，说明这类开放权重模型能在 128GB 的 Apple Silicon 机器上以流式方式运行，但速度数据为作者自测。
| Hey y'all, just following up on my PR to get the new vision tower for 1017 working in ds4 with the quant I promised (the language model loads already, but vision needed my PR). I built to the exact same recipe antirez used for his original DS4.1 Q2: 2-bit routed experts with 8-bit or higher for everything else, combined with the original FP8 engram table. This worked well for the original DS4.1, and initial testing looks good here too. The imatrix is built from 1017, not the original checkpoint. The wired backbone of the model is just over 150GB, so it will stream from SSD on 128GB systems with nearly all of the routed experts held in the hot cache, allowing for decode speeds up to 20tps on an M5 Max. The ~200GB engram table is paged in from SSD on-demand, not held in RAM. This model is significantly revamped, with ~40% of the routed expert tensors changed, 30% of engrams, and ~50% of the vision tower's expert tensors altered. By all appearances, it's the new leading open-weights model, so I had to give it a try. If you are curious too, let me know what you think! submitted by /u/returnity [link] [comments] |
#DeepSeek#DeepSeek V4.1 Flash#Dwarfstar#ds4#antirez#Apple M5 Max