Community dev ships DeepSeek V4.1 Flash 1017 quant for Dwarfstar with vision support
Reddit user /u/returnity released a quantized build of DeepSeek v4.1 Flash 1017 for Dwarfstar (ds4), noting that the language model already loads but the vision tower required their pull request to work. The quant follows the recipe antirez used for the original DS4.1 Q2 build: 2-bit routed experts with 8-bit or higher for everything else, plus the original FP8 engram table, with the imatrix built from the 1017 checkpoint rather than the original. The developer says the wired backbone is just over 150GB, so it streams from SSD on 128GB systems while the roughly 200GB engram table is paged in on demand, reaching decode speeds up to 20 tokens per second on an M5 Max.
The post offers a concrete quantization recipe and memory-footprint figures for running a large open-weights model on a 128GB Apple Silicon machine, though the speed figures are the author's own informal measurements.
| Hey y'all, just following up on my PR to get the new vision tower for 1017 working in ds4 with the quant I promised (the language model loads already, but vision needed my PR). I built to the exact same recipe antirez used for his original DS4.1 Q2: 2-bit routed experts with 8-bit or higher for everything else, combined with the original FP8 engram table. This worked well for the original DS4.1, and initial testing looks good here too. The imatrix is built from 1017, not the original checkpoint. The wired backbone of the model is just over 150GB, so it will stream from SSD on 128GB systems with nearly all of the routed experts held in the hot cache, allowing for decode speeds up to 20tps on an M5 Max. The ~200GB engram table is paged in from SSD on-demand, not held in RAM. This model is significantly revamped, with ~40% of the routed expert tensors changed, 30% of engrams, and ~50% of the vision tower's expert tensors altered. By all appearances, it's the new leading open-weights model, so I had to give it a try. If you are curious too, let me know what you think! submitted by /u/returnity [link] [comments] |
#DeepSeek#DeepSeek V4.1 Flash#Dwarfstar#ds4#antirez#Apple M5 Max