ISTA-DASLab releases 1.75–2.25-bit DeepSeek-V4.1-Flash GGUFs
ISTA-DASLab announced three GSQ-RCO GGUF builds of DeepSeek-V4.1-Flash-1017: IQ1_M at 1.75 bits per weight, IQ2_XXS at 2.00 bpw, and IQ2_XS at 2.25 bpw, with backbones ranging from 120.7GB to 155.2GB. The release post says IQ2_XXS matched the base model on the evaluated benchmarks at reasoning effort 50, while a community discussion questioned the saturated benchmarks and noted that no independent KLD or perplexity measurements were supplied. The models reportedly run through Strata on a single 12GB GPU but cannot yet run in llama.cpp because architecture support has not been merged.
These releases offer low-bit local builds of a 763.2B-parameter sparse mixture-of-experts model, but their quality claims and compatibility remain dependent on vendor testing and narrow runtime support.
| We have released DeepSeek-V4.1-Flash-1017 quantized with GSQ and RCO, at 1.75 to 2.25 bits per weight. DeepSeek-V4.1-Flash-1017 is a sparse mixture-of-experts model: 384 routed experts per layer, of which 6 are active per token, 763.2B parameters and a 1M-token context window. It is released with FP8 dense weights and FP4 routed experts. What's inside Results At 2.00 bpw the model matches the base model on every benchmark evaluated. All numbers were measured at reasoning effort 50. Running it llama.cpp does not support DeepSeek-V4.1 yet, so the GGUFs do not run there for now. They run in Strata, which serves every build on a single NVIDIA GPU with 12 GB of VRAM, with a one-click installer and an OpenAI-compatible endpoint on localhost. Links The repository ships the complete per-tensor RCO allocation for every build (1008 tensors each). Feedback is welcome. Requests for models to quantize are also welcome. From the ISTA Deep Algorithms and Systems Lab. submitted by /u/Loginhe [link] [comments] |
#ISTA-DASLab#DeepSeek-V4.1-Flash-1017#GSQ#RCO#Strata#llama.cpp