DeepSeek V4.1 Flash低位宽GGUF量化版
AI 总结
ISTA-DASLab 发布了 DeepSeek-V4.1-Flash-1017 的三种 GSQ-RCO GGUF 构建：IQ1_M 为 1.75 位、IQ2_XXS 为 2.00 位、IQ2_XS 为 2.25 位，主干文件大小为 120.7GB 至 155.2GB。发布帖称 IQ2_XXS 在所测基准中与基础模型持平，但社区讨论质疑基准分数已饱和，并指出没有提供独立的 KLD 或困惑度测试结果。据报道，这些模型可通过 Strata 在单张 12GB GPU 上运行，但因架构支持尚未合并，暂时无法在 llama.cpp 中运行。
深度见解
该发布为这款 763.2B 参数稀疏混合专家模型提供了低位宽本地构建，但其质量结论及兼容性仍依赖厂商测试和有限的运行时支持。
| We have released DeepSeek-V4.1-Flash-1017 quantized with GSQ and RCO, at 1.75 to 2.25 bits per weight. DeepSeek-V4.1-Flash-1017 is a sparse mixture-of-experts model: 384 routed experts per layer, of which 6 are active per token, 763.2B parameters and a 1M-token context window. It is released with FP8 dense weights and FP4 routed experts. What's inside Results At 2.00 bpw the model matches the base model on every benchmark evaluated. All numbers were measured at reasoning effort 50. Running it llama.cpp does not support DeepSeek-V4.1 yet, so the GGUFs do not run there for now. They run in Strata, which serves every build on a single NVIDIA GPU with 12 GB of VRAM, with a one-click installer and an OpenAI-compatible endpoint on localhost. Links The repository ships the complete per-tensor RCO allocation for every build (1008 tensors each). Feedback is welcome. Requests for models to quantize are also welcome. From the ISTA Deep Algorithms and Systems Lab. submitted by /u/Loginhe [link] [comments] |
#ISTA-DASLab#DeepSeek-V4.1-Flash-1017#GSQ#RCO#Strata#llama.cpp