Unsloth releases preview GGUFs for DeepSeek-V4.1-Flash-1017
AI summary
Unsloth released preview Dynamic v3.1 GGUF quantizations for DeepSeek-V4.1-Flash-1017 and said they can run on devices with 128GB of RAM or unified memory, or with 64GB of RAM using engram offloading. The vendor reported stronger results than ISTA's GSQ-RCO GGUFs on nearly all listed metrics across seven benchmarks, but said it could not reproduce ISTA's published AIME25 result. Unsloth averaged at least three runs per benchmark, with 16 runs for AIME25.
In-depth insight
The release gives local-model users additional quantization options for DeepSeek-V4.1-Flash-1017, while the published comparisons provide context for choosing among GGUFs.
| Hey everyone! We’re releasing new DeepSeek-V4.1-1017 Dynamic V3.1 (preview) GGUFs which showcases how Unsloth’s GGUFs retain the most accuracy for its size across 7 different benchmarks. We benchmarked ISTA's GSQ-RCO GGUFs but could not reproduce their reported results, even when using the highest scores. The biggest discrepancy was on AIME25 where they claimed 100% accuracy recovery to baseline, but was only ~91.88% in benchmark tests. Across AIME25, LiveCodeBench, GPQA Diamond, Aider Polyglot, and more, Unsloth Dynamic V3.1 outperforms GSQ-RCO GGUFs on nearly all metrics. Even our old v3 does better on nearly all metrics than ISTA. We ran each benchmark at least 3 times (AIME25 was 16 times) and averaged the results to ensure a fair, reliable comparison. More results coming next week! All benchmark data, methodologies and results are in our blog. Separately, for KLD, there were queries whether it’s a reliable measure of accuracy. The primary goal of quantization is to minimize KLD but also minimize disk space. If you get a KLD of 0, this means your quantized model == 16bit model, as logits are perfectly modelled. Our benchmarks show on KLD and top-1% across multiple domains (coding, agentic coding, chat etc), UDv3.1 does much better than GSQ-RCO across every domain. We show that lower bit quants do think somewhat longer than the unquantized model - we plan to investigate and make quants even better. However, we're still much ahead vs GSQ-RCO on even reasoning length as well, and provide plots for truncated answers in our blog. The model will be able to run on 128GB RAM/unified memory devices or even on 64GB RAM with engram offloading. Blog with all details and more benchmarks: https://unsloth.ai/docs/models/deepseek-v4.1-flash GGUF: https://huggingface.co/unsloth/DeepSeek-V4.1-Flash-1017-GGUF We also introduced lots of new things to Unsloth Desktop including Browser Use, auto compaction, a whole new TTS pipeline (with voice cloning, separation etc), sandboxing, and background customization and more: https://github.com/unslothai/unsloth Hope you all have a Happy Halloween! Also table of numbers below which is also described with more plots and data in our blog: submitted by /u/danielhanchen [link] [comments] |