Unsloth发布DeepSeek-V4.1 Flash预览GGUF
AI 总结
Unsloth发布了DeepSeek-V4.1-Flash-1017的Dynamic v3.1预览版GGUF量化模型，并称其可在配备128GB内存或统一内存的设备上运行，也可通过engram卸载以使用64GB内存。该厂商报告称，其版本在七项基准测试的多数指标上优于ISTA的GSQ-RCO GGUF，但未能复现ISTA公布的AIME25结果。Unsloth表示每项基准测试至少运行三次并取平均值，其中AIME25运行了16次。
深度见解
此次发布为DeepSeek-V4.1-Flash-1017本地部署用户增加了量化选项，同时公布的对比数据可帮助用户在不同GGUF版本之间进行选择。
| Hey everyone! We’re releasing new DeepSeek-V4.1-1017 Dynamic V3.1 (preview) GGUFs which showcases how Unsloth’s GGUFs retain the most accuracy for its size across 7 different benchmarks. We benchmarked ISTA's GSQ-RCO GGUFs but could not reproduce their reported results, even when using the highest scores. The biggest discrepancy was on AIME25 where they claimed 100% accuracy recovery to baseline, but was only ~91.88% in benchmark tests. Across AIME25, LiveCodeBench, GPQA Diamond, Aider Polyglot, and more, Unsloth Dynamic V3.1 outperforms GSQ-RCO GGUFs on nearly all metrics. Even our old v3 does better on nearly all metrics than ISTA. We ran each benchmark at least 3 times (AIME25 was 16 times) and averaged the results to ensure a fair, reliable comparison. More results coming next week! All benchmark data, methodologies and results are in our blog. Separately, for KLD, there were queries whether it’s a reliable measure of accuracy. The primary goal of quantization is to minimize KLD but also minimize disk space. If you get a KLD of 0, this means your quantized model == 16bit model, as logits are perfectly modelled. Our benchmarks show on KLD and top-1% across multiple domains (coding, agentic coding, chat etc), UDv3.1 does much better than GSQ-RCO across every domain. We show that lower bit quants do think somewhat longer than the unquantized model - we plan to investigate and make quants even better. However, we're still much ahead vs GSQ-RCO on even reasoning length as well, and provide plots for truncated answers in our blog. The model will be able to run on 128GB RAM/unified memory devices or even on 64GB RAM with engram offloading. Blog with all details and more benchmarks: https://unsloth.ai/docs/models/deepseek-v4.1-flash GGUF: https://huggingface.co/unsloth/DeepSeek-V4.1-Flash-1017-GGUF We also introduced lots of new things to Unsloth Desktop including Browser Use, auto compaction, a whole new TTS pipeline (with voice cloning, separation etc), sandboxing, and background customization and more: https://github.com/unslothai/unsloth Hope you all have a Happy Halloween! Also table of numbers below which is also described with more plots and data in our blog: submitted by /u/danielhanchen [link] [comments] |