Reddit 用户实测 Unsloth 量化版 Qwen Flash-Next 仅需 12GB 显存
AI 总结
r/LocalLLaMA 一篇帖子称,Unsloth 为 Qwen Flash-Next 和 Qwen-27B 推出的 Dynamic v3.1 量化版本,配合 ISTA-DASLab 的高级量化,让模型更易获取且质量保持良好。作者称使用 Unsloth 的 Iq2_XS 量化版 Qwen Flash-Next,通过 Strata 引擎在 12GB 显存与 32GB 内存上运行,1K prefill、平均约 50 tokens/s 解码,覆盖 131K 上下文窗口,Strata 默认自动调优下质量损失很小。帖子还提到目前已有面向 8GB 显存与 32GB 内存的配置,并称 Qwen 4 预计于当周发布。
深度见解
针对 Qwen Flash-Next 的重度量化版在消费级硬件上的实测数据表明,前沿级本地推理正以更低门槛落地,可能让更多用户在自有机器上运行能力较强的模型。
| Hey r/LocalLLaMA ! We're releasing new new Unsloth Dynamic v3.1 GGUFs for Qwen3.8-27B! They're better by 5-10% accuracy on DeepSWE, Terminal Bench and more vs UD3, GSQ-RCO and Exllama-3! We're also improving our algorithms further so we can apply them to Qwen4 so folks can have efficient and accurate quants! On other benchmarks like for GSQ-RCO and Exllama3: More details and benchmarks at https://unsloth.ai/docs/models/qwen3.8#dynamic-v3.1-update-november-11th-2026 All quants are updated in https://huggingface.co/unsloth/Qwen3.8-27B-GGUF! Thank you and have a nice day! We're super excited for Qwen4! submitted by /u/danielhanchen [link] [comments] |