网友称Qwen3.8-Flash-Next低比特量化可单卡5090跑通
AI 总结
r/LocalLLaMA 用户 u/Unstable_Llama 发帖称，Qwen3.8-Flash-Next 的 1.66bpw exl3 量化版本占用约 36GB 以内，可完全放进单张 RTX 5090，且无需卸载专家到内存。该用户称推理过程稳定、没有出现推理死循环，速度约为每秒 200–300 token，但也指出单张 32GB 显卡留给上下文的空间依然紧张，即使启用量化缓存也需要频繁压缩上下文。帖子称权重已发布在 Hugging Face 的 turboderp/Qwen3.8-Flash-Next-exl3 下；上述性能数据均为发帖人自述，尚未获得独立验证。
深度见解
若情况属实，这意味着 Qwen Flash 级别模型可以以低于 2 比特的形式在单张消费级显卡上本地运行，但发帖人表示上下文容量仍是主要限制。
| Fits entirely in a 5090, no offloaded experts. Super-solid sub-2bpw! (Liquid cat demo video generated by the quantized model) From the Cat's mouth: Stable inference throughout, no reasoning loops, just smooth sailing at 200-300 t/s (and a lot of compacting since one 32 GB GPU still doesn't leave a lot of room for context, even with cache quantization. https://huggingface.co/turboderp/Qwen3.8-Flash-Next-exl3 submitted by /u/Unstable_Llama [link] [comments] |
#Qwen3.8-Flash-Next#Qwen#Hugging Face#RTX 5090#exl3#turboderp