Reddit user tests Unsloth Qwen Flash-Next quants at 12 GB VRAM
A r/LocalLLaMA post says Unsloth's new Dynamic v3.1 quantizations for Qwen Flash-Next and Qwen-27B, alongside ISTA-DASLab's advanced quantizations, make the models more accessible while keeping quality high. The author reports testing Unsloth's Iq2_XS quantization for Qwen Flash-Next with the Strata engine, running on 12 GB VRAM and 32 GB RAM at 1K prefill and about 50 tokens per second average decode across a 131K context window, with minimal quality loss under Strata's default auto-tuner. The post also says configurations targeting 8 GB VRAM with 32 GB system RAM are now in view, and that Qwen 4 was expected the same week.
Concrete community measurements for heavily quantized Qwen Flash-Next builds show frontier-adjacent local inference running on modest consumer hardware, which could widen who can run capable models on their own machines.
| Hey r/LocalLLaMA ! We're releasing new new Unsloth Dynamic v3.1 GGUFs for Qwen3.8-27B! They're better by 5-10% accuracy on DeepSWE, Terminal Bench and more vs UD3, GSQ-RCO and Exllama-3! We're also improving our algorithms further so we can apply them to Qwen4 so folks can have efficient and accurate quants! On other benchmarks like for GSQ-RCO and Exllama3: More details and benchmarks at https://unsloth.ai/docs/models/qwen3.8#dynamic-v3.1-update-november-11th-2026 All quants are updated in https://huggingface.co/unsloth/Qwen3.8-27B-GGUF! Thank you and have a nice day! We're super excited for Qwen4! submitted by /u/danielhanchen [link] [comments] |