Basalt 0.2.0 adds a new NVFP4-DQ8 build and faster Qwen 3.8 Flash Next inference
The Basalt developer posted on r/LocalLLaMA that version 0.2.0 of the Blackwell-only inference engine for Qwen 3.8 Flash Next adds a new NVFP4-DQ8 quantization build on Hugging Face, with KLD of 0.0561 versus 0.0624 for the default NVFP4-DQ6_K build at roughly 7% slower prose throughput. Quality numbers are measured against Unsloth's BF16 GGUF served through llama.cpp (reference PPL 3.2749) and scored with basalt-bench --quality, with Basalt Q8 at 0.0200 KLD and 95.34% same top-1. On a rig with an RTX 5090 and RTX 5060 Ti 16GB plus 128GB DDR5 RAM, the developer reports about 603.7 to 594.2 tokens per second for structured output, 295.0 to 299.9 tokens per second for prose, and roughly 10,221 to 10,229 tokens per second for 64K prefill. The .basalt file format is now version 2, and older files can be migrated rather than redownloaded.
The release gives local-model users concrete deployment data — measured quantization quality plus hundreds of tokens per second of structured output on consumer Blackwell cards — while the new DQ8 build offers a quality-versus-speed tradeoff and the format migration avoids re-downloading existing weight files.
#Basalt#Qwen 3.8 Flash Next#Unsloth#llama.cpp#basalt-bench#Hugging Face#NVIDIA Blackwell#RTX 5090#RTX 5060 Ti