Lamina

Headlines · 1

1. 5d
   Qwen3.8 Flash-Next guide posts high local inference speeds amid quantization dispute

   A community guide reports that Basalt, Unsloth's UD-IQ3_XXS quantization, and the Lamina drafter delivered 195.6 tokens/s average decode and about 5,700 tokens/s prompt reads for Qwen3.8 Flash-Next in an OpenCode session on an RTX 5080. The author says Basalt reached roughly five times llama.cpp's drafted decode speed and seven times its prompt-read speed, while Unsloth's quantization achieved 3.247 WikiText perplexity versus BF16's 3.273. A separate Reddit post says Unsloth disputed the reproducibility of ISTA-DASLab's GSQ-RCO benchmark numbers and found similarly sized Unsloth quantizations performing better in its testing.

   r/LocalLLaMA · 🔥 0

Experience and discussion from the community

Share my Lamina experienceAsk about Lamina