Qwen3.8-27B and Qwen3.8-Flash-Next local inference benchmark roundup
An r/LocalLLaMA user, FeydRowan, compiled single-request decode speeds for Qwen3.8-27B and the 125B MoE Qwen3.8-Flash-Next on consumer hardware, marking rows as “verified” only when backed by a LocalMaxxing run or a paired run on the same machine. The author's own capy-flash rows were measured on 2× RTX 5070 Ti with 96 GB DDR5, while most other figures are as reported by each project; the post also compares tensor parallelism versus layer splitting, expert offload, quant formats and drafters, and calculates tokens per second per €1,000 using EU VAT-inclusive prices. It puts 2× R9700 on top for Flash-Next cost-efficiency and says 96 GB of DDR5 is about 40% of the bill for such builds, but the performance and price data are largely user-compiled and self-reported.
It consolidates scattered local-inference engine results under one methodology and adds hardware-cost normalization, giving local deployers a comparable view of Qwen3.8 throughput and price-performance on consumer devices, though most figures are not independently verified.
I've been keeping a list of the engines that run Qwen3.8-27B and Flash-Next on consumer hardware, mostly to see where my own engine (capy-flash) stands. Sharing it because most numbers going around aren't comparable. Our rows are measured on my box: 2× RTX 5070 Ti, 96 GB DDR5.
"Verified" means a verified LocalMaxxing run or a paired run on the same machine; everything else is what the author reports. Decode is tok/s, one request.
What actually makes the difference
- Tensor parallel vs pipeline. TP (capy-flash, SGLang, vLLM, ExLlamaV3, Paiton on AMD) adds up the bandwidth of two cards for one request. Layer split (Strata, llama.cpp) adds memory, not speed.
- Experts out of VRAM (Flash-Next). Hot experts in VRAM, misses computed on CPU (Strata, capy-flash, Infernix, fngine). What counts is how many experts miss per step.
- Quant format. NVFP4 is better quality but more bytes. 2–3-bit IQ GGUF is faster at about twice the KL (Strata's numbers: 0.144 vs 0.070).
- Drafters. MTP gives 1.3–1.7×. On the 27B, DFlash2 and DSpark give 2–3×: that's where most 400+ tok/s headlines come from. A DFlash2 drafter for Flash-Next came out today (incoai), so far only run on SGLang.
- Single-model kernels (MegaCapybara, NInfer, capy-flash) beat general engines by 2–6× at batch 1. vLLM and SGLang win once you have many users.
#Qwen3.8-27B#Qwen3.8-Flash-Next#capy-flash#SGLang#vLLM#llama.cpp#ExLlamaV3#Strata#NVIDIA GeForce RTX 5090#NVIDIA GeForce RTX 5070 Ti#AMD Radeon R9700#DGX Spark