Qwen3.8本地推理引擎与成本实测汇总
r/LocalLLaMA 用户 FeydRowan 汇总了 Qwen3.8-27B 与 125B MoE 模型 Qwen3.8-Flash-Next 在消费级硬件上的单请求解码速度，并把 LocalMaxxing 验证运行或同机配对运行标为“已验证”。作者自己的 capy-flash 数据在 2× RTX 5070 Ti、96 GB DDR5 上测得，其余多数数字来自各项目作者报告；帖子还比较了张量并行与分层切分、专家权重放在显存外、量化格式和草稿模型的影响，并按欧盟含税价计算每千欧元吞吐。其中 2× R9700 在 Flash-Next 上性价比居前，96 GB DDR5 约占相关整机成本四成，但这些性能与价格多为个人整理和自报，需谨慎看待。
帖子把分散的本地推理引擎数据按统一口径和硬件成本放在一起，便于本地部署者比较 Qwen3.8 系列在消费级设备上的实际吞吐与每欧元性价比，尽管多数数字未独立验证。
I've been keeping a list of the engines that run Qwen3.8-27B and Flash-Next on consumer hardware, mostly to see where my own engine (capy-flash) stands. Sharing it because most numbers going around aren't comparable. Our rows are measured on my box: 2× RTX 5070 Ti, 96 GB DDR5.
"Verified" means a verified LocalMaxxing run or a paired run on the same machine; everything else is what the author reports. Decode is tok/s, one request.
What actually makes the difference
- Tensor parallel vs pipeline. TP (capy-flash, SGLang, vLLM, ExLlamaV3, Paiton on AMD) adds up the bandwidth of two cards for one request. Layer split (Strata, llama.cpp) adds memory, not speed.
- Experts out of VRAM (Flash-Next). Hot experts in VRAM, misses computed on CPU (Strata, capy-flash, Infernix, fngine). What counts is how many experts miss per step.
- Quant format. NVFP4 is better quality but more bytes. 2–3-bit IQ GGUF is faster at about twice the KL (Strata's numbers: 0.144 vs 0.070).
- Drafters. MTP gives 1.3–1.7×. On the 27B, DFlash2 and DSpark give 2–3×: that's where most 400+ tok/s headlines come from. A DFlash2 drafter for Flash-Next came out today (incoai), so far only run on SGLang.
- Single-model kernels (MegaCapybara, NInfer, capy-flash) beat general engines by 2–6× at batch 1. vLLM and SGLang win once you have many users.
#Qwen3.8-27B#Qwen3.8-Flash-Next#capy-flash#SGLang#vLLM#llama.cpp#ExLlamaV3#Strata#NVIDIA GeForce RTX 5090#NVIDIA GeForce RTX 5070 Ti#AMD Radeon R9700#DGX Spark