社区测试：DFlash2 让 Qwen3.8-Flash-Next 解码提速 17%
AI 总结
r/LocalLLaMA 的一名测试者称，llama.cpp v0.8.0 是首个支持 Qwen3.8-Flash-Next 的 DFlash2 草稿模型的版本，他自行量化了 drafter 的 GGUF 后进行了测试。在 Apple M5 Max 上使用 Q4_K_XL，DFlash2 的几何平均解码速度为 87.9 tps，MTP 为 75.1 tps，提升 17%；其中代码提示 95.1 tps（+18%）、列表提示 127 tps（+39%），但散文提示略降为 51.9 tps（-3%）。在约 25K 上下文长度下 DFlash2 大致与 MTP 持平并出现一定回落；测试者还提到其 Q4_0 草稿头仅约 350MB，而 ggml MTP 头为 2.2GB，但所需计算缓冲抵消了显存节省。
深度见解
推测解码提速是本地模型用户可直接受益的环节，且报告给出了硬件、量化与参数设置供他人复测，尽管结果随提示类型变化且仅来自单一测试者。
| Not long ago, DFlash2 for Qwen3.8-Flash-Next was released, but initially llama.cpp compatibility wasn't there. Now it is. I quantized my own GGUFs of the drafter and tested it out in llama.cpp v0.8.0, the first release that supports DFlash2 for FN. Speed: Runs were conducted after an extensive sweep of parameters for MTP and DFlash2 to find optimal settings for both drafters on Apple Silicon (M5 Max). Your settings will likely be different, but mine were n-max = 3, p-min = 0.75 for MTP and n-max = 5, p-min = 0.75 for DFlash2. At longer context (25K), DFlash2 roughly equals MTP, so there is some drop-off whereas MTP stays nearly flat. Acceptance was compared between the Q4_0 draft head and Q8_0, and there was no measurable difference in either speed or acceptance, so I'd recommend choosing Q4_0 for the memory savings. Speaking of which, in addition to being significantly faster for all categories except prose, DFlash2's draft heads are TINY! The Q4_0 head is only ~350MB, compared to 2.2GB for the Q4_0 ggml MTP head I was using. However, these savings don't translate into VRAM, because the compute buffer required for DFlash2 basically equals it back out. Finally, unlike DFlash2 for 3.8 27B, images work! The draft models drafts with vision enabled and test image prompts went through and got coherent responses with good acceptance. Overall, DFlash2 is a significant upgrade in decode performance for every type of prompt except prose, where the regression is minor and within noise. submitted by /u/returnity [link] [comments] |
#DFlash2#Qwen3.8-Flash-Next#llama.cpp v0.8.0#MTP#Apple M5 Max