AlayaNeW open-sources Alaya-DFlash2, claiming 3.41x GLM-5.3 decode speedup
AlayaNeW (九章云极) released and open-sourced Alaya-DFlash2, a speculative-decoding acceleration scheme built for GLM-5.3 that pairs a draft model able to propose candidates in parallel with a path-selection mechanism. In its own evaluation at TP=8 on 8×H200 with single concurrency, decode speed was about 288.2 tokens/s versus 84.5 tokens/s for native autoregressive decoding, a 3.41x gain, with total output throughput rising from 78.95 to 236.47 tokens/s. The report also says acceptance length exceeded IncoAI-DFlash2 and RedHat-DSpark across 1k–32k low-entropy contexts, and that at 64-way concurrency throughput reached 608.71 tokens/s, 44.9% above native decoding.
The vendor says the scheme raises both single-user decode speed and aggregate output throughput for long-context and higher-concurrency serving, offering a measurable acceleration path for GLM-5.3 deployments; the figures come from its own evaluation and cover only the tested load range.
长文本的等待，发生在输出阶段。
32k 的文档读进去了，模型开始回答。第一个字很快，后面每个字都要等——主模型一次只往前一步，每生成一个 token，都要把整段上下文重新算一遍。上下文越长，这一步越贵。
推测解码换了个做法：让一个更小的草稿模型先往后猜几步，主模型一次性验证。猜中的部分直接接收，一轮就能推进好几个 token。猜得准不准，用一个指标衡量——接受长度，即每次验证平均能通过多少个候选。
Alaya-DFlash2 是九章云极为 GLM-5.3 打造的一套推测解码加速方案：一个能并行起草的草稿模型，加一套路径选择机制，让它在长上下文里依然猜得准。评测显示，它在长上下文下保持了较高的草稿接受长度，并在从单并发到 64 并发的全部已测档位中，同时取得高于原生自回归解码的单用户解码速度与总输出吞吐。
#九章云极 (AlayaNeW)#GLM-5.3#Alaya-DFlash2#AlayaNeW 智算引擎#IncoAI-DFlash2#RedHat-DSpark#Nvidia H200