SpecTr-GBV unifies multi-draft speculation with block verification
Researchers from Renmin University of China's Center for Applied Statistics and the China Medical University's School of Intelligent Medicine propose SpecTr-GBV, which folds multi-draft speculative decoding and greedy block verification (GBV) into one framework, per Machine Heart. The method casts verification of multiple draft token blocks against the target token block as an optimal transport problem, seeking a longer expected acceptance length while preserving output-distribution consistency. The authors prove that, within their conditionally independent couplings class, SpecTr-GBV attains the optimal expected acceptance length for a fixed draft count K, that the bound rises with K and reduces to GBV at K=1. Experiments on HumanEval, GSM8K, MGSM, LM1B and Alpaca with DeepSeek-33B/1.3B and DeepSeek-6.7B/1.3B target-draft pairs report higher block efficiency and speedup; the article notes that wall-clock speedup is typically below block efficiency, that speedup first rises then falls as draft length grows, and that the current theory covers i.i.d. draft sequences but not tree-structured candidates.
The authors argue that redesigning verification alone, without changing the draft model, can extract more from the same candidate tokens, offering a route to faster speculative decoding distinct from compression, distillation or a stronger draft model.
机器之心 2026-10-15 12:17 北京
SpecTr-GBV，将 multi-draft 与 greedy block verification（GBV）统一进同一个推测解码框架。
大语言模型的生成过程天然具有自回归特性，即模型基于已有上下文逐 token 预测并生成后续内容。随着模型规模不断增大，单步前向计算的开销也随之增加，使得自回归解码面临较高的推理延迟。推测解码（Speculative Decoding, SD）通过引入一个更轻量的 draft model 预先生成候选 token，再由 target model 对这些候选进行并行验证，从而减少昂贵的 target model 前向计算次数，已成为加速大语言模型推理的重要技术路线。
#SpecTr-GBV#SpecTr#greedy block verification#DeepSeek-33B#DeepSeek-6.7B#Renmin University of China#China Medical University#EMNLP 2026