greedy block verification

Headlines · 1

1. 4d
   SpecTr-GBV unifies multi-draft speculation with block verification

   Researchers from Renmin University of China's Center for Applied Statistics and the China Medical University's School of Intelligent Medicine propose SpecTr-GBV, which folds multi-draft speculative decoding and greedy block verification (GBV) into one framework, per Machine Heart. The method casts verification of multiple draft token blocks against the target token block as an optimal transport problem, seeking a longer expected acceptance length while preserving output-distribution consistency. The authors prove that, within their conditionally independent couplings class, SpecTr-GBV attains the optimal expected acceptance length for a fixed draft count K, that the bound rises with K and reduces to GBV at K=1. Experiments on HumanEval, GSM8K, MGSM, LM1B and Alpaca with DeepSeek-33B/1.3B and DeepSeek-6.7B/1.3B target-draft pairs report higher block efficiency and speedup; the article notes that wall-clock speedup is typically below block efficiency, that speedup first rises then falls as draft length grows, and that the current theory covers i.i.d. draft sequences but not tree-structured candidates.

   机器之心 · 🔥 0

Experience and discussion from the community

Share my greedy block verification experienceAsk about greedy block verification