vLLM 自适应验证支持无置信头的草稿模型
vLLM 博客称，自适应验证（Adaptive Verification）此前的逐 token 接受概率估计依赖 DSpark 草稿模型自带的置信头，因此 EAGLE、MTP、DFlash 等缺少置信头的草稿模型无法使用该优化。新方案在缺少置信头时，用草稿模型 logits 中最可能 token 的 log-odds 作为特征，通过一个轻量逻辑回归在线拟合每个草稿位置的接受概率，参数随实际流量持续更新。该功能已在 PR #52228 落地，对无置信头的草稿模型通过 enable_adaptive_verification 开启，并用 DeepSeek-V4-Flash + DSpark 与 MiMo-V2.5-Pro-FP4 + DFlash 给出了对比结果。
据 vLLM 博客，现有草稿模型 checkpoint 无需重新训练即可获得自适应验证，把这项节省算力的优化扩展到 EAGLE、MTP 和 DFlash 等草稿模型。
TL;DR
Confidence scoring is now supported for all draft-model-based speculator types. In the absence of a confidence head, an online-fitted estimator derives a feature from the drafter's logits to predict the target model's probability of accepting it, which is used directly as a confidence score for Adaptive Verification. The estimator's coefficients are continuously fitted, adapting to the incoming data traffic. This feature landed in PR #52228, and is enabled with enable_adaptive_verification for draft models without a confidence head.
Background
vLLM's Adaptive Verification (AV) enables verification of a subset of the drafted tokens during speculative decoding. Draft tokens with low probability of acceptance are trimmed from the batch before running the target model's forward pass, saving on compute. This optimization is particularly beneficial while serving large batch sizes, because in that regime the GPU becomes compute-bound. See this vLLM blog post for more details.
#vLLM#Inferact#Red Hat#NVIDIA#DeepSeek-V4-Flash#DSpark#EAGLE#DFlash#MiMo-V2.5-Pro-FP4