vLLM Adaptive Verification works without a confidence head via online estimator
A vLLM blog post says Adaptive Verification previously required a per-token acceptance estimate, so it only supported DSpark drafters that ship with a confidence head, leaving EAGLE, MTP, DFlash and some DSpark checkpoints unable to use the optimization. The new approach, when no confidence head exists, derives a feature from the log-odds of the drafter's most likely token and fits a lightweight logistic regression online, with one shared slope and one intercept per draft position, continuously updated from the target model's accept/reject decisions. The feature landed in PR #52228 and is enabled with enable_adaptive_verification for draft models without a confidence head, with results reported for DeepSeek-V4-Flash + DSpark and MiMo-V2.5-Pro-FP4 + DFlash.
According to the vLLM blog, any existing draft checkpoint gets Adaptive Verification without retraining, extending the compute-saving optimization to drafters such as EAGLE, MTP and DFlash.
TL;DR
Confidence scoring is now supported for all draft-model-based speculator types. In the absence of a confidence head, an online-fitted estimator derives a feature from the drafter's logits to predict the target model's probability of accepting it, which is used directly as a confidence score for Adaptive Verification. The estimator's coefficients are continuously fitted, adapting to the incoming data traffic. This feature landed in PR #52228, and is enabled with enable_adaptive_verification for draft models without a confidence head.
Background
vLLM's Adaptive Verification (AV) enables verification of a subset of the drafted tokens during speculative decoding. Draft tokens with low probability of acceptance are trimmed from the batch before running the target model's forward pass, saving on compute. This optimization is particularly beneficial while serving large batch sizes, because in that regime the GPU becomes compute-bound. See this vLLM blog post for more details.
#vLLM#Inferact#Red Hat#NVIDIA#DeepSeek-V4-Flash#DSpark#EAGLE#DFlash#MiMo-V2.5-Pro-FP4