Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

NarrowSpec enables narrower speculative-decoding drafts

AI summary

A paper submitted to arXiv on November 18, 2026 proposes that speculative-decoding draft models need not have the same hidden width as their target models. The authors report that DFlash and EAGLE-3 drafts using half or one-quarter of the target width largely preserve token acceptance. In tests with Qwen3-8B (NVFP4) and GPT-OSS-20B (MXFP4) running in SGLang, the authors report end-to-end decoding speedups of 1.06–1.12x on an NVIDIA B200 and 1.13–1.44x on a DGX Spark (GB10) at batch size 1 and temperature 0.

In-depth insight

The reported results indicate that narrower draft models can improve inference efficiency while largely preserving acceptance in the tested configurations. The findings are currently author-reported results from a newly submitted preprint and have not been independently verified in the supplied source.

Computer Science > Computation and Language

arXiv:2611.23835 (cs)

[Submitted on 18 Nov 2026]

Title:NarrowSpec: Draft Hidden Width Need Not Match the Target in Speculative Decoding

Authors:Alagappan Valliappan, Ankit More, Dheevatsa Mudigere, Jatin Mitra

View a PDF of the paper titled NarrowSpec: Draft Hidden Width Need Not Match the Target in Speculative Decoding, by Alagappan Valliappan and 3 other authors

#NarrowSpec#DFlash#EAGLE-3#Qwen3-8B#GPT-OSS-20B#SGLang#NVIDIA B200#DGX Spark

Original · 3 min readRead the original
Group chat

No comments yet. Start the conversation.