NarrowSpec
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- NarrowSpec enables narrower speculative-decoding drafts
A paper submitted to arXiv on November 18, 2026 proposes that speculative-decoding draft models need not have the same hidden width as their target models. The authors report that DFlash and EAGLE-3 drafts using half or one-quarter of the target width largely preserve token acceptance. In tests with Qwen3-8B (NVFP4) and GPT-OSS-20B (MXFP4) running in SGLang, the authors report end-to-end decoding speedups of 1.06–1.12x on an NVIDIA B200 and 1.13–1.44x on a DGX Spark (GB10) at batch size 1 and temperature 0.
Hugging Face · Papers · 🔥 0
Experience and discussion from the community
Share my NarrowSpec experienceAsk about NarrowSpec
Nobody has shared their experience with NarrowSpec yet.