创建

登录 ReadmeX

或

了解我们如何处理你的信息: 隐私政策

ReadmeX

陪你读懂 AI 动态

今天,想先了解什么?

选一份动态总结,或直接聊聊你的问题。

社区和关注动态总结只属于你自己。

资讯

Qwen推出SpecOPD推测解码框架

AI 总结

Qwen研究人员介绍了SpecOPD,这是一种分支并行的在线策略蒸馏框架,可跨独立的提议树分支并行执行草稿采样和目标监督。该方法采用直接针对接受长度的E2E-TV代理目标,并在8B、27B和125B目标模型、三种草稿架构及四类任务上进行了评估。Qwen报告称,其平均接受长度最高提升41.45%,在原生配置下监督词元吞吐量较顺序滚动OPD达到16.09倍,生成速度较自回归解码最高提升3.56倍。

深度见解

SpecOPD通过并行化训练监督并直接优化接受长度,针对推测解码的关键瓶颈,Qwen报告其在多种模型和任务配置下取得了显著的推理效率提升。

Speculative decoding (SD) accelerates large language model inference by reducing the sequential cost of token-by-token generation. Advanced SD methods reduce proposal cost with parallel drafters that predict entire blocks in one forward pass, but overall speedup still depends on acceptance length. On-policy distillation (OPD) can push well-trained drafters beyond acceptance-length plateaus, yet conventional pipelines collect supervision through sequential draft-and-verify rollouts. Our key insight is that onpolicy supervision need not inherit these dependencies: a draft block can be supervised using only its reference prefix and draft-sampled history. We propose SpecOPD, a branch-parallel OPD framework that parallelizes draft sampling and target supervision across independent branches drawn from the evolving drafter policy at reference anchors. We pack these branches into proposal trees to share target-prefix computation, using tree masks for full attention, branch-consistent indexer groups for sparse attention, and indexed state replay for linear attention to preserve each branch’s history. We train the drafter with an E2E-TV surrogate that directly targets acceptance length. Our study spans 8B full-attention (GQA), 27B full–linear (GQA–GDN), and 125B sparse–linear (QSA–GDN) targets, three parallel draft architectures (DFlash, DSpark, and DFlash V2), and four task domains (dialogue, coding, mathematics, and agentic). SpecOPD achieves up to 41.45% higher mean acceptance length, 16.09× supervised-token throughput over sequential-rollout OPD under native configurations, and up to 3.56× generation speedup over autoregressive decoding. submitted by /u/pmttyji [link] [comments]

#Qwen#SpecOPD#DFlash#DSpark#DFlash V2

原文约 1 分钟读完阅读原文
群聊

还没有评论,来说说你的看法。