创建

登录 ReadmeX

或

了解我们如何处理你的信息: 隐私政策

ReadmeX

陪你读懂 AI 动态

今天,想先了解什么?

选一份动态总结,或直接聊聊你的问题。

社区和关注动态总结只属于你自己。

资讯

MMFlash加速视觉语言模型推测解码

AI 总结

MMFlash 作者发布了代码和草稿模型,面向 Qwen3.5-4B、Qwen3.5-9B、Qwen3.8-27B 与 Gemma 4 E4B-it 等视觉语言模型加速推测解码。作者称,该方法会自适应加权视觉依赖较强的词元,并以稀疏注意力处理长视频输入,在多模态基准上实现最高 3.58 倍无损解码加速;这些结果来自作者,尚无独立验证。

深度见解

MMFlash 针对多模态推理中的具体问题:草稿模型对依赖图像的词元接受率较低,而其公开代码和模型旨在改善这些预测,同时保持目标模型输出不变。

Speculative decoding accelerates generation by letting a lightweight drafter propose tokens for parallel verification by a target model. Diffusion drafters such as DFlash propose an entire token block in a single forward pass, but mainly focus on text-only tasks, leaving multimodal drafting underexplored. To study this setting, we first introduce a metric of token-level visual dependence based on the KL divergence between next-token distributions with and without visual input. We find that acceptance rates decrease from 46% in the lowest visual-dependence quartile to 22% in the highest, with this disparity persisting after multimodal training. Motivated by this observation, we propose MMFlash, a simple and effective framework that adaptively upweights visually dependent tokens that the drafter predicts poorly. For long video inputs, MMFlash further employs sparse attention to reduce redundant attention computation and improve drafting quality. Across various multimodal benchmarks, MMFlash achieves lossless decoding speedups of up to 3.58×, outperforming EAGLE-3 and DFlash by up to 195.4% and 17.1%, respectively. ⭐ Updates 💪 Highlights 💻 Models The drafts are not standalone language models: each is paired with its target in a speculative-decoding server. All of them are in the MMFlash collection; every model card has the serving command and per-benchmark results. submitted by /u/pmttyji [link] [comments]

#MMFlash#Qwen3.5-4B#Qwen3.5-9B#Qwen3.8-27B#Gemma 4 E4B-it#DFlash#EAGLE-3#SGLang#vLLM

原文约 1 分钟读完阅读原文
群聊

还没有评论,来说说你的看法。