创建

登录 ReadmeX

或

了解我们如何处理你的信息: 隐私政策

ReadmeX

陪你读懂 AI 动态

今天,想先了解什么?

选一份动态总结,或直接聊聊你的问题。

社区和关注动态总结只属于你自己。

资讯

vLLM扩展多模态编码器分离能力

AI 总结

vLLM现已支持编码器、预填充与解码分离部署,既可将三个阶段独立部署,也可合并预填充与解码。近期工作新增了面向DeepSeek V4.1 Flash的仅元数据请求、NIXL与Mooncake远程嵌入传输、共享缓存、独立编码器执行及编排集成,核心能力已包含在vLLM v0.32.0中。vLLM Blog还报告称,在使用Qwen3.5-35B-A3B和独立四卡GB200节点的30帧MMVU评测中,端到端吞吐量提升约94%至115%,但该结果来自项目自身测试。

深度见解

将多模态编码与预填充、解耦部署后,运维人员可以独立扩展各阶段,并通过传输嵌入避免重复媒体预处理。

Multimodal inference adds an encoding stage before language-model execution. Images, video frames, and audio must be processed into embeddings before the language model can use them. The demand on this stage varies with media size, modality, and request mix, and can change independently of prefill and decode demand.

Encoder / Prefill / Decode (EPD) disaggregation separates these stages into independently deployed vLLM instances. Recent work in the vLLM community has expanded EPD across model execution, remote embedding transfer, shared caching, multimodal input handling, and request routing.

DeepSeek's inference deployment Q&A describes an E/P/D architecture with a separately deployed vision encoder. vLLM has supported encoder disaggregation since our December 2025 architecture post, and continues to improve that support. vLLM main now also supports metadata-only EPD for DeepSeek V4.1 Flash.

This post explains the engineering behind that progress, with a focus on five changes:

  • Metadata-only requests that let downstream instances consume remote embeddings without repeating media preprocessing.
  • Dedicated encoder execution with a separate Model Runner V2 path, encoder-only CUDA Graph support, and reduced checkpoint-loading I/O.
  • Remote embedding transfer through NIXL and Mooncake integrations.
  • Cross-encoder output reuse through Mooncake Store.
  • Lower orchestration overhead in the Python EPD proxy and EPD orchestration with vLLM Router, SMG, and llm-d.

#vLLM#vLLM v0.32.0#DeepSeek V4.1 Flash#Qwen3.5-35B-A3B#NIXL#Mooncake#Mooncake Store#vLLM Router#SMG#llm-d#MiMo-V2-Omni

原文约 12 分钟读完阅读原文
群聊

还没有评论,来说说你的看法。