vLLM集成FlyDSL MonoKernels以降低AMD解码延迟
vLLM博客介绍将FlyDSL MonoKernels集成到vLLM,用于AMD Instinct GPU上的低并发交互式大模型服务。该方案在单次内核启动中使用持久化GPU调度,将多个相互依赖的算子阶段结合起来,以减少重复的内核启动、排空与调度开销。博客称,同节点测试中,并发为1时的交互性能较原生vLLM提升1.20倍至2.26倍,同时指出结果会因模型、工作负载和系统配置而异。
编程智能体必须先生成当前动作,才能调用下一项工具,因此解码延迟位于任务关键路径上。报告指出,低并发服务侧重单用户响应速度,因此降低解码延迟与交互式AI应用直接相关。
Introduction
Interactive LLM applications depend on low decode latency. A coding agent must finish generating its current action before it can call the next tool, so decode sits on the task’s critical path. Serving these workloads at low concurrency prioritizes per-user responsiveness over aggregate throughput.
A decode step executes a chain of GPU kernels for operations such as projections, attention, and expert computation. At small batch sizes, many of these kernels do only a little work, yet each still incurs GPU-side dispatch, startup, and completion costs. A dependent kernel cannot proceed until its inputs are ready, so whole-kernel boundaries also limit overlap. Repeated across a long chain, these costs can account for a substantial part of decode latency, while the small amount of parallel work leaves GPU capacity unused. HIP graphs reduce CPU launch overhead, but do not eliminate these GPU-side costs. Figure 1 illustrates the repeated stages; widths and gaps are schematic, not measured durations or an overhead breakdown.
#vLLM#FlyDSL#AMD Instinct#DeepSeek-V4.1-Flash#MiniMax-M3#Muse Spark#AgentX