创建

登录 ReadmeX

或

了解我们如何处理你的信息: 隐私政策

ReadmeX

陪你读懂 AI 动态

今天,想先了解什么?

选一份动态总结,或直接聊聊你的问题。

社区和关注动态总结只属于你自己。

资讯

vLLM集成FlyDSL MonoKernels以降低AMD解码延迟

AI 总结

vLLM博客介绍将FlyDSL MonoKernels集成到vLLM,用于AMD Instinct GPU上的低并发交互式大模型服务。该方案在单次内核启动中使用持久化GPU调度,将多个相互依赖的算子阶段结合起来,以减少重复的内核启动、排空与调度开销。博客称,同节点测试中,并发为1时的交互性能较原生vLLM提升1.20倍至2.26倍,同时指出结果会因模型、工作负载和系统配置而异。

深度见解

编程智能体必须先生成当前动作,才能调用下一项工具,因此解码延迟位于任务关键路径上。报告指出,低并发服务侧重单用户响应速度,因此降低解码延迟与交互式AI应用直接相关。

Introduction

Interactive LLM applications depend on low decode latency. A coding agent must finish generating its current action before it can call the next tool, so decode sits on the task’s critical path. Serving these workloads at low concurrency prioritizes per-user responsiveness over aggregate throughput.

A decode step executes a chain of GPU kernels for operations such as projections, attention, and expert computation. At small batch sizes, many of these kernels do only a little work, yet each still incurs GPU-side dispatch, startup, and completion costs. A dependent kernel cannot proceed until its inputs are ready, so whole-kernel boundaries also limit overlap. Repeated across a long chain, these costs can account for a substantial part of decode latency, while the small amount of parallel work leaves GPU capacity unused. HIP graphs reduce CPU launch overhead, but do not eliminate these GPU-side costs. Figure 1 illustrates the repeated stages; widths and gaps are schematic, not measured durations or an overhead breakdown.

A GPU-side chain of Kernel A, B and C, each with Start, Work and Drain stages.
Figure 1. Repeated startup and completion around short kernels.

#vLLM#FlyDSL#AMD Instinct#DeepSeek-V4.1-Flash#MiniMax-M3#Muse Spark#AgentX

原文约 17 分钟读完阅读原文
群聊

还没有评论,来说说你的看法。