Kestrel_VLLM Cuts GDN Kernel Launches by Moving the Batch M Loop Inside
Kestrel_VLLM developer measurements show that moving the batch M loop into CUDA kernels reduced per-layer launches from 27 to 9 at M=4 and from 51 to 9 at M=8. In three interleaved M=4 rounds, the merged path averaged 24.063 ms versus 28.182 ms for the per-M path, a reported 14.6% reduction, while memcmp tests found bitwise-identical outputs across real mixed batches from M=2 through M=8. The merged route became the default with an environment-variable fallback, but the evidence comes from one self-published account and some performance figures were not reproduced in the final checklist.
The report describes a substantial AI-inference optimization that decouples GDN kernel-launch count from batch size, with the developer reporting lower latency without changing numerical outputs.
35 · 把 m 循环搬进内核:一次「先量发射、再合并发射」的完整回合
一句话:第 34 篇把批量 M 行建出来了,但只证明算术是对的。第一次真正量它的性能时,实测比模型预测高约 35%——本篇把这笔账从对不上走到结清:逐相分解 → 定位到 GDN 子层 → 造两个诊断档把「发射」与「工作」分开 → 坐实「每行成本 = 发射主导」→ 把 6 次/行的发射合并成 1 次 → 默认化。拿到了阶段五单笔最大的一刀(M=4 的 verify 段 2.130× → 2.495×,默认化后整步 2.521×)。 前置:先读 第 34 篇(批量 M 行的五个块);对照 第 29 篇(「发射次数多 ≠ 发射开销大」)。 环境:x86-64 · NVIDIA RTX 5090 · Qwen3.5-27B(q4 打包)· CUDA 后端;同会话交错 3 轮。
一、问题与结论
| 指标 | 数值 / 结果 | 判定 |
|---|---|---|
| 「高了 35%」先变成分相账 | 缺口落在 ② 子层,病态的是 GDN 子层(attn-64 0.930 / gdn-64 3.260 / mixed 3.501 ms/行) | 实测 |
| 门禁:LF2 每行增量 | 0.38–0.54 ms/行,低于 attn 档的 0.930,远低于 real 的 3.26–3.50 | 实测 |
| 落地:每层发射数 | M=4:27 → 9;M=8:51 → 9(与 M 无关) | 实测 |
| M=4 verify 段 | 28.182 → 24.063 ms,省 4.119 ms(−14.6%);提速 2.130× → 2.495× | 实测 |
| M=8 | 省 9.34 ms(2.460× → 3.207×) | 实测 |
| 默认化后整步 | M=4 2.521× / M=8 3.174× | 实测 |