Helion加入编译时内核配置启发式
Helion现在可在编译时分析内核中间表示和数据流,直接选择配置而无需自动调优。其启发式将内核分为矩阵乘、归约或逐元素类型,并确定块大小、并行线程数和流水阶段。PyTorch Blog称,项目方在线性注意力和vLLM归约内核测试中报告了性能提升,该功能已在Triton后端的H100和B200上默认启用。
当开发者无法针对固定形状摊销自动调优成本时,这组启发式提供了比搜索数千种配置更快的替代方案,但部分场景仍更适合完整自动调优。
TL;DR
Helion now uses compile-time heuristics that analyze a kernel’s IR and dataflow to pick fast settings without slow autotuning. It classifies the kernel as matmul, reduction, or pointwise and chooses block sizes, warps, and stages, giving near-autotuned speed for free.
Introduction
Helion, PyTorch’s domain-specific language (DSL) for performance portable machine learning kernels, relies heavily on autotuning for its performance. Autotuning time can be a major bottleneck: for example, a previous team tuning a single paged-attention kernel across 72 shapes spent 10 hours in Helion’s quick mode, and 25 at full effort. To address these issues, we’ve developed a set of heuristics that read a kernel’s IR and dataflow at compile time and emit a config directly, without any autotuning.
We classify kernels into broad families based on the operations they contain – matmul, reduction, or pointwise – and apply a family-specific heuristic to each. These families are intentionally general: the matmul heuristic, for example, looks for any kernel containing a matmul operation such as hl.dot or torch.baddbmm, not just standard m-n-k matrix multiplications. These are designed to be generic and apply broadly to custom kernels that users may write.
#Helion#PyTorch#Triton#NVIDIA H100#NVIDIA B200#Flash Linear Attention#vLLM