Helion Adds Compile-Time Kernel Configuration Heuristics
Helion now analyzes kernel intermediate representation and dataflow at compile time to select configurations without autotuning. Its heuristics classify kernels as matmul, reduction, or pointwise and choose block sizes, warp counts, and pipeline stages. According to PyTorch Blog, the project reported gains over handwritten implementations in linear-attention and vLLM reduction benchmarks, with the feature enabled by default for H100 and B200 on the Triton backend.
The heuristics provide a faster alternative to searching thousands of configurations when developers cannot amortize autotuning across fixed shapes, although full autotuning remains preferable in some cases.
TL;DR
Helion now uses compile-time heuristics that analyze a kernel’s IR and dataflow to pick fast settings without slow autotuning. It classifies the kernel as matmul, reduction, or pointwise and chooses block sizes, warps, and stages, giving near-autotuned speed for free.
Introduction
Helion, PyTorch’s domain-specific language (DSL) for performance portable machine learning kernels, relies heavily on autotuning for its performance. Autotuning time can be a major bottleneck: for example, a previous team tuning a single paged-attention kernel across 72 shapes spent 10 hours in Helion’s quick mode, and 25 at full effort. To address these issues, we’ve developed a set of heuristics that read a kernel’s IR and dataflow at compile time and emit a config directly, without any autotuning.
We classify kernels into broad families based on the operations they contain – matmul, reduction, or pointwise – and apply a family-specific heuristic to each. These families are intentionally general: the matmul heuristic, for example, looks for any kernel containing a matmul operation such as hl.dot or torch.baddbmm, not just standard m-n-k matrix multiplications. These are designed to be generic and apply broadly to custom kernels that users may write.
#Helion#PyTorch#Triton#NVIDIA H100#NVIDIA B200#Flash Linear Attention#vLLM