创建

登录 ReadmeX

或

了解我们如何处理你的信息: 隐私政策

ReadmeX

陪你读懂 AI 动态

今天,想先了解什么?

选一份动态总结,或直接聊聊你的问题。

社区和关注动态总结只属于你自己。

资讯

Helion加入编译时内核配置启发式

AI 总结

Helion现在可在编译时分析内核中间表示和数据流,直接选择配置而无需自动调优。其启发式将内核分为矩阵乘、归约或逐元素类型,并确定块大小、并行线程数和流水阶段。PyTorch Blog称,项目方在线性注意力和vLLM归约内核测试中报告了性能提升,该功能已在Triton后端的H100和B200上默认启用。

深度见解

当开发者无法针对固定形状摊销自动调优成本时,这组启发式提供了比搜索数千种配置更快的替代方案,但部分场景仍更适合完整自动调优。

TL;DR

Helion now uses compile-time heuristics that analyze a kernel’s IR and dataflow to pick fast settings without slow autotuning. It classifies the kernel as matmul, reduction, or pointwise and chooses block sizes, warps, and stages, giving near-autotuned speed for free.

Introduction

Helion, PyTorch’s domain-specific language (DSL) for performance portable machine learning kernels, relies heavily on autotuning for its performance. Autotuning time can be a major bottleneck: for example, a previous team tuning a single paged-attention kernel across 72 shapes spent 10 hours in Helion’s quick mode, and 25 at full effort. To address these issues, we’ve developed a set of heuristics that read a kernel’s IR and dataflow at compile time and emit a config directly, without any autotuning.

We classify kernels into broad families based on the operations they contain – matmul, reduction, or pointwise – and apply a family-specific heuristic to each. These families are intentionally general: the matmul heuristic, for example, looks for any kernel containing a matmul operation such as hl.dot or torch.baddbmm, not just standard m-n-k matrix multiplications. These are designed to be generic and apply broadly to custom kernels that users may write.

#Helion#PyTorch#Triton#NVIDIA H100#NVIDIA B200#Flash Linear Attention#vLLM

原文约 13 分钟读完阅读原文
群聊

还没有评论,来说说你的看法。