Google adds cycle-level Pallas kernel profiling to XProf on TPU v7
Google has added a kernel performance profiling suite to XProf, its open-source profiler for TPU workloads, giving developers cycle-level detail on custom Pallas kernels. Such kernels — built with Pallas, Mosaic or Triton — skip standard XLA passes and previously appeared as a single opaque block in traces, skewing static cost-model metrics such as "optimal FLOPs" and effective throughput. The suite exposes lowered MLIR and per-cycle low-level operation (LLO) data, and samples hardware counters at a 1µs floor (sub-microsecond with external triggers). In a Google blocked matrix-multiplication example, sync_wait counters revealed memory stalls; adding triple buffering cut kernel execution from 125.5µs to 88µs, about 30%, though the article stresses this is one demo kernel, not a benchmark.
For engineers tuning TPU kernels, register-level hardware counters are more trustworthy than compiler static estimates, shifting the basis for optimization from what the model predicts to what the silicon shows.
Google 为 XProf" 添加了"一个内核性能分析套件。XProf 是其面向 TPU" 工作负载的开源性能分析器。现在，开发者可以查看自定义 Pallas 内核"中的周期级细节。此前，这些内核在跟踪记录中显示为单个不透明块。在 TPU v7（Ironwood）"上，XProf 会在运行时对硬件性能计数器进行采样。在一个分块矩阵乘法示例中，Google 发现了内存停顿问题。通过添加三重缓冲，他们将内核执行时间从 125.5µs 缩短至 88µs，降幅约为 30%。
据 Google AI Infra 团队的 Yogesh SY 介绍，这一差距源于性能分析器处理自定义编译路径的方式。使用 Pallas、Mosaic 或 Triton 创建的内核会跳过标准 XLA Pass。这可能使编译期静态成本模型产生偏差。因此，“optimal FLOPs”和有效吞吐效率等指标可能不准确，甚至完全无法工作。静态工具可能会将一个 MXU 指令块标记为已充分利用，但该单元实际上正在空闲等待 HBM，因为静态分析忽略了时间因素。