torch-spyre

Headlines · 2

1. 4d
   torch-spyre moves its compiler onto PyTorch Inductor

   A PyTorch blog post explains that torch-spyre, the PyTorch integration for IBM's Spyre accelerator, has moved its torch.compile backend from sendnn — a compilation stack built entirely from scratch — to backend=inductor, built by extending PyTorch's own open-source Inductor. Most of Inductor carried over untouched, with the work concentrated in three hardware differences: Spyre moves and computes on 128-byte sticks rather than individual elements; each of its 32 cores has only a 2MB local scratchpad with no hardware-managed caching, so every data movement must be an explicit compiler-emitted instruction; and the cores run as a static dataflow machine, so work division must be decided at compile time. The authors say the resulting compiler is roughly 7x smaller than what it replaced, and report that changing input and output tensor tiling cut batched matmul kernel time by 29–69% across batch sizes and shapes when tensors were already resident in device memory. The Spyre passes currently hook Inductor by monkey-patching GraphLowering.codegen(), and the team is working with upstream PyTorch to formalize those extension points.

   PyTorch Blog · 🔥 0
2. Oct 8
   torch-spyre brings IBM Spyre in as a native PyTorch device

   IBM's torch-spyre team published a PyTorch blog post describing how it made Spyre, IBM's dataflow AI accelerator for inference, a native PyTorch device. The integration uses PrivateUse1 registration to give Spyre a device identity named "spyre", then maps PyTorch's allocator, stream, event and Inductor compiler abstractions onto Spyre runtime constraints such as ordered queues of typed operations, separate compute and data-movement pipelines, and ahead-of-time compiled programs with fixed layouts. IBM says this gives one path for eager and compiled execution, keeps tensors resident on device="spyre", and cuts launch overhead; no performance or benchmark figures are given, and the source text is truncated near the end.

   PyTorch Blog · 🔥 0

Experience and discussion from the community

Share my torch-spyre experienceAsk about torch-spyre