ApexW

Headlines · 1

1. Oct 27
   ApexW PyTorch optimizer targets large GPU workloads

   A developer shared ApexW, a PyTorch optimizer that uses compact optimizer states and a Triton-based execution path for large GPU workloads. In the developer's optimizer-only tests, the Triton compact-state path was about 1.5 times faster than PyTorch's fused AdamW on workloads with a few large tensors while using roughly half the optimizer-state memory. The developer said these results did not represent full model training, that automatic mode did not consistently achieve the same speedup, and that workloads with many small tensors were much slower than PyTorch's fused optimizer. The developer also reported that PyTorch was slightly faster overall in an end-to-end MLP test.

   r/MachineLearning · 🔥 0

Experience and discussion from the community

Share my ApexW experienceAsk about ApexW