TorchNeuron

Headlines · 2

1. 10h
   Deepgram ports Nova-3 speech model to AWS Trainium2, reports 11–70% more audio per dollar

   Deepgram worked with AWS's Forward Deployed Engineering team to port its Nova-3 speech-to-text model from standard PyTorch code to AWS Trainium2, tuning it for offline batch workloads such as contact-center analytics and medical transcription. The final setup runs eight model replicas on a single Amazon EC2 trn2.3xlarge instance, using TorchNeuron, prebuilt NKILib kernels, hand-written NKI kernels and a Rust data-parallel runtime. According to the AWS blog, that configuration processed 11–70 percent more audio per dollar than the compute options Deepgram had been running on those offline throughput-oriented workloads, measured with instance pricing at evaluation time; no independent verification is cited.

   AWS Machine Learning Blog · 🔥 8
2. 14h
   Amazon launches TorchNeuron, an open-source PyTorch backend for Trainium

   Amazon announced TorchNeuron on October 19, describing it as an open-source native PyTorch backend that lets existing PyTorch code run on its Trainium AI chips by changing a single device setting instead of rewriting code. The tool is set to enter public beta on October 20 at the PyTorch Conference in San Jose under the Apache 2.0 license, supporting Trainium2 and Trainium3. Amazon says a demo showed the same training script learning at the same rate on a standard chip and on Trainium, but according to The Next Web the first release lacks some PyTorch features such as dynamic input shapes and skips about 22,000 PyTorch tests; early users include Cartesia and Deepgram.

   PyTorch Blog · 🔥 50

Experience and discussion from the community

Share my TorchNeuron experienceAsk about TorchNeuron