cuDLA Python bindings bring native DLA inference to Jetson Orin
NVIDIA says cuda-bindings 13.3.0 introduces cuda.bindings.cudla, a Python layer over the cuDLA C++ API for managing and orchestrating inference on NVIDIA Deep Learning Accelerator hardware. On Jetson AGX Orin, the bindings support hybrid workflows that coordinate GPU preprocessing and postprocessing with DLA inference through CUDA streams. The documented image-classification example covers loading a compiled network, allocating buffers, submitting work, synchronization, and retrieving output, while standalone mode remains unsupported in this release.
The release lets Python developers manage DLA inference directly on Jetson AGX Orin and RTX Spark without writing a separate C++ runtime-management layer.
AI applications at the edge often need to run multiple neural networks within limited compute and power budgets. NVIDIA Deep Learning Accelerator (DLA) platforms provide dedicated hardware for deep learning inference. On NVIDIA Jetson AGX Orin, DLA executes supported neural network operations, allowing applications to offload inference work from the GPU. This can improve power efficiency and leave GPU capacity available for other application tasks.
NVIDIA cuDLA is the runtime API for managing execution on this accelerator. Applications use it to create DLA devices, load compiled networks, register memory buffers, and submit inference tasks. The cuda.bindings.cudla module exposes these APIs directly to Python, so developers can integrate DLA execution into their applications without writing C++ code to manage the runtime.
This post walks through the inference lifecycle on Jetson AGX Orin using an image classification example. It briefly covers installation and compiling a model into a DLA loadable, then follows execution from creating a device and loading the network to managing buffers, synchronizing execution, and reading the output. The example also shows how GPU and DLA stages work together in a single Python application.
#NVIDIA#cuDLA#cuda-bindings 13.3.0#cuda.bindings.cudla#Jetson AGX Orin#Deep Learning Accelerator#RTX Spark#JetPack