cuDLA Python绑定支持Jetson原生DLA推理
NVIDIA称,cuda-bindings 13.3.0新增了cuda.bindings.cudla,通过Python封装cuDLA C++ API,用于管理和编排NVIDIA深度学习加速器上的推理。在Jetson AGX Orin上,它支持由CUDA流协调GPU预处理、DLA推理和GPU后处理的混合工作流。官方图像分类示例涵盖加载编译网络、分配缓冲区、提交任务、同步和读取输出,但此版本尚不支持独立模式。
这一版本让Python开发者无需另写C++运行管理层,即可直接在Jetson AGX Orin和RTX Spark上管理DLA推理。
AI applications at the edge often need to run multiple neural networks within limited compute and power budgets. NVIDIA Deep Learning Accelerator (DLA) platforms provide dedicated hardware for deep learning inference. On NVIDIA Jetson AGX Orin, DLA executes supported neural network operations, allowing applications to offload inference work from the GPU. This can improve power efficiency and leave GPU capacity available for other application tasks.
NVIDIA cuDLA is the runtime API for managing execution on this accelerator. Applications use it to create DLA devices, load compiled networks, register memory buffers, and submit inference tasks. The cuda.bindings.cudla module exposes these APIs directly to Python, so developers can integrate DLA execution into their applications without writing C++ code to manage the runtime.
This post walks through the inference lifecycle on Jetson AGX Orin using an image classification example. It briefly covers installation and compiling a model into a DLA loadable, then follows execution from creating a device and loading the network to managing buffers, synchronizing execution, and reading the output. The example also shows how GPU and DLA stages work together in a single Python application.
#NVIDIA#cuDLA#cuda-bindings 13.3.0#cuda.bindings.cudla#Jetson AGX Orin#Deep Learning Accelerator#RTX Spark#JetPack