CUDA 13.4为Kubernetes引入显存强制共享
NVIDIA Technical Blog称，CUDA 13.4为GPU时分共享加入对应用透明的显存强制机制，并保留Multi-Process Service已有的SM共享计算模式。工作负载可申请固定显存或保底显存与可选的弹性上限，新开源GPU Fractioning Operator则通过Pod注解在Kubernetes中实施这些限制和计算模式。NVIDIA还表示KAI Scheduler v0.18.0可选择GPU份额并写入所需注解，但目前没有独立测试结果佐证。
该版本提供了厂商声明的显存与计算限额机制，让多个AI工作负载按明确配额共享GPU，直接涉及GPU利用率和集群编排。
Many AI workloads need only a fraction of a GPU. Giving each one an entire device can leave capacity idle; sharing GPUs lets platform teams serve more workloads with the infrastructure they already have.
NVIDIA GPUs already support several approaches to sharing. Multi-instance GPU (MIG) splits a GPU into hardware-isolated instances that are transparent to the application, time-slicing lets several processes take turns without memory enforcement, and Multi-Process Service (MPS) lets them run concurrently in separate address spaces (sm-sharing).
CUDA 13.4 now enforces a memory boundary that is transparent to the application with a new time-sharing mode, alongside the compute sharing mode (sm-sharing) that already existed in MPS.
Here is what the CUDA release brings:
- An exact memory amount: A workload asks for a memory size and CUDA holds it there. An allocation above that size fails, just as it would on a smaller GPU.
- A range: A workload gets a guaranteed minimum plus a maximum it can grow into when memory is free.
- A compute mode: Each workload can choose to use its compute in an sm-sharing mode rather than using the default time-sharing mode.
- A transparent memory limit: Each workload sees only a fraction of the GPU memory.
#CUDA 13.4#NVIDIA#GPU Fractioning Operator#KAI Scheduler#NVIDIA GPU Operator#Multi-Process Service#Kubernetes