NVIDIA CMX
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- NVIDIA Links Full-Stack Codesign to Higher AI Memory Efficiency
NVIDIA published two technical posts on November 5 describing a “full-stack codesign” approach that designs GPUs, CPUs, networking, storage and software together to increase useful output per byte of memory during AI inference. The company says HBM is the fastest, most expensive and most contested tier, where model weights and KV cache compete, while CPU memory, local SSDs and shared storage trade speed for capacity and lower cost per bit. The approach includes NVFP4 for weights and KV cache, weight distribution across up to 72 GPUs in NVL72 systems, large-capacity LPDDR in the Vera CPU, Dynamo KV-cache offloading and reuse, CMX context memory storage, and keeping RAG indexes outside HBM. Citing its own SemiAnalysis AgentX results, NVIDIA says Vera Rubin NVL72 delivers up to 30 times the throughput per megawatt of GB300 NVL72 at 160 tokens per second per user. A separate comparison says Vera Rubin delivers up to 30 times the throughput per dollar at 125 tokens per second per user, while HBM capacity per GPU increased about 3% from Blackwell Ultra. These figures have not been independently verified in the supplied reports.
NVIDIA Blog · 🔥 0
Experience and discussion from the community
Share my NVIDIA CMX experienceAsk about NVIDIA CMX
Nobody has shared their experience with NVIDIA CMX yet.