NVIDIA Dynamo

Headlines · 5

1. 2d
   NVIDIA releases Nemotron 3.5 Super VL and two Bolt edge models

   NVIDIA released Nemotron 3.5 Super VL on Oct. 15, an open multimodal model with 120B total and 12B active parameters in a mixture-of-experts design that combines reasoning, instruction following and tool use with text, image, document and video understanding for long-running agents. NVIDIA claims it reaches up to 5x the agentic throughput of other open models in its size class on a single GPU — over 600 output tokens per second at 100,000 input tokens — scores 80.8% on the CharXiv chart-reasoning benchmark (84% with Python code interpretation), and supports NVFP4 quantization and multi-token prediction; BF16 and NVFP4 weights are on Hugging Face with same-day vLLM and Baseten support. Alongside it, NVIDIA launched two Nemotron 3.5 Bolt edge models: Bolt 74B-A9B 24GB, which uses native 2-bit quantization to fit in 24GB of memory and is available now, and Bolt E4B 4GB, due Oct. 23.

   vLLM Blog · 🔥 64
2. 6d
   SGLang trends on GitHub's Python list

   SGLang is an open-source inference framework hosted by LMSYS for large language, vision-language and diffusion models, optimized for agentic workloads, large-scale serving and RL rollouts; the repository also includes SGLang Diffusion, its built-in image and video generation engine. It supports a wide range of GPUs, TPUs, NPUs, CPUs and Apple Silicon, including NVIDIA B200/B300 and H100, AMD Instinct, Google TPU, Huawei Ascend, and ships via Docker images or a uv install. The project offers a Cookbook, docs and learning resources such as Mini-SGLang, and long-term contributors can apply for coding-agent sponsorship covering Cursor, Claude Code or OpenAI Codex.

   GitHub Trending · Python · 🔥 0
3. Oct 8
   NVIDIA Dynamo adds session-aware routing and KV cache for agentic inference

   A PyTorch blog post details how NVIDIA Dynamo turns request-level serving into program-aware serving using one session-level identifier, with child sessions carrying a parent_session_id. Dynamo recognizes the session headers that Claude Code, Codex and OpenCode already emit, while other harnesses opt in via X-Dynamo-Session-ID, and plugins exist for Hermes and OpenClaw. The post also covers session-aware routing and admission control (porting a scheduler from the ThunderAgent paper), a shared KV cache indexer, the Session-Prefix Indexer and proposed KvHint interface, plus DYN_REQUEST_TRACE=1 trace capture with AISimulate offline replay and AIPerf live benchmarking.

   PyTorch Blog · 🔥 0
4. Oct 7
   Nebius, NVIDIA and Uber on squeezing more value from every unit of compute

   At a panel hosted by The Information's AI Agenda Live, executives from Nebius, NVIDIA and Uber discussed how they are optimizing compute as agentic workflows threaten to multiply token consumption. Nebius CRO Marc Boroditsky said the company tailors optimizations in its Token Factory inference platform to each customer's goals—speed, reliability, quality or price—using techniques such as speculative decoding, where smaller models predict likely next tokens that a larger model verifies in bulk, tuned with the customer's own traffic. NVIDIA's Dion Harris pointed to a Dynamo feature that locates already-computed data in a cluster's KV cache so teams avoid recalculating it, while Uber infrastructure VP Mattie Toia said its token costs have stabilized despite growing agentic use, helped by caching pushed down to individual sub-agents and by giving engineers visibility into their own usage; Uber also built an internal "context graph" of roughly 24 million nodes that cut query times from 20 minutes to 30 seconds.

   The Information · 🔥 0
5. Oct 6
   Vast pitches tiered storage to ease AI agent memory pressure

   In an interview with theCUBE at CoreWeave's Fully Connected 2026 event, Vast Data co-founder and CTO Alon Horev said agent memory differs from ordinary inference: it includes both in-session context and long-term memory that lets an agent review past interactions. Vast's approach tiers storage — GPU memory first, then CPU memory on the same machine, then persistent media holding petabytes of KV cache — with Nvidia's Dynamo software orchestrating the process. Horev said a 500,000-token session can occupy one-tenth to one-twentieth of a GPU's memory, and offloading such sessions to storage avoids repeat recalculation; enterprises also need to record and retain everything their agents do, data that can feed fine-tuning or purpose-built models.

   SiliconANGLE AI · 🔥 0

Experience and discussion from the community

Share my NVIDIA Dynamo experienceAsk about NVIDIA Dynamo