NVIDIA Dynamo adds session-aware routing and KV cache for agentic inference
A PyTorch blog post details how NVIDIA Dynamo turns request-level serving into program-aware serving using one session-level identifier, with child sessions carrying a parent_session_id. Dynamo recognizes the session headers that Claude Code, Codex and OpenCode already emit, while other harnesses opt in via X-Dynamo-Session-ID, and plugins exist for Hermes and OpenClaw. The post also covers session-aware routing and admission control (porting a scheduler from the ThunderAgent paper), a shared KV cache indexer, the Session-Prefix Indexer and proposed KvHint interface, plus DYN_REQUEST_TRACE=1 trace capture with AISimulate offline replay and AIPerf live benchmarking.
Agentic serving economics hinge on KV cache reuse and scheduling rather than raw compute, so session-level primitives could become standard in inference stacks.