AWS adds managed Dynamo and global routing to SageMaker Inference
AWS Machine Learning Blog says Amazon SageMaker Inference added managed NVIDIA Dynamo, expanded HyperPod routing and multi-Region networking, and introduced Inference Gateway and Global Inference Gateway services. The latter provides one endpoint across clusters and Regions with failover in under two minutes, while managed Dynamo supports vLLM, SGLang, and TensorRT-LLM where HyperPod is available. AWS also said inference recommendations are available through SageMaker AI Studio, the SageMaker Python SDK, and coding agents.
The updates give SageMaker users managed model-serving frameworks, GPU-aware and cross-Region routing, and cross-account endpoint routing within the same inference stack.
From first endpoint to global fleet: 14 more launches for managed endpoints and HyperPod Inference
In September, Amazon SageMaker Inference: 2026 year-to-date launches in review covered 13 launches that shortened the path from a model to a production endpoint. Most of them answered one question: how do I get this model serving quickly and reliably? This post covers the 13 launches that came next, bringing the total for 2026 to 26.
Once a model is in production, the questions change. Teams want to know which configuration the model should run on now that traffic is real, how to fit more models onto GPUs they already pay for, what to do when the next model is too large for one machine, why requests wait in one Region while GPUs sit idle in another, and why latency rose when nobody changed anything. This post takes those questions in the order teams usually meet them.
For HyperPod Inference, it listed additional routing strategies, broader support for model serving frameworks, and expanded multi-Region networking. The Inference Gateway and the Global Inference Gateway deliver the routing and multi-Region work, and managed NVIDIA Dynamo is a new serving framework. For managed endpoints, the roadmap pointed to deeper integration with the rest of the AWS AI landscape. Inference recommendations now run from Amazon SageMaker AI Studio, the SageMaker Python SDK, and the coding agents engineers already use, and endpoints can now route requests across Regions and accounts.
#Amazon Web Services#Amazon SageMaker#Amazon SageMaker HyperPod#NVIDIA Dynamo#Amazon SageMaker Inference Gateway#Amazon SageMaker Global Inference Gateway#Amazon SageMaker AI Studio#Amazon SageMaker Python SDK#vLLM#SGLang#TensorRT-LLM