AWS更新SageMaker推理与全局路由
AWS Machine Learning Blog称,Amazon SageMaker Inference新增托管NVIDIA Dynamo、扩展HyperPod路由和多区域网络,并推出Inference Gateway与Global Inference Gateway。后者可跨集群和区域提供单一端点,并实现两分钟内故障转移;托管Dynamo则在HyperPod可用区域支持vLLM、SGLang和TensorRT-LLM。AWS还表示,推理建议现可通过SageMaker AI Studio、SageMaker Python SDK及编程智能体使用。
这些更新让SageMaker用户在统一推理栈内使用托管模型服务框架、GPU感知及跨区域路由,并支持跨账户端点路由。
From first endpoint to global fleet: 14 more launches for managed endpoints and HyperPod Inference
In September, Amazon SageMaker Inference: 2026 year-to-date launches in review covered 13 launches that shortened the path from a model to a production endpoint. Most of them answered one question: how do I get this model serving quickly and reliably? This post covers the 13 launches that came next, bringing the total for 2026 to 26.
Once a model is in production, the questions change. Teams want to know which configuration the model should run on now that traffic is real, how to fit more models onto GPUs they already pay for, what to do when the next model is too large for one machine, why requests wait in one Region while GPUs sit idle in another, and why latency rose when nobody changed anything. This post takes those questions in the order teams usually meet them.
For HyperPod Inference, it listed additional routing strategies, broader support for model serving frameworks, and expanded multi-Region networking. The Inference Gateway and the Global Inference Gateway deliver the routing and multi-Region work, and managed NVIDIA Dynamo is a new serving framework. For managed endpoints, the roadmap pointed to deeper integration with the rest of the AWS AI landscape. Inference recommendations now run from Amazon SageMaker AI Studio, the SageMaker Python SDK, and the coding agents engineers already use, and endpoints can now route requests across Regions and accounts.
#Amazon Web Services#Amazon SageMaker#Amazon SageMaker HyperPod#NVIDIA Dynamo#Amazon SageMaker Inference Gateway#Amazon SageMaker Global Inference Gateway#Amazon SageMaker AI Studio#Amazon SageMaker Python SDK#vLLM#SGLang#TensorRT-LLM