创建

登录 ReadmeX

或

了解我们如何处理你的信息: 隐私政策

ReadmeX

陪你读懂 AI 动态

今天,想先了解什么?

选一份动态总结,或直接聊聊你的问题。

社区和关注动态总结只属于你自己。

资讯

AWS更新SageMaker推理与全局路由

AI 总结

AWS Machine Learning Blog称,Amazon SageMaker Inference新增托管NVIDIA Dynamo、扩展HyperPod路由和多区域网络,并推出Inference Gateway与Global Inference Gateway。后者可跨集群和区域提供单一端点,并实现两分钟内故障转移;托管Dynamo则在HyperPod可用区域支持vLLM、SGLang和TensorRT-LLM。AWS还表示,推理建议现可通过SageMaker AI Studio、SageMaker Python SDK及编程智能体使用。

深度见解

这些更新让SageMaker用户在统一推理栈内使用托管模型服务框架、GPU感知及跨区域路由,并支持跨账户端点路由。

From first endpoint to global fleet: 14 more launches for managed endpoints and HyperPod Inference

In September, Amazon SageMaker Inference: 2026 year-to-date launches in review covered 13 launches that shortened the path from a model to a production endpoint. Most of them answered one question: how do I get this model serving quickly and reliably? This post covers the 13 launches that came next, bringing the total for 2026 to 26.

Once a model is in production, the questions change. Teams want to know which configuration the model should run on now that traffic is real, how to fit more models onto GPUs they already pay for, what to do when the next model is too large for one machine, why requests wait in one Region while GPUs sit idle in another, and why latency rose when nobody changed anything. This post takes those questions in the order teams usually meet them.

For HyperPod Inference, it listed additional routing strategies, broader support for model serving frameworks, and expanded multi-Region networking. The Inference Gateway and the Global Inference Gateway deliver the routing and multi-Region work, and managed NVIDIA Dynamo is a new serving framework. For managed endpoints, the roadmap pointed to deeper integration with the rest of the AWS AI landscape. Inference recommendations now run from Amazon SageMaker AI Studio, the SageMaker Python SDK, and the coding agents engineers already use, and endpoints can now route requests across Regions and accounts.

#Amazon Web Services#Amazon SageMaker#Amazon SageMaker HyperPod#NVIDIA Dynamo#Amazon SageMaker Inference Gateway#Amazon SageMaker Global Inference Gateway#Amazon SageMaker AI Studio#Amazon SageMaker Python SDK#vLLM#SGLang#TensorRT-LLM

原文约 21 分钟读完阅读原文
群聊

还没有评论,来说说你的看法。