AWS推出SageMaker AI跨区域推理路由
AWS宣布Amazon SageMaker AI的跨区域推理路由已在大多数商业AWS区域正式可用,支持实时和异步推理。其推理路由器可统一调用不同区域和账户中的端点及推理组件,优先使用健康的本地容量,在必要时转移新请求,并在恢复后逐步恢复本地流量。AWS称该功能不额外收费,但仍按SageMaker AI和CloudWatch标准价格计费;该功能不支持响应流式传输,并受部分配额和区域限制。
这项功能为SageMaker AI用户提供了托管式跨区域故障转移方案,无需自行构建和运行路由逻辑,但可靠性数据目前主要来自AWS陈述。
Serving a model from more than one AWS Region is a well-established pattern for highly available inference: when one Region runs short of GPU capacity, requests move to another. The pattern is simple to describe and hard to operate. Until now, if you served models on Amazon SageMaker AI managed endpoints, cross-Region routing was a piece you still had to build yourself.
Today, we’re introducing cross-Region inference routing in Amazon SageMaker AI. It uses a new resource, the inference router, that gives your application one invocation path across the endpoints and inference components you already run, in one or more AWS Regions and AWS accounts. You register the destinations that serve a model, and your application invokes the router instead of a specific endpoint. The router keeps requests in its own Region while the local destination is healthy, shifts new requests to another registered destination when it isn’t, and returns traffic gradually after recovery.
In this post, we explain how cross-Region inference routing decides where each request runs, walk through an example with one model in two Regions, and cover monitoring, limits, and what you pay. For a cross-account deployment that serves several models, see How Salesforce is advancing global-scale model serving with cross-Region inference routing in Amazon SageMaker AI.
#AWS#Amazon SageMaker AI#SageMaker AI inference router#Amazon CloudWatch#AWS PrivateLink