创建

登录 ReadmeX

或

了解我们如何处理你的信息: 隐私政策

ReadmeX

陪你读懂 AI 动态

今天,想先了解什么?

选一份动态总结,或直接聊聊你的问题。

社区和关注动态总结只属于你自己。

资讯

AWS推出SageMaker AI跨区域推理路由

AI 总结

AWS宣布Amazon SageMaker AI的跨区域推理路由已在大多数商业AWS区域正式可用,支持实时和异步推理。其推理路由器可统一调用不同区域和账户中的端点及推理组件,优先使用健康的本地容量,在必要时转移新请求,并在恢复后逐步恢复本地流量。AWS称该功能不额外收费,但仍按SageMaker AI和CloudWatch标准价格计费;该功能不支持响应流式传输,并受部分配额和区域限制。

深度见解

这项功能为SageMaker AI用户提供了托管式跨区域故障转移方案,无需自行构建和运行路由逻辑,但可靠性数据目前主要来自AWS陈述。

Serving a model from more than one AWS Region is a well-established pattern for highly available inference: when one Region runs short of GPU capacity, requests move to another. The pattern is simple to describe and hard to operate. Until now, if you served models on Amazon SageMaker AI managed endpoints, cross-Region routing was a piece you still had to build yourself.

Today, we’re introducing cross-Region inference routing in Amazon SageMaker AI. It uses a new resource, the inference router, that gives your application one invocation path across the endpoints and inference components you already run, in one or more AWS Regions and AWS accounts. You register the destinations that serve a model, and your application invokes the router instead of a specific endpoint. The router keeps requests in its own Region while the local destination is healthy, shifts new requests to another registered destination when it isn’t, and returns traffic gradually after recovery.

In this post, we explain how cross-Region inference routing decides where each request runs, walk through an example with one model in two Regions, and cover monitoring, limits, and what you pay. For a cross-account deployment that serves several models, see How Salesforce is advancing global-scale model serving with cross-Region inference routing in Amazon SageMaker AI.

#AWS#Amazon SageMaker AI#SageMaker AI inference router#Amazon CloudWatch#AWS PrivateLink

原文约 15 分钟读完阅读原文
群聊

还没有评论,来说说你的看法。