SageMaker HyperPod新增三层推理数据捕获
Amazon Web Services为Amazon SageMaker HyperPod推出内置Data Capture,可在请求路径的三个独立层级将实时推理输入和输出记录到Amazon S3。配套演练使用合成流量,通过Amazon Athena查询捕获数据、用SQL标注,再微调家用电器客服路由模型;相关脚本、清单、数据集和预期输出已发布至GitHub仓库。其准确率对比属于厂商演示,并非独立的真实环境评估。
这项能力为生产模型团队提供了一条从捕获推理数据、使用Amazon Athena分析,到将记录用于模型改进的内置流程。
Foundation models deployed on Amazon SageMaker HyperPod meet inputs that no pre-curated dataset covers: domain-specific jargon, unexpected formats, new products, and changing user behavior. Without a record of production inference data, teams can’t detect drift, find failure patterns, or use real interactions to improve the model.
Amazon SageMaker HyperPod now provides Data Capture, a built-in capability that records live inference inputs and outputs to Amazon Simple Storage Service (Amazon S3) at three independent tiers, with minimal latency impact and zero impact on model availability.
In this post, we walk through one complete example on a single SageMaker HyperPod cluster. A home-appliance support router captures its production traffic, Amazon Athena queries surface a new-product blind spot, a SQL policy labels the captured records, and a fine-tune on that data teaches the router the new product. We measure accuracy before and after, on data that could not have existed before the model was deployed. The inference operator serves the model on one node, and the training operator runs the fine-tuning on another.
We use synthetic data for the walkthrough. The scripts, manifests, SQL, synthetic datasets, and expected outputs are available in the accompanying GitHub repository.
#Amazon SageMaker HyperPod#Amazon SageMaker HyperPod Data Capture#Amazon Web Services#Amazon S3#Amazon Athena#Amazon Elastic Kubernetes Service#SageMaker Ground Truth#Amazon Key Management Service#Amazon Comprehend#SageMaker HyperPod training operator