AWS发布SageMaker AI多轮强化学习实操指南
AWS Machine Learning Blog 发布了 Amazon SageMaker AI 多轮强化学习(MTRL)端到端指南,涵盖智能体部署、数据集准备、训练、MLflow 监控及按部署设置评估。该流程使用整个多步任务的最终结果作为奖励,无需提供正确的动作轨迹,并支持部署在 Bedrock AgentCore 或客户自有基础设施上的智能体。AWS 在 Wordle 示例中称,gpt-oss-20b 在默认推理强度下的解题率从 61% 提升至 84%,在训练所用的低强度设置下从 15% 提升至 69%,训练令牌成本约为 339 美元。
该指南为只能评估最终结果、缺少标准动作轨迹的业务多步智能体提供了具体训练流程,并记录了奖励设计和训练失败案例。
Most fine-tuning teaches a model to produce a better answer to a single prompt. Agentic tasks are different. An agent that files an expense report or triages a support ticket takes a sequence of actions, and each step depends on what came before. There’s rarely one correct sequence to imitate, and the only reliable feedback is whether the whole sequence succeeded. Supervised fine-tuning has no labels for that, and single-turn reinforcement learning cannot credit a decision made three steps before the outcome.
Multi-Turn Reinforcement Learning (MTRL) on Amazon SageMaker AI trains models on exactly that signal. SageMaker AI sends prompts to an agent you host on Amazon Bedrock AgentCore, or on your own infrastructure, such as Amazon Elastic Kubernetes Service (Amazon EKS), Amazon Elastic Compute Cloud (Amazon EC2), or AWS Fargate, through an AWS Lambda forwarder. The example here uses AgentCore. The agent calls the policy model being trained, acts in your environment over as many turns as the task needs, and reports one reward when the episode ends. You don’t specify the right trajectory, only how to score the result.
#Amazon SageMaker AI#Amazon Bedrock AgentCore#gpt-oss-20b#AWS MLflow#Amazon Elastic Kubernetes Service#Amazon Elastic Compute Cloud#AWS Fargate#Strands