创建

登录 ReadmeX

或

了解我们如何处理你的信息: 隐私政策

ReadmeX

陪你读懂 AI 动态

今天,想先了解什么?

选一份动态总结,或直接聊聊你的问题。

社区和关注动态总结只属于你自己。

资讯

Apple 发布 RISED:用评分量规做智能体多环境训练

AI 总结

Apple 机器学习研究团队发布论文 RISED,提出用评分量规(rubrics)引导单一 LLM 智能体在多个交互环境中的联合训练。方法上,由 LLM 评审用跨环境共享的量规词表为每次 rollout 打标签,据此选择与混合环境批次行为构成一致且重叠更少的数据;正面量规为同策略自蒸馏教师提供特权上下文做 token 级监督,负面量规则引导后续 rollout 避开反复出现的失败模式。论文称在多个模型骨干上,RISED 取得各环境最高平均通过率,并在每个单独环境中排名第一或第二。

深度见解

它把评分量规从单纯奖励扩展为数据筛选与策略监督信号,为多环境智能体强化学习提供了一条可复用的思路。

research area Data Science and Annotation, research area Speech and Natural Language Processing

content type paperpublished October 2026

RISED: Rubrics for Agentic Multi-Environment Selection and Self-Distillation

AuthorsJingtan Wang†**, Sirajul Salekin, Young mok Jung, Javier Movellan, Bryan Kian Hsiang Low†, Manjot Bilkhu

View publication

Training a single LLM agent jointly across diverse interactive environments has attracted increasing attention as a route to generalist agents. Existing curriculum and data-selection strategies often allocate training at the environment level or prioritize local reward-based signals, without explicitly considering relationships between current rollouts across environments for prompt-group selection. Meanwhile, as environments are learned at different rates, all-failure and all-success rollout groups can coexist within a batch, leaving those data without group-relative reward signals. Both challenges highlight limitations of relying solely on scalar rewards in multi-environment RL: they provide limited information about cross-environment relationships and no within-group reward contrast when rewards are identical. This motivates richer textual feedback, such as rubrics describing rollout behaviours, to guide learning. Beyond rubrics’ usage as reward, we repurpose rubrics to guide both online data selection and policy supervision. An LLM judge tags each rollout using a predefined rubric vocabulary shared across environments. The resulting profiles guide the selection of data that aligns with the overall behavioural composition of the mixed-environment batch while limiting overlap with already-selected data. Available positive rubrics (describing desired behaviours) provide privileged context for an on-policy self-distillation teacher, supplying additional token-level supervision, while negative rubrics (describing undesired behaviours) guide subsequent rollout generation away from recurring failure modes. Together, these components form RISED. Across model backbones, RISED achieves the highest mean pass rate across environments and ranks first or second in every individual environment. Rubric-based analysis of RISED can further characterize the behavioural changes accompanying these gains.

#Apple#Apple Machine Learning Research#National University of Singapore

原文约 2 分钟读完阅读原文
群聊

还没有评论,来说说你的看法。