ALFWorld

Headlines · 3

1. 4d
   Reality check: self-distillation methods in agentic RL show limited gains over GRPO

   A new arXiv paper offers a reality check on self-distillation in agentic reinforcement learning. The authors compared ten representative methods against a GRPO baseline on ALFWorld, ScienceWorld, Search-QA and WebShop across three model sizes, running 132 training runs that totaled 12,124.4 recorded H800 GPU-hours, with repeated evaluations over eight rollout seeds for the pretrained backbones and all eleven trained methods. They report three findings: gains over GRPO remain limited in scope, with no method consistently outperforming it; strong final performance can mask unstable training; and additional training cost does not consistently translate into better performance.

   Hugging Face · Papers · 🔥 0
2. 5d
   DeltaTTG trains multi-turn agents with a Token-to-Go progress meter

   An arXiv paper introduces DeltaTTG, which learns a reusable progress signal through a shared progress meter for reinforcement learning of multi-turn agents; its supervision target, Token-to-Go (TTG), is the number of tokens remaining to be generated until task completion, used as an observable proxy for remaining completion effort. The method uses low-cost supervision from recorded trajectories of diverse policies without task-specific progress rules, and during actor training, changes in the frozen meter's estimates redistribute outcome-derived advantages across steps without altering the reward. On ALFWorld, PaperSearchQA and BFCLv3, DeltaTTG improves the average score over GRPO by 5.7 and 7.2 points with Qwen3-4B and Llama-3.1-8B-Instruct respectively, outperforms all compared RL baselines on average while keeping actor-training GPU-hours close to GRPO, and the authors also report improved sample efficiency and out-of-domain performance; code is said to be available.

   Hugging Face · Papers · 🔥 0
3. Oct 6
   UniSkill lets one policy co-evolve with its own skillbank

   An arXiv preprint introduces UniSkill, in which a shared policy interacts with the environment and proposes skillbank edits (Add, Update, or No Edit) drawn from the resulting trajectories, so the policy and skillbank co-evolve. Contrastive action feedback supplies an actor-alignment signal by measuring how swapping the retrieved skill for a proposed one changes the actor's action log-likelihood gap between previously collected successful and failed trajectories on the same task, avoiding a new rollout per proposal; skill-edit support regularization is added to preserve exploration. The authors report 98.4% success on ALFWorld and 84.7% on WebShop with stable joint training, and say the approach still works with a smaller backbone policy; code is released.

   Hugging Face · Papers · 🔥 0

Experience and discussion from the community

Share my ALFWorld experienceAsk about ALFWorld