WebShop

Headlines · 2

1. 4d
   Reality check: self-distillation methods in agentic RL show limited gains over GRPO

   A new arXiv paper offers a reality check on self-distillation in agentic reinforcement learning. The authors compared ten representative methods against a GRPO baseline on ALFWorld, ScienceWorld, Search-QA and WebShop across three model sizes, running 132 training runs that totaled 12,124.4 recorded H800 GPU-hours, with repeated evaluations over eight rollout seeds for the pretrained backbones and all eleven trained methods. They report three findings: gains over GRPO remain limited in scope, with no method consistently outperforming it; strong final performance can mask unstable training; and additional training cost does not consistently translate into better performance.

   Hugging Face · Papers · 🔥 0
2. Oct 6
   UniSkill lets one policy co-evolve with its own skillbank

   An arXiv preprint introduces UniSkill, in which a shared policy interacts with the environment and proposes skillbank edits (Add, Update, or No Edit) drawn from the resulting trajectories, so the policy and skillbank co-evolve. Contrastive action feedback supplies an actor-alignment signal by measuring how swapping the retrieved skill for a proposed one changes the actor's action log-likelihood gap between previously collected successful and failed trajectories on the same task, avoiding a new rollout per proposal; skill-edit support regularization is added to preserve exploration. The authors report 98.4% success on ALFWorld and 84.7% on WebShop with stable joint training, and say the approach still works with a smaller backbone policy; code is released.

   Hugging Face · Papers · 🔥 0

Experience and discussion from the community

Share my WebShop experienceAsk about WebShop