UniSkill lets one policy co-evolve with its own skillbank
An arXiv preprint introduces UniSkill, in which a shared policy interacts with the environment and proposes skillbank edits (Add, Update, or No Edit) drawn from the resulting trajectories, so the policy and skillbank co-evolve. Contrastive action feedback supplies an actor-alignment signal by measuring how swapping the retrieved skill for a proposed one changes the actor's action log-likelihood gap between previously collected successful and failed trajectories on the same task, avoiding a new rollout per proposal; skill-edit support regularization is added to preserve exploration. The authors report 98.4% success on ALFWorld and 84.7% on WebShop with stable joint training, and say the approach still works with a smaller backbone policy; code is released.
It offers a rollout-free way to judge skill proposals, relevant to researchers building LLM agents that improve themselves over time.