Qwen3-4B

Headlines · 5

1. 2d
   Waypoint-OPD gates policy-shift distillation with a mid-training teacher

   An arXiv paper introduces Waypoint-OPD, a new take on policy-shift distillation, which reuses the difference between a teacher model's log probabilities before and after reinforcement learning (RL) as supervision on student-generated trajectories. The authors argue that this endpoint shift alone cannot show whether local action preferences evolved consistently during teacher training, since the same endpoint difference can come from early and late shifts that reinforce or partly cancel each other. Waypoint-OPD therefore uses an intermediate teacher checkpoint from the same RL run to compare early and late shifts in relative action preferences at student-visited prefixes, keeping the endpoint shift only when the two point in the same direction and suppressing it when they oppose; this adds just one extra frozen-teacher scoring pass. Using RL-trained Qwen3-4B teachers with Qwen3-8B, Qwen3-14B and Qwen3-30B-A3B students on mathematical and code reasoning, the method posts the highest domain-average score among compared distillation methods in all six domain-student settings, and ablations with the Qwen3-8B student show that removing or shuffling the gates degrades math reasoning.

   Hugging Face · Papers · 🔥 0
2. 3d
   ActionOPD: action-level on-policy distillation for LLM agents

   An arXiv preprint introduces ActionOPD, an action-level on-policy distillation method for language-model agents. At states the student actually encounters, it samples actions from both the student and a black-box teacher and rewards student actions by their semantic agreement with the sampled teacher actions, aligning behavior without needing teacher token probabilities or a shared tokenizer. Training Qwen3-4B and Qwen3-8B with two teachers on curated tool-use tasks yields average gains of 3.3–6.9 percentage points across four out-of-distribution benchmarks, and on τ²-bench it is competitive with offline distillation and token-level OPD; the authors also explore a Self-ActionOPD variant using a same-model teacher with privileged context.

   Hugging Face · Papers · 🔥 0
3. 4d
   CoSIR: Training LLMs for multi-agent collaboration with verifiable rewards

   An arXiv paper posted on 13 Oct 2026, "Training LLMs for Multi-Agent Collaboration with Verifiable Information Rewards," introduces CoSIR (Collaboration via Surprisal-based Information Rewards), a post-training approach that uses a simple, verifiable environment to teach collaboration behaviors meant to generalize to more complex, unseen tasks. Its Bayesian evaluator scores shared information by the reduction in team surprisal — the sum of negative log probabilities assigned to the correct option given each agent's available evidence — scores requests by the potential reduction in the same quantity if the requested information were truthfully provided, and adds separate rewards for using received information in private choices and calculations. The authors post-train Qwen3-4B, Qwen3-8B and Gemma-3-4B-it through self-play and test transfer to seven unseen natural-language collaboration tasks: CoSIR achieves the highest aggregate score while remaining competitive on general tasks, and behavioral analyses show more disclosure of private evidence and fewer judged failures to use teammates' input. The paper runs 23 pages.

   Hugging Face · Papers · 🔥 0
4. 4d
   DeltaTTG trains multi-turn agents with a Token-to-Go progress meter

   An arXiv paper introduces DeltaTTG, which learns a reusable progress signal through a shared progress meter for reinforcement learning of multi-turn agents; its supervision target, Token-to-Go (TTG), is the number of tokens remaining to be generated until task completion, used as an observable proxy for remaining completion effort. The method uses low-cost supervision from recorded trajectories of diverse policies without task-specific progress rules, and during actor training, changes in the frozen meter's estimates redistribute outcome-derived advantages across steps without altering the reward. On ALFWorld, PaperSearchQA and BFCLv3, DeltaTTG improves the average score over GRPO by 5.7 and 7.2 points with Qwen3-4B and Llama-3.1-8B-Instruct respectively, outperforms all compared RL baselines on average while keeping actor-training GPU-hours close to GRPO, and the authors also report improved sample efficiency and out-of-domain performance; code is said to be available.

   Hugging Face · Papers · 🔥 0
5. 5d
   TPPO: Tracking-Value PPO tops standard PPO for LM reasoning

   An arXiv preprint introduces Tracking-Value PPO (TPPO), which analyzes how successive prefix values relate and shows they admit a Bayesian decomposition with additive updates in log-odds space; the method uses a cumulative value parameterization that sums position-wise scalar outputs into value logits, learned from terminal-outcome supervision. The authors report aggregate mathematical reasoning gains over PPO of 7.0 and 8.4 points on Qwen3-4B-Base and Qwen3-8B-Base respectively, along with improved out-of-distribution generalization. They say gains also extend to Qwen3-4B, Qwen2.5-Math-1.5B and Qwen3.5-9B, plus countdown and scientific coding tasks, with code released publicly.

   Hugging Face · Papers · 🔥 0

Experience and discussion from the community

Share my Qwen3-4B experienceAsk about Qwen3-4B