τ²-Bench

Headlines · 3

1. 3d
   SpecAgent has weak models draft tool-call trajectories and strong models verify

   An arXiv paper introduces SpecAgent, a collaboration protocol that turns a strong model from the author of each agent step into the reviewer of steps drafted by a weaker model. The weak model executes read-only tool calls while drafting so each step is grounded in real observations, and stops at the first action that would modify the environment; the strong model then verifies the drafted segment in a single prefill pass, commits the longest prefix it judges semantically valid, and repairs the first rejected step. The authors report SpecAgent comes within one point of the strong model's success rate on τ²-Bench while roughly halving memory traffic and wall time, recovers 44–75% of the quality gap on AutomationBench and VitaBench, and in an edge-cloud MCP-Atlas deployment closes 53–75% of the weak-strong quality gap with the strong model generating at most half as many tokens.

   Hugging Face · Papers · 🔥 0
2. 4d
   ActionOPD: action-level on-policy distillation for LLM agents

   An arXiv preprint introduces ActionOPD, an action-level on-policy distillation method for language-model agents. At states the student actually encounters, it samples actions from both the student and a black-box teacher and rewards student actions by their semantic agreement with the sampled teacher actions, aligning behavior without needing teacher token probabilities or a shared tokenizer. Training Qwen3-4B and Qwen3-8B with two teachers on curated tool-use tasks yields average gains of 3.3–6.9 percentage points across four out-of-distribution benchmarks, and on τ²-bench it is competitive with offline distillation and token-level OPD; the authors also explore a Self-ActionOPD variant using a same-model teacher with privileged context.

   Hugging Face · Papers · 🔥 0
3. Oct 5
   DAEDALUS: Bootstrapping agent memory from self-generated tasks

   An arXiv paper introduces DAEDALUS, a method that bootstraps reusable agent memory from self-generated practice without existing tasks or an oracle verifier. An explorer agent creates challenging but solvable tasks while a solver agent attempts them; heuristics derived from solver failures are accepted only after repeated in-context success, then consolidated into a memory bank. Across AppWorld, τ²-bench and AutomationBench, it improves mean success rates by up to 15.9 points and pass^5 by up to 2.2x over a no-memory baseline, at lower inference cost than most comparable methods.

   Hugging Face · Papers · 🔥 0

Experience and discussion from the community

Share my τ²-Bench experienceAsk about τ²-Bench