Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

JobBench

AI overview

Sign in and the AI will write an overview from our coverage.

Headlines · 3

  1. Google open-sources Harness², a self-improving agent harness

    Google Research has open-sourced Harness², a framework that recursively rewrites an agent's harness without changing model weights, under Apache 2.0 on GitHub with support for OpenCode and Codex. It combines task-level propose-probe-compose edits with a cross-task improvement library that stores evidence-backed lessons. Google reports gains with Gemini 3.5 Flash and GLM-5 on WorkBuddy-Bench, JobBench and LAB — for example, GLM-5 rose from 64.2 to 75.7 on WorkBuddy-Bench under OpenCode — while noting the project is research, not an officially supported product, and that extra recursion costs more test-time compute.

    r/LocalLLaMA · 🔥 32
  2. Data World synthesizes data environments from scratch and ships a benchmark

    The arXiv paper "Data World: Scaling Data Environments from Scratch" introduces Data World, a multi-agent engine that synthesizes environments, tasks and rubrics from explicit mechanisms, with independent audits checking that targets can be recovered from the observed files. Human preference-based curation selects 100 tasks across 50 held-out worlds and 16 domains for DataWorld-Bench, while the engine also yields 2,135 training tasks from 1,037 disjoint worlds. Evaluation of ten frontier models shows the benchmark remains challenging, particularly for insight discovery, with a consistent computation-insight gap; RL-zero training on the synthesized tasks produces DataWorld-35B-A3B from Qwen3.6-35B-A3B without trajectory distillation, which the authors report improves over its base by 7.06 points on DataWorld-Bench, 14.28 on DAComp, 20.90 on DABstep-Hard and 12.13 on JobBench.

    Hugging Face · Papers · 🔥 0
  3. WorkRover trains long-horizon cowork agents via recursive environment evolution

    An arXiv paper introduces WorkRover, a framework for training long-horizon cowork agents through recursive environment evolution: its harness-driven synthesis has construction agents jointly build semantically aligned work contexts, executable tasks and evidence-grounded verifiers, adapting their actions to workspace state and execution feedback instead of following a fixed pipeline. To support environment evolution the authors propose Factorial Compatibility Replay (FCR), which uses rollout feedback to evaluate combinations of component revisions and selects the smallest validated repair, plus Verifier-to-Turn Credit Assignment (VTCA), which augments trajectory-level GRPO advantages with deterministic workspace-progress signals. Post-training Qwen3.5-35B-A3B with WorkRover yields WorkRover-35B-A3B, which the authors report exceeds the base model by 26.27 percentage points on JobBench, 29.32 points on OfficeQA Pro and 413.25 Elo points on GDPval v2.

    Hugging Face · Papers · 🔥 0

Experience and discussion from the community

Share my JobBench experienceAsk about JobBench

Nobody has shared their experience with JobBench yet.