Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Doc2Harness

AI overview

Sign in and the AI will write an overview from our coverage.

Headlines · 1

  1. Doc2Harness benchmarks long-horizon harness engineering for coding agents

    Yuyao Ge and co-authors have posted Doc2Harness on arXiv, a two-stage benchmark that evaluates long-horizon harness engineering for coding agents. In Stage 1, a build model inside a fixed builder harness receives an interface-only contract skeleton and a development document consolidating behavior across twelve frontier harnesses, then constructs a candidate harness in an isolated container; in Stage 2, a scripted mock model drives the frozen candidate harness end to end in separate containers, with scoring across model requests, files on disk, process exit codes and command output. The authors report that across nineteen frontier models, three independent builds each and 328 test cases, GPT-6 Astra had the highest pass rate at 53.05%, while the weakest model reached 27.44%—a gap of roughly 26 percentage points—and even the strongest model failed nearly half the cases.

    Hugging Face · Papers · 🔥 0

Experience and discussion from the community

Share my Doc2Harness experienceAsk about Doc2Harness

Nobody has shared their experience with Doc2Harness yet.