Doc2Harness
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- Doc2Harness benchmarks long-horizon harness engineering for coding agents
Yuyao Ge and co-authors have posted Doc2Harness on arXiv, a two-stage benchmark that evaluates long-horizon harness engineering for coding agents. In Stage 1, a build model inside a fixed builder harness receives an interface-only contract skeleton and a development document consolidating behavior across twelve frontier harnesses, then constructs a candidate harness in an isolated container; in Stage 2, a scripted mock model drives the frozen candidate harness end to end in separate containers, with scoring across model requests, files on disk, process exit codes and command output. The authors report that across nineteen frontier models, three independent builds each and 328 test cases, GPT-6 Astra had the highest pass rate at 53.05%, while the weakest model reached 27.44%—a gap of roughly 26 percentage points—and even the strongest model failed nearly half the cases.
Hugging Face · Papers · 🔥 0
Experience and discussion from the community
Share my Doc2Harness experienceAsk about Doc2Harness
Nobody has shared their experience with Doc2Harness yet.