Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Coding-Agent Harness Evolution Can Trigger Catastrophic Forgetting

AI summary

A paper by Shangziqi Zhao, Yitong Zhang, and Jia Li studies continual harness self-evolution across three software-engineering datasets, six evolution directions, and two harness-evolution methods. The authors report that later evolution can substantially reduce performance on earlier tasks: with DeepSeek-V4-Flash-0731, Meta-Harness reportedly fell from 94.0% on SWE-Bench after evolution to 32.0% by the fifth round of subsequent FeatureBench evolution. The paper also presents insights intended to guide continual harness self-evolution.

In-depth insight

The study highlights a reliability problem for coding agents whose surrounding harness evolves across multiple task types, while the reported 66% relative performance decrease shows that gains on a new dataset may coincide with severe forgetting of previous capabilities.

Computer Science > Software Engineering

arXiv:2611.03219 (cs)

[Submitted on 2 Nov 2026]

Title:Can Coding Agents Sustain Continual Evolution without Forgetting?

Authors:Shangziqi Zhao, Yitong Zhang, Jia Li

View a PDF of the paper titled Can Coding Agents Sustain Continual Evolution without Forgetting?, by Shangziqi Zhao and 2 other authors

#DeepSeek-V4-Flash-0731#Meta-Harness#SWE-Bench#FeatureBench

Original · 3 min readRead the original
Group chat

No comments yet. Start the conversation.