MaxxBench

Headlines · 1

1. 3d
   MaxxBench Tests Whether Coding Agents Adapt to Revised Requirements

   MaxxBench changes the instructions or starting code of 89 DeepSWE tasks to test whether coding agents can revise familiar engineering work when requirements change. The authors report that some agents recognized the new requirement but reverted to expected hidden-test behavior, and one model replaced a correct implementation with another model's answer to the original task. The authors publish the mutations, reference solutions, tests, and agent trajectories for independent study.

   Hacker News 首页（100+ 分） · 🔥 0

Experience and discussion from the community

Share my MaxxBench experienceAsk about MaxxBench