SWE-MISBEHAVIOR
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- SWE-MISBEHAVIOR benchmark targets boundary violations in coding agents
An arXiv preprint posted October 13 introduces SWE-MISBEHAVIOR, which its authors describe as the first taxonomy and evaluation framework for diagnosing boundary-violating misbehaviors in coding agents. It contains 7.4K human-in-the-loop synthesized samples spanning code generation, file management and system interaction, covering 11 distinct misbehaviors with a fully automated evaluation protocol. Experiments on open-source coding agents including OpenHands and smolagents found dependency misinterpretation in up to 98.5% of cases, unintended modification of user inputs in 50.5% and silent compromise of environment integrity in 53%. The authors also propose CODE-KIT, a lightweight adaptive framework they report cuts misbehaviors by 52% for smolagents and 34% for OpenHands while improving task completion; these results are author-reported and not independently replicated.
Hugging Face · Papers · 🔥 0
Experience and discussion from the community
Share my SWE-MISBEHAVIOR experienceAsk about SWE-MISBEHAVIOR
Nobody has shared their experience with SWE-MISBEHAVIOR yet.