Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

SWE-Game benchmark tests whether coding agents can build games

AI summary

An arXiv paper introduces SWE-Game, a benchmark of 247 tasks grounded in 41 executable reference Godot games spanning 13 gameplay categories in 2D and 3D, with five task types: brief-to-game, implementation from a design document, skeleton completion, repair of 83 injected-fault cases, and Godot-to-Unity porting. Across six evaluated models, Opus5 scored highest overall in all five task types, yet best overall scores on the three construction tasks stayed below 60 out of 100, with Brief-to-Game at 50.38; the authors flag requirement omissions and gameplay logic errors as the predominant problems. The paper reports executable checks reaching 92.59% balanced accuracy on human-labeled behaviors from 100 agent-built games versus 78.41% for a video-based VLM judge, and rubric visual scores with a 0.829 Spearman correlation to human ratings.

In-depth insight

Game development gives coding agents a long-horizon, automatically verifiable testbed, and this benchmark suggests current models still struggle to ship complete games that match the brief.

Computer Science > Artificial Intelligence

arXiv:2609.33678 (cs)

[Submitted on 27 Sep 2026 (v1), last revised 7 Oct 2026 (this version, v3)]

Title:SWE-Game: Can Coding Agents Build the Games We Want?

Authors:Xiaoyu Chen, Lai Wei, Jin Wang, Xiangyu Zou, Ruochen Fan, Enze Luo, Mingzhe Yao, Jiahui Zhu, Yuhua Wen, Linghe Kong, Weiran Huang

View a PDF of the paper titled SWE-Game: Can Coding Agents Build the Games We Want?, by Xiaoyu Chen and 10 other authors

#SWE-Game#Godot#Unity#Claude Opus 5

Original · 3 min readRead the original
Group chat

No comments yet. Start the conversation.