LoLBench
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- LoLBench Tests Coding Agents on Large-Scale Feature Delivery
The LoLBench paper introduces a benchmark for evaluating coding agents that must turn community feature proposals into cross-module implementations in large, evolving repositories. Its 100 tasks cover 29 systems, eight languages and five domains, with repositories averaging about 2.4 million lines of code. The paper reports that the best of 28 model-and-scaffold combinations resolved 14% of tasks, while all combinations had a median resolution rate of 2%.
新智元 · 🔥 0
Experience and discussion from the community
Share my LoLBench experienceAsk about LoLBench
Nobody has shared their experience with LoLBench yet.