Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

SWE-Bench Verified

AI overview

Sign in and the AI will write an overview from our coverage.

Headlines · 17

  1. mini-swe-agent v2 launches as a minimal software-engineering agent

    The SWE-agent project released mini-swe-agent v2, a compact command-line AI agent whose agent class contains roughly 100 lines of Python and uses only Bash for actions. The project says it scores above 74% on SWE-bench Verified, now powers Ramp SWE-Bench, and outperforms Claude Code and Codex on DeepSWE, though these performance claims are not independently verified in the supplied report.

    GitHub Trending · Python · 🔥 4
  2. Vera Rubin NVL72 Leads Provisional MLPerf Endpoints Agentic Results

    NVIDIA reports that Vera Rubin NVL72 delivered more than 10 times the throughput per provisioned kilowatt of GB300 NVL72 on the Kimi K3 workload in MLPerf Endpoints v1.0, while meeting a shared minimum end-to-end interactivity threshold of 50 tokens per second per user. Its submissions also covered Qwen3.6-35B-A3B and DeepSeek-V4.1-Flash, with checks of generated tool calls, workflow intent and task-solving performance. GB300 results were verified by MLCommons Association, but the Vera Rubin results remain provisional and subject to peer review.

    NVIDIA Technical Blog · 🔥 0
  3. JD Cloud launches upgraded JoyCode coding agent

    JD Cloud launched an upgraded JoyCode intelligent coding tool centered on end-to-end autonomous decision-making. It combines project context, CodeGraph, CodeWiki, and an Agent Loop for planning, execution, verification, and iteration, while adding multi-agent collaboration, configurable models, and reusable tools and workflows. JD Cloud claims JoyCode serves more than 10,000 internal developers, contributes over 80% of AI-generated code, and helps shorten the development cycle by about 40%.

    掘金 · 人工智能 · 🔥 0
  4. Penn State study measures coding agents’ termination lag

    A Penn State paper titled “Termination Lag: Coding Agents Solve Early but Stop Late” measures the interval between a coding agent’s first correct patch and when it stops. The study logged repository snapshots during agent runs and replayed them against official SWE-bench tests. Across five models, two agent frameworks and three SWE-bench task sets, the report says a median 35%–60% of steps in successful runs occurred after the first correct patch, while some later changes turned a correct solution into an incorrect one. The findings are based here on a secondary account of the paper.

    掘金 · 人工智能 · 🔥 0
  5. Tsinghua's CoACT cuts coding-agent token use by 33%

    Machine Intelligence reports that researchers from Tsinghua University proposed CoACT, an observation-compression method that preserves a coding agent’s next action rather than measuring similarity to the original observation. On 200 SWE-bench Verified instances with three agent models, CoACT reportedly reduced total token consumption by 33.0% on average, while average PASS@1 rose from 69.0% to 71.0% and interaction steps fell from 56.5 to 51.2. The study reports that CoACT can also be combined with AgentDiet, with estimated total cost falling to $25.88 while task success remained similar.

    机器之心 · 🔥 0
  6. HOPE brings higher-order pruning to MoE language models

    The authors introduce HOPE, a released expert-pruning method for Mixture-of-Experts language models that considers pairwise interactions rather than scoring each expert independently. They evaluated it on Qwen3.5-122B, Qwen3.5-35B, and GLM-4.5-Air, reporting the best average rank at 50% pruning and gains of up to 6.1% on agentic coding tasks over the next-best method. HOPE uses the same one-pass calibration as other pruning methods, while its added quadratic-programming step reportedly takes one to two seconds per layer.

    Hugging Face · 🔥 0
  7. CR-Bench Exposes Limits in AI Code Review and LLM Scoring

    Nutanix researchers' CR-Bench-v2 tested eight code-review agents across 584 real pull requests with known defects; the strongest agent detected only about one-third, while Gemini Code Assist recognized 8.1%. A separate replay study by Masoud Fetanat found that a strong LLM judge awarded high scores to invalid Python, prompting the recommendation to report syntax-pass rates alongside model-based scores.

    掘金 · 人工智能 · 🔥 0
  8. Parsimony Benchmark Measures AI Coding Complexity

    Reddit user lasushin released Parsimony, an open-source benchmark that counts syntax units in AI-generated patches and compares them with human fixes for the same issues. Its initial analysis reports a median of about 1.9 times the human-fix code on SWE-bench Verified and about one times it on DeepSWE, while harness comparisons using the same two open-weight models ranged from 0.80 times for Codex to 0.99 times for mini-SWE-agent and OpenClaw.

    r/LocalLLaMA · 🔥 0
  9. Co-Design Bench exposes gaps in real-world coding agents

    Alibaba researchers and researchers from several Asian universities built Co-Design Bench from real coding-change requests, including multimodal inputs such as images, audio, video, and documents. The paper reports that GPT-6 Astra led the ten tested models by satisfying about 70% of task requirements on average, but was fully correct in only 26.4% of tasks; performance also varied with the agent harness.

    The Decoder · 🔥 0
  10. Chinese Edition of Anthropic’s “Building Effective Agents”

    A Chinese translation and illustrated edition of Anthropic’s “Building Effective Agents,” credited to original authors Erik Schluntz and Barry Zhang and dated December 19, 2024. The report describes it as a complete translation with seven redrawn architecture diagrams. It distinguishes workflows, which follow predefined code paths, from agents, which dynamically direct their own processes and tool use. The guide covers prompt chaining, routing, parallelization, orchestrator-workers, and evaluator-optimizer patterns, and recommends starting with direct LLM API calls, keeping designs simple and transparent, and carefully designing tool interfaces. The supplied report does not independently verify the translation’s completeness or editorial fidelity.

    掘金 · 人工智能 · 🔥 0
  11. Google Researchers Open-Source EnvHarness for Adaptive Agent Training

    Google researchers have open-sourced EnvHarness, a framework that adapts training environments to an agent’s weaknesses without modifying the agent, underlying environment, or evaluation system. It wraps existing environments through standard reset() and step() interfaces and includes EnvRigger to identify weaknesses and generate targeted changes. The researchers reported gains of 2.9–3.7 percentage points across WebArena, SWE-bench Verified, and ALFWorld for several tested models.

    InfoQ 中文 AI&大模型 · 🔥 0
  12. LMSYS trains Qwen3 coding agent with asynchronous RL on AMD MI355X

    LMSYS Blog reports that its team trained Qwen3-30B-A3B-Instruct for coding-agent tasks using fully asynchronous agentic reinforcement learning on AMD Instinct MI355X, with Miles, SGLang, Megatron-LM, and Harbor. On a vendor-run, 100-task stratified subset of SWE-bench Verified, pass@1 rose from 7.3% to 27.5% after supervised fine-tuning and agentic RL. The system used separate inference and training nodes, capped trajectories at 64K tokens, and moved sandboxes to E2B cloud microVMs, supporting 384 concurrent trajectories and about 3,480 trajectories per hour.

    LMSYS Blog · 🔥 0
  13. Coding-Agent Harness Changes Raise Costs Without Reliable Gains

    Shandong University researchers Liu and Han compared Claude Code, OpenCode, and mini-SWE-agent while holding the language model constant. Most performance differences were similar to rerun noise, while the harnesses consumed two to three times as many tokens as the leanest option; OpenCode showed a significant deficit on the larger task pool. A small cross-check on Terminal-Bench 2.1 showed the same pattern.

    The Decoder · 🔥 0
  14. ModelAnalysis.si launches LLM cost, benchmark, and VRAM comparison tool

    Developer Zack Ku announced ModelAnalysis.si, a free dashboard for comparing LLM benchmarks, API token pricing, VRAM requirements, and self-hosting economics. The tool supports side-by-side and radar-chart comparisons, API spend simulation, cloud-GPU break-even calculations, and deployment commands for Ollama and vLLM. According to Ku, the site draws on benchmark and pricing data covering more than 250 models, while a later post says it covers over 280 models. These coverage and accuracy claims have not been independently verified.

    r/OpenAI · 🔥 0
  15. HarnessAdapt-RL trains coding agents by adapting their harness during RL

    A new arXiv preprint by Lingxiao Tang, Wei Tao, Yiwen Guo and Lingfeng Bao introduces HarnessAdapt-RL, a reinforcement-learning method that trains a coding agent's policy while also adapting its harness. A frozen teacher generates task-specific mechanism-level skills for flawed repair strategies, while a separate teacher model evolves tools and hooks, keeping each change only if it improves the observed pass rate or observed trainable rate without lowering either. With a 9B base model, the authors report pass@1 gains over GRPO of 9.8, 12.6 and 6.4 percentage points on SWE-bench Verified, Multilingual and Pro, and say it outperforms GRPO with the human-designed OpenHands harness; the results are self-reported and not independently verified.

    Hugging Face · Papers · 🔥 0
  16. Study finds dynamic concurrency in coding agents helps long tasks but hurts short ones

    Researchers from Nanjing University and UIUC ran paired experiments on the dynamic concurrency mechanisms of three coding agents — Codex, Claude Code and Kimi Code — across 354 software engineering tasks and 2,124 executions, analysing full main-agent and sub-agent traces. Results varied by task type and system: on LoopsBench, Claude Code's completion rate rose from 7.1% sequentially to 21.4% with concurrency, and Codex rose from 34.6% to 57.7% on NL2Repo; on SWE-bench Verified, Claude Code fell from 83.0% to 59.0% and Kimi Code from 77.0% to 59.0%, while Codex stayed at 71.0% in both modes. Trace analysis produced a taxonomy of 4 categories, 13 subcategories and 28 concurrency failure modes (such as missing owners, pseudo-concurrency, load imbalance, mistimed result merging and shared-environment conflicts), plus three mechanisms where concurrency paid off: complementary work, parallel exploration and independent verification.

    新智元 · 🔥 0
  17. Inside the AI data boom: from labeling to RL environments

    In this Silicon Valley 101 podcast episode, He Yunzhong, who works on post-training and evaluation research at Scale AI, and Sun Yiyou, a UC Berkeley postdoc on the Agents’ Last Exam (ALE) project, unpack the fast-growing but opaque market for AI training data. They note AfterQuery was valued at $300 million in its April A round and, per media reports, $3.2 billion by September, while Bespoke Labs announced $40 million in combined seed and A funding in July, and trace the shift from crowd labeling and pre-training toward rubrics, verifiable signals and RL environments bundling tasks, tools and verification. They also discuss procurement bottlenecks in vertical domains, the blurred line between benchmarks and data sales — including a claim that some data firms sold evaluation data — expert fabrication and contamination, and OpenAI's February decision to stop reporting SWE-bench Verified scores. Guest views are personal.

    36氪 人工智能 · 🔥 0

Experience and discussion from the community

Share my SWE-Bench Verified experienceAsk about SWE-Bench Verified

The Past and Present of Gödel Machines: The Research Path Toward RSI

A coding agent can fix a program, yet the next time it encounters an error, it may still follow the same process of debugging, editing, and testing. The task outcome may have changed, but the method that produced it may not have. When the bottleneck lies precisely in that method, who is responsible for improving the program responsible for improvement? Gödel Machine (GM) [1] framed this as a proo…

RSIGödelMachine自我改进开放式进化Agent