Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

EvoMap Community

Evolver: Let Exploration Leave Behind Strategies

After an Agent completes a task, what it leaves behind is often more than an answer. There is also an expensive exploration path: which files it read, which tools it tried, where it failed, and what checks it used to confirm the result. The problem is that the next execution may not be able to use those experiences. The model parameters have not changed, and a new session will not naturally inherit the judgments made in the previous round, so the exploration cost already paid is easily paid all over again. The EvoMap research team (Infinite Evolution Lab) addressed this problem with an external experience system in the paper “Evolver: Turning Agent Exploration into Evolving Capabilities.” It does not train the foundation model; instead, it organizes the procedural knowledge generated during execution into strategic assets and feeds them back into the context of subsequent tasks. What deserves more attention here is not “Will Agents automatically become smarter?” but an engineering question: how can one turn a one-off execution record into a basis for action that has applicability conditions, can be retrieved, and can also be revoked? 1. Externalizing Exploration 1.1 Records and Strategies Let us first distinguish between two things. An execution record answers “what happened this time,” while a strategy answers “when encountering a certain situation, how can one act?” The former needs to preserve task details, tool outputs, and the order of events, while the latter needs to extract reusable judgments from those details. Saving all logs does not mean this transformation has been completed. For example, a record may contain multiple directory scans, several failed invocation approaches, and the final checks that passed. If the entire record is stuffed back into the Agent the next time, it still has to determine which steps were necessary and which were merely detours. The experience has not disappeared, but the work of organizing that experience has simply been postponed until after each retrieval. Evolver chooses to externalize reusable operational guidance in advance, making the strategy the primary unit of reuse rather than asking the full trajectory to serve that role directly. This also explains its relationship with static workflow packages. The paper describes Skill as a packaged set of instructions, workflows, and resources, and uses Gene as a compact strategic representation. Both can influence execution, but Evolver focuses on generating strategies from historical interactions and continuously managing the state of those strategies. Merely changing the name does not prove that Gene outperforms Skill on every task; the baseline in this experiment still retains the Agent’s native tools and built-in skills. 1.2 External State Once externalized, the system can be divided into three actions: first, select a strategy based on the task and the current experience repository; next, have the original Agent execute it; finally, update the state of the experience based on the trajectory and feedback. The foundation model handles reasoning and tool use, while the experience system stores, selects, and maintains action guidance. Figure 1: Experience accumulates outside execution and then enters the next round of tasks. The value of this separation is that experience management does not have to be tied to a particular Agent. The paper describes integration methods including hooks, MCP, and host integrations, with the original runtime framework still responsible for execution. However, “without changing parameters” does not mean “without additional computation”: collecting, extracting, reviewing, and retrieving experiences can all incur overhead. The system structure and the cost-benefit analysis need to be discussed separately. 2. Strategic Assets 2.1 The Content of Genes If the entire log is not reused directly, what should a strategy preserve? The paper describes Gene using five types of information: an experience summary, procedural steps, applicability conditions, task-matching signals, and an optional verification plan. These five types of information each have their own role. The summary helps clarify the strategy’s intent, the steps provide the order of action, the applicability conditions explain when it is worth considering, the matching signals help the retrieval system find it, and the verification plan indicates how to check the result after execution. Saving only the steps can easily cause a locally effective method to be applied to an unsuitable environment; saving only the summary may instead produce an “best practice” that cannot be put into practice. There is an important distinction here: applicability conditions are guidance, not runtime-enforced constraints, and the presence of a verification plan does not mean that the strategy has already been verified. After a Gene enters the context, it is still the host Agent that understands and applies it. The asset format can clarify the questions that need to be considered, but it cannot replace checks at the execution layer. 2.2 Content and State Evolver does not ask Genes to carry all the information. Capsule associates a strategy with a specific execution case to provide context for reuse; AntiGene stores actions or approaches that should be avoided; Event records asset operations, while separate source and review records preserve where an asset came from, whether it is currently trusted, and whether it is approved for use. Figure 2: Strategy content, execution cases, and eligibility for use are stored separately. This separation has a very practical reason: strategy content may need to be retained for a long time, but its eligibility for use can change. The paper describes a content-addressed, append-only asset store, with review and source state kept in separate records. Rejecting a strategy or revoking trust in its source does not require overwriting the original content; the system can preserve its identity while changing whether it can be used in the future. As a result, “what was remembered” and “what is currently allowed to be trusted” are no longer the same question. This leaves room for tracking and revocation while avoiding the interpretation of the experience repository as a prompt collection that can only grow. Whether specific paths actually perform these state checks still depends on the integration method; the storage structure alone cannot establish that every entry point is equally and consistently secure. 3. Selection and Updates 3.1 Candidate Admission Strategies can come from more than one source. Once session ingestion is enabled, adapters can collect task context, tool interactions, execution errors, and available results before organizing them into bounded experience material. The paper mentions heuristic sensitive-information redaction, but this is not a complete guarantee of identifying all private information. During extraction, the default automatic drafting path extracts concise procedural cues from the assistant’s expressions and marks them as unproven. Optional LLM distillation further generates structured strategies, cases, and negative guidance. Automatically drafted or model-distilled candidates enter isolated review, while certain explicit submission paths can persist assets directly. Why separate “generating a strategy” from “approving a strategy for use”? Because a successful execution does not mean that every suggestion extracted from that execution is valid, and a failed execution does not mean that there is no worthwhile experience to retain about what to avoid. The extractor proposes candidate knowledge; it cannot automatically promote that candidate into a reliable capability. 3.2 Reuse on Demand After admission, the question turns to selection. If there is too little experience, the Agent will still have to repeat its exploration; if there is too much, irrelevant suggestions may be brought into the context. Evolver provides several retrieval paths: a session-startup hook supplies a bounded number of locally eligible Genes, optional prompt-time recall matches signals against the task description, MCP supports on-demand invocation, and autonomous execution has a heuristic selection mechanism that combines relevance, quality, and historical reuse feedback. Figure 3: Candidate experiences undergo review, and subsequent feedback affects their usage state. These entry points are not all identical rankers. Their context budgets, eligibility checks, and selection strategies differ, and directly retrieving external assets and importing them locally before use may follow different review paths. EvoMap’s network interface extends the sources of experience, but “coming from the network” does not automatically mean “already trusted”: imported assets can initially be marked as untrusted and then explicitly promoted, while some direct retrieval interfaces do not impose the same restrictions. Feedback is then used to change the state of experiences. Depending on the execution path, the system can collect task results, verification results, and reuse signals; an optional trial mechanism supports promotion based on recorded successes and failures, while autonomous execution also considers historical reuse feedback and failure limits. Here, “evolution” first refers to the evolution of external strategies and their eligibility state; it does not mean that the model’s capabilities necessarily improve every time an experience is used. At this point, the overall design is clear: experiences have content, cases, and sources, as well as paths for entering and leaving subsequent execution. The next question is what experimental conditions produced benefits from these experiences. 3.3 System Closed Loop Overall, the system has two interconnected paths: one turns task exploration into experience, while the other returns experience to subsequent execution. Gene, Capsule, retrieval, and review are no longer disconnected terms; they are the connecting points between these two paths. Figure 4: Evolver connects task exploration, experience assets, and subsequent execution into a feedback loop. A task first enters the host Agent, which makes tool calls, retries, and verifies results; the experience extraction module then drafts strategies and extracts signals from those interactions, or performs optional model distillation. The resulting knowledge enters the persistent asset area, where Gene stores action guidance, Capsule preserves execution cases, and events, sources, and review records provide a basis for management. The retrieval and injection modules feed applicable experiences back into the host, corresponding to the feedback path returning to the Agent at the top of the figure: past exploration influences the next action without replacing the original execution framework. Execution results and reuse signals enter experience management, supporting approval, rejection, promotion, or withdrawal from subsequent use. The EvoMap interface connects this local loop to an external experience network, while source tracking, approval policies, execution safeguards, and data and network boundaries form the system’s trust and security layer. Evolver’s overall design therefore does not append a summary at the end of a task; it connects “exploration produces experience” with “experience guides execution” while preserving a feedback path for updating eligibility for use. This closed loop provides a system structure for the continuous accumulation of experience. 4. Reuse Experiments 4.1 Three-Stage Protocol The paper used Evolver v2.0.33 on two runtime frameworks, Claude Code and Codex, each paired with deepseek-v4.1-flash and gpt-5.6-sol. The evaluation covered Terminal-Bench 2.1, SWE-bench Multilingual, SWE-rebench, and GAIA, involving terminal engineering, software issue fixing, and multistep information-retrieval reasoning, respectively. The two frameworks, two models, and four benchmarks formed 16 configurations, with each configuration retaining 20 paired tasks, for a total of 320 paired observations. Before understanding these numbers, however, we need to look at the Baseline–Extraction–Gene, or B–E–G, protocol: B, baseline execution: Complete the task using native tools and built-in skills, without injecting any additional Gene. E, experience extraction: Distill strategies from valid baseline trajectories, with both successful and failed trajectories eligible to participate. The experience pools are isolated by evaluation batch, benchmark, framework, and model, and candidate assets are frozen before enhanced execution. G, enhanced execution: Restore the task environment, execute the original task in a new session, and inject the selected Gene. Experiences from the same task are used first, followed by other matching candidates from the isolated pool. The injection budget is 12,000 bytes. Figure 5: Enhanced execution prioritizes baseline experience from the same task. B and G retain the same foundation model and framework configuration and use the native scorer for the corresponding benchmark. Environment restoration and the new session reduce the direct influence of inheriting the previous execution environment, but the strategy may still contain experience from the same task. This experiment therefore measures the effect of redoing a task with historical experience; it does not evaluate generalization to previously unseen tasks or validate long-term online evolution. Sample selection has a second limitation. The study initially selected the first 20 tasks from each benchmark in their original order and replaced empty responses caused by anomalous output, timeouts, or execution failures with subsequent tasks. All configurations used the same set of 20 tasks. The final analysis retained valid results and complete token records while excluding executions interrupted by timeouts or token limits. The paper also explicitly states that the final task selection considered the observed improvement in results and token usage. It was a filtered diagnostic set rather than a random sample of the complete benchmarks, so its average gains cannot be treated directly as general expectations after deployment. 4.2 Success and Differences In this diagnostic sample, the success rate increased from 83.13% to 87.50%. Converting this to the 320 paired tasks, the baseline succeeded 266 times and enhanced execution succeeded 280 times, for a net gain of 14. Aggregated by runtime framework, Claude Code increased from 85.00% to 87.50%, while Codex increased from 81.25% to 87.50%. Figure 6: The success-rate gains vary by runtime framework, model, and task type. Terminal-Bench 2.1 had the largest aggregate change, rising from 64/80 to 71/80, an increase of 8.75 percentage points. SWE-rebench and GAIA increased by 5.00 and 3.75 percentage points, respectively, while SWE-bench Multilingual remained at 96.25% under both conditions. This shows that the effects of experience reuse depend on the task, model, and host interaction rather than adding a uniform capability increment to all executions. However, an unchanged success rate does not mean that experience has no value. If the task was already completed correctly, the new question is: is it still necessary to follow an equally long exploration path? 5. Execution Costs 5.1 Maintaining Success The paper analyzed changes in resource usage separately. Across all 320 paired tasks, average recorded task-execution tokens fell from 1,641.1K to 1,243.3K, a 24.24% decrease based on total usage. A more targeted comparison is the 265 task pairs that succeeded under both conditions. On this subset, average tokens fell from 1,278.2K to 1,053.6K, a decrease of 17.57%. This distinguishes “fewer tokens were spent because the task was not completed” from “the task was still completed correctly, but execution consumption decreased,” supporting the execution-efficiency value of experience reuse. Figure 7: Recorded execution tokens still decreased on tasks that succeeded in both rounds. If one experience is used for many subsequent tasks, its extraction and maintenance costs may be amortized; if it is used only once, the additional overhead may offset the execution savings. 5.2 Cases and Implications A token-counting task from the Terminal-Bench 2.1 dataset required processing a data subset with a specified tokenizer and outputting an integer result. Both rounds of execution with Claude Code and GPT succeeded, and after experience was injected, token consumption fell by 86.5%, while model calls fell from 112 to 17. Another task involved fixing a concurrent null-pointer exception with RxJava’s ObservableSwitchMap. Both rounds of execution with Codex and DeepSeek passed the benchmark evaluation, with token consumption falling by 58.2% and model calls falling from 86 to 52. Figure 8: Two successful cases reduced token usage and model calls after experience reuse. Returning to the original question, Evolver does not offer a promise of automatically becoming stronger. Instead, it provides a way to organize experience: extracting action knowledge from execution logs, using Gene to store strategies, using Capsule to preserve cases, separating sources, reviews, and eligibility for use from content, and then returning the experience to the existing Agent’s context through retrieval. It has demonstrated gains in success rate and execution efficiency in selected experiments on reusing experience from the same task. The next steps that will genuinely affect engineering judgments are whether these assets can transfer to unseen tasks, whether different integration paths can consistently enforce eligibility checks, and whether long-term reuse still yields net benefits after extraction and maintenance costs are included. Letting exploration leave behind strategies is the first step; ensuring that the strategies left behind remain worth using is the long-term work an experience system must accomplish. References Evolver: Turning Agent Exploration into Evolving Capabilities: https://arxiv.org/abs/2611.07595 EvoMap/evolver (GitHub): https://github.com/EvoMap/evolver

EvolverGEPEvoMapEvolutionSkill

Comments

…