EnvOpt: Making Environments More Suited to Intelligent Agents

When an intelligent agent handles enterprise tasks, it often spends considerable effort on preparatory work: finding APIs, browsing directories, parsing spreadsheets, and determining which script generated a particular table. Change the question, and all that preparation gets repeated. Models can certainly keep getting stronger, but if many tasks take place in the same workspace, we can ask another question: which explorations should the environment complete in advance? EnvOpt turns this question into an automated optimization process. It fixes the target model and execution framework, while another coding intelligent agent observes task trajectories, modifies how information is represented in the shared environment, and then tests the gains on subsequent tasks. What makes this paper noteworthy is that it advances information organization from manual preparation into a system design that can be measured, iterated, and reused across tasks. 1. Environment Representation 1.1 Three Variables First, let’s break down an intelligent agent system. The model handles understanding and decision-making; the harness, meaning the execution framework, organizes calls, context, and the flow of tool execution; the environment provides the information it needs to read, the objects it operates on, and entry points for getting close to those objects. All three parts jointly determine the cost of completing a task. Consider searching a stretch of Slack history. The same conversation can sit behind a remote API, requiring the intelligent agent to discover the API before querying, paginating, and reading it. Or it can be organized as searchable text that can be located directly through familiar commands. The information content is the same, but the path to finding and understanding it has changed. The paper uses I to represent the underlying information and E(I) to represent the way that information is organized and exposed to the intelligent agent. What EnvOpt searches over is E: preserving the information a task needs while adjusting its format, directory, index, helper programs, and access entry points so that the fixed target intelligent agent can complete tasks at a lower cost. Figure 1: The underlying information stays the same, while the environment representation changes how the intelligent agent accesses it The most useful distinction here is between “knowing how to do it” and “easily finding what you need.” The former often falls under model capabilities or execution strategies, while the latter can be implemented directly as a directory, a mapping from tables to source code, or an access program that returns compact records. EnvOpt establishes a separate optimization space for the latter. 1.2 Shared Conditions Why is it worth optimizing the environment specifically? Because it has reuse value. Many enterprise assistant questions depend on the same set of business systems, data engineering tasks revolve around the same repository, and multiple bug fixes may take place within the same codebase. Directory structures, relationships among tables, and API semantics are often more stable than any individual question. The paper calls this kind of environment a stationary environment. Here, “stationary” means that multiple tasks share the same or highly similar environment structure. The data for each task may change, and the reusable part may also be a transformation program: when encountering a new environment, it can regenerate directories, summaries, or access views according to rules it has already learned. This condition determines where EnvOpt applies. If every task faces a completely unfamiliar world with a different structure, offline preparation is difficult to amortize. If subsequent tasks continually use the same information foundation, optimizing the way it is accessed once can generate repeated gains. 2. Offline Evolution 2.1 Trajectory Feedback Once we know that the environment can be changed, the next step is deciding where to make changes. The manual approach usually starts with “adding an index” or “writing some documentation,” whereas EnvOpt starts with the target intelligent agent’s actual trajectories: which tools it called, what it read, where it repeatedly searched, and which outputs entered the context without helping advance the task. There are two roles in the process. Research agent R is responsible for modifying the environment, while target agent T is responsible for executing tasks. At each step, R receives the current environment, a set of training tasks, permission to run T, and a limited experimental budget. It first runs the target intelligent agent, reads the complete trajectory, edits the environment, and measures again. Effective changes are retained, ineffective ones are discarded, and this continues until the step budget is exhausted or the search ends. The research agent can build databases, transform files, add access scripts, or leave navigation instructions for the target intelligent agent. After each step, the external process retains only approved environment-layer artifacts; other changes to the target model and execution framework do not carry over into the next version. The next step uses a different, non-overlapping set of training tasks to check whether these modifications can serve new problems. Figure 2: The research agent modifies the environment, the target agent executes tasks, and the validation set determines whether the process continues This turns environment optimization into a measurable feedback loop. The research agent’s claim that “this directory should be more useful” must translate into something the target intelligent agent actually does less of, while task completion quality must be preserved. Cost measurements identify the gains, and trajectories explain where those gains come from. 2.2 Quality Gate This process also makes a notable choice: training steps do not provide the research agent with ground-truth answers or a hidden scorer. It can only judge whether a change is reasonable based on signals such as task requirements, execution trajectories, final actions, and answer consistency. This lowers the preparation threshold for training tasks. Because relatively few tasks are available in Workspace-Bench and Data-Eng-Bench, the authors synthesize similarly distributed problems for training without needing to write a complete scorer for every question at the same time. Ground-truth answers and scoring rules also reduce opportunities for the research agent to optimize directly against scoring details. Quality assessment still requires an independent responsibility. EnvOpt retains a validation set with verifiers and measures cost and pass rate on this set after each step. According to the algorithm description, if cost increases, or the pass rate drops by more than the tolerance compared with the previous step, evolution stops and the system rolls back to the preceding version. Final performance is then measured on a test set that did not participate in the search. Thus, the value of “training without ground-truth answers” is that more tasks can contribute clues about how to improve the environment. What actually determines whether a version of the environment survives is a separate set of tasks with quality signals. The paper also documents a case where the research agent used answer length and keyword matching to assess quality, which harmed performance on later tasks, prompting the authors to adjust the instructions. This failure illustrates exactly why the validation set needs to be separate from the training steps. 3. Access Layer 3.1 Preparing in Advance When the successful modifications from the paper are considered together, they all address one kind of repetitive labor: the target intelligent agent has to reacquaint itself with the environment at the start of every task. The most direct category is format preparation. In AutomationBench, the research agent wrote a program to read Gmail messages and organize the sender, subject, message ID, and message excerpt into text. In Workspace-Bench, some .xlsx files were converted into easier-to-read .txt files. This lets the target intelligent agent focus on content while reducing the reasoning cost spent shuttling back and forth between formats and interfaces. One step further is navigation. The file index in Workspace-Bench records paths, sizes, and types, allowing the intelligent agent to search the directory before opening files. The SQLite catalog in Data-Eng-Bench records table locations, script outputs, and table content types, turning relationships dispersed across many files into queryable data. A program in the Bleve codebase uses keyword matching against bug descriptions and outputs the eight most likely Go files, helping the intelligent agent identify a starting point for its investigation. The technologies behind these modifications are all familiar, but their responsibilities have changed: they express relatively stable structures in the environment in advance so that every task does not have to infer those structures again. Candidate-file ranking provides a starting point, while the original code remains available for the intelligent agent to continue investigating. The table catalog handles localization, while the target intelligent agent still performs the actual SQL logic. 3.2 Dynamic Views AutomationBench provides an even more concrete implementation point. Every task has a newly generated data world, so an optimized artifact cannot be merely a fixed summary. The research agent writes access-layer programs that read-only prefetch the current data through the underlying API at the start of the task, organize it into a compact view, and then hand it to the target intelligent agent. The Help Scout example in the appendix makes this intuitive. One environment contains 11 conversations. The original workflow requires discovering the API, listing the conversations, and then reading records containing multiple fields. After transformation, the intelligent agent starts with one line of representation for each conversation, along with the API needed to add an internal note. Here is a simplified rendering of one record: mig_hs1 | "Invoice correction needed" | status=active | tags=[billing] | mig_cust1 | customer: My last invoice was incorrect. When the same program encounters another task world, it reads the new records and generates a new view. What is reused is “how to organize the information,” while the actual business content needed by each question still comes from the current environment. Write operations, such as sending emails or creating records, are still initiated by the target intelligent agent while it executes the task. This shows that an environment representation can contain two layers at once: one layer consists of organization rules established in advance, and the other consists of an information view generated according to those rules at the start of the task. Ordinary programs handle deterministic retrieval and organization, while models handle task understanding and action selection. EnvOpt optimizes the repetitive work along this boundary of responsibility. 3.3 Behavioral Guidance After the information has been organized, where exactly does the cost go down? The paper’s token breakdown provides a more specific answer than total cost alone. In AutomationBench, input tokens per task fell from 171,641 to 41,466, a reduction of roughly 76%; output tokens fell from 5,065 to 3,881, a reduction of roughly 23%. The largest change came from tool returns for API discovery: approximately 123k dropped to 1.1k. Meanwhile, the initial prompt itself increased from roughly 12k to 31k—more information was delivered in advance, but it replaced much more expensive repeated discovery later on. Figure 3: Most of the savings come from environment feedback, so the initial context and subsequent exploration must be measured together Data-Eng-Bench shows a similar structure: input tokens decreased by roughly 72%, output by roughly 20%, and most of the savings came from the returns of data-inspection tools. Tool calls related to verification actually increased. These results support a more precise interpretation: once the target intelligent agent obtains useful information more quickly, it can shift its work toward concrete implementation and checking. However, the environment can also induce new exploration. The research agent once consolidated the possible values of every database column into one large file, but as a result, the intelligent agent saw more unexplored content and made more tool calls. Another search tool reduced search costs, but returned too many irrelevant files, leading the intelligent agent to spend more time reading them. The optimization objective therefore needs to cover the entire task trajectory. A single search may become cheaper, but subsequent reading may increase; the initial context may grow, while later API discovery may fall sharply. A good environment representation needs to answer two questions at once: how to make relevant information easy for the intelligent agent to obtain, and how to make it easy for the intelligent agent to decide where to focus its attention next. 4. Independent Gains 4.1 Task Outcomes Access-layer modifications may seem reasonable on paper, but they still need to be tested on unseen tasks. The paper covers five benchmarks: AutomationBench, Workspace-Bench, Data-Eng-Bench, and bug-fixing tasks in the Bleve and Tenacity codebases. The 600 public tasks in AutomationBench were divided into 200 test tasks, 96 validation tasks, and a remaining training-task pool, with each training step using 20 tasks. The authors ran the main results three times and reported the average pass rate and total test-set cost. For Haiku 4.5, the pass rate increased from 2.33% to 16.67%, a gain of 14.34 percentage points; total test-set cost fell from $42.36 to $11.55, a reduction of roughly 72.7%. These absolute pass rates also illustrate the difficulty of the tasks: environment optimization significantly improved this configuration, but a large number of tasks still remained incomplete. Figure 4: On AutomationBench, cost and pass rate when the same model uses different environments and harnesses When reading this figure, begin by following the points of the same color rather than immediately ranking all the models. The same color represents the same model; a point moving left means test-set cost has decreased, while a point moving upward means the pass rate has increased. EnvOpt’s results generally shift left, while the magnitude of the change in pass rate varies by model. The horizontal axis shows the total dollar cost of the test-task set and uses a logarithmic scale. 4.2 Tool Ablation At this point, one question remains: EnvOpt allows new tools to be added, so do the gains come mainly from modifying the tools? The authors conducted an ablation by putting the new tools from the final environment back into the original environment and comparing that with the full EnvOpt setup. In the Haiku 4.5 configuration on AutomationBench, optimizing only the tools yielded a 6.33% pass rate at a cost of $15.75; full environment optimization yielded 16.67% at $11.55. For Opus 5, both approaches had the same 35.17% pass rate, but full optimization reduced cost further, from $100.37 in the tool ablation to $75.26. Figure 5: Tool modifications and full environment optimization produce different gains, so cost and pass rate must be compared separately Data-Eng-Bench also provides a useful trade-off. For GPT-5.6-luna, the tool ablation had a 57.08% pass rate, higher than the 53.88% from full optimization; full optimization cost $7.51, substantially below the tool ablation’s $14.60. In several other configurations, merely adding new tools increased cost. This shows that access tools and information organization need to be designed together. A new tool placed in the original environment may elicit different calling behavior when it faces an organized catalog, database, and view. The evidence presented in the paper supports the idea that “there is additional optimization space at the environment layer.” For a specific configuration, the balance between cost and pass rate should be chosen according to quality requirements. 4.3 Transfer and Stacking Another way to test this is to give an environment optimized for one target intelligent agent to other models or execution frameworks. If the gains persist, at least part of the modification addresses a shared information bottleneck. In AutomationBench, an environment optimized for Haiku 4.5 was transferred to Opus 5. Test-set cost fell from $236.18 to $75.26, a reduction of roughly 68.1%, while the pass rate rose from 33.83% to 35.17%. In Workspace-Bench, environments optimized with Codex and GPT-5.4-mini as targets could also serve Claude Code and Opus 5; for the latter, cost decreased by roughly 55.9%, while the pass rate slipped slightly from 62.27% to 61.14%. These two types of workspaces also differ in evaluation scale. Workspace-Bench uses 80 test tasks and 20 validation tasks, with Sonnet 5 serving as the judge. Data-Eng-Bench uses 73 test tasks and 19 validation tasks. The costs in the tables are totals for their respective test sets, and transfer effects should be compared before and after within the same benchmark and configuration. Figure 6: Optimized environments can transfer across models and execution frameworks, although the specific pass rate still varies by configuration Looking further at the stacking experiment, on AutomationBench, Haiku 4.5 with Meta-Harness achieved a 7.00% pass rate at a cost of $75.33. Adding EnvOpt to that optimized framework yielded 18.17% and $12.47, cutting cost by another roughly 83.4% relative to Meta-Harness alone and increasing the pass rate by 11.17 percentage points. This result provides a strong answer to “is environment optimization still necessary?” Even when the execution framework has already been automatically searched, the information-access layer can still provide new gains. EnvOpt alone cost $11.55 with a 16.67% pass rate, while the stacked version traded slightly higher cost for a higher pass rate. The two layers of optimization can be combined, and each retains its own trade-offs. The paper describes environment optimization as a third direction orthogonal to model and harness optimization. In engineering terms, it is better to understand “orthogonal” here as a separation of responsibilities that can be optimized independently and stacked. Tool instructions, startup context, and access interfaces may already straddle the boundary between the environment and the framework. For this set of comparisons, the paper states that the task pools, research model, and target model were aligned, while the full search budgets of the methods still need to be compared separately. 4.4 Codebases The software engineering experiments apply the same idea to code navigation and verification workflows. In Bleve, EnvOpt completed tasks in 74 seconds, compared with 130 seconds for GSkill; their pass rates were 97.7% and 100.0%, respectively. Here, the gain is retaining a near-perfect pass rate while reducing time by roughly 56.9%. In Tenacity, EnvOpt reduced the average number of steps to 12.4, compared with 14.4 for Advisor Models; the pass rate increased from 61.2% to 70.1%, a gain of 8.9 percentage points. This environment also added a helper testing program responsible for locating relevant tests, running them, and returning the results, reducing the number of steps the target intelligent agent had to spend organizing those actions. Figure 7: The codebase experiments measure completion time and execution steps separately while preserving the corresponding quality trade-offs These two results use different efficiency metrics: one uses time, and the other uses steps. Viewed mechanistically, their more valuable commonality is that the environment gives the intelligent agent a better starting point and a more direct entry point for verification, allowing subsequent reasoning to focus on the specific bug. 5. Engineering Boundaries 5.1 Environment Correctness If the environment starts taking on more preparatory work, it also becomes a software component that needs to be validated. The paper documents a typical issue: in Data-Eng-Bench, the research agent wrote a program that split SQL by semicolons in its first step, but it did not correctly handle semicolons inside quotation marks. This was not discovered until the fifth step, and it continued to increase costs over the intervening four steps. To the target intelligent agent, the error might appear as “my query failed again,” so it keeps spending tokens rewriting the query. The actual fault lies in the environment transformation layer. The more complex preparatory work is moved into the environment, the more important it becomes to ensure transformation errors can be identified and the original information is retained as a basis for tracing them back. Another failure occurred in Bleve: the research agent added checks that detected specific code snippets for some training tasks, but later rejected correct fixes as well. This reveals a risk of environment optimization: codifying the experience of one batch of tasks as a universal constraint may disrupt the recovery paths of other tasks. Therefore, when implementing this in my own system, I would prioritize three capabilities: catalogs and summaries must be traceable to their original materials; preprocessing and helper tools must have independent tests; and new environments must be versioned so that the previous version can be restored when quality rolls back. If data continues to change, indexes and views also need explicit refresh rules; when enterprise materials are involved, the visibility scope of reorganized data must continue to respect the original permissions. The latter two are engineering tasks that must be added when moving from a shared environment to real deployment. 5.2 Cost Amortization Finally, offline investment must be brought back into the ledger. The paper reports that the environment search on AutomationBench cost approximately $150.10 and 6.6 hours in total, including the cost and time of launching the target intelligent agent during the search. Differences across environments were substantial: Workspace-Bench cost $339.44 and 15.4 hours, while Data-Eng-Bench cost $36.73 and 25.4 hours. Figure 8: The fixed cost of one environment optimization can be amortized through sustained savings on subsequent tasks The left side of the original figure uses Opus 4.8 as an example: the cost per task fell from approximately $1.73 to $0.39, and after adding approximately $150 in offline investment, the cumulative cost curve in the figure crosses at roughly 109 tasks. This break-even volume comes from the sampling curve for that configuration, while the right side summarizes the process of incremental modification and validation-based stopping. For a system preparing to deploy, the most direct calculation is whether the number of future tasks multiplied by the expected savings per task covers the investment in offline search, environment construction, and ongoing maintenance. The more stable the shared structure and the more subsequent tasks there are, the more likely this approach is to hold up. At the same time, prefetching APIs, running transformation scripts, and maintaining SQLite indexes still consume ordinary computing resources and external services. The paper primarily presents agent token pricing costs, along with changes in time or steps for software engineering tasks. During deployment, these added resource and update costs must be included in the total, especially in environments where data changes frequently or API calls themselves are expensive. 6. Summary Returning to the opening question: which explorations should the environment complete in advance? EnvOpt’s answer is preparatory work that repeats across tasks and can be extracted from shared information structures. Catalogs, format conversions, mappings between tables and source code, and interface navigation can form a reusable access layer, while models continue to handle the understanding, decisions, and execution of the current task. The paper’s core contribution is to establish this division of labor as an automated search and validation loop. Trajectories identify repetitive labor, environment modifications transfer it to more direct representations, and validation tasks check cost and quality. Transfer experiments across models and stacking experiments with Meta-Harness further demonstrate that information organization is worth treating as an independent design object. For intelligent agent system designers, the most important takeaway is that part of the cost of a task comes from structures the model has to reconstruct on the fly when facing the environment. In addition to strengthening models and adjusting execution frameworks, we can explicitly hand these structures over to the model. The next question is how to keep this access layer effective as data, permissions, and task distributions change—and that will determine how long a successful offline optimization remains reusable in a real system. References Mingxuan Li et al., EnvOpt: Automated Optimization for Agent Environments (arXiv:2611.07372v1, 2026) https://arxiv.org/abs/2611.07372

Harness Env Evolution EnvOpt