Parallel Agent Teams: Small-Team Collaboration, Multi-Team Exploration

Optimizing a GPU kernel or completing a set of formal proofs often requires repeated trial and error: try a different implementation, run an evaluation, and keep iterating in whichever direction proves effective. When giving a model more test-time compute, the most direct approach is to let more agents try simultaneously; if the task only needs one best result, the value of the other attempts lies in the routes explored and the discoveries left behind. Junlin Wang and colleagues’ “Test-Time Compute Scales Better with Parallel Agent Teams” breaks this problem into two levels: who should handle each individual attempt, and how multiple attempts should be organized. The paper’s proposed solution is to share discoveries within teams, explore independently across teams, and then have a verifier select the best result. We’ll first look at why these two levels are worth separating, then examine how this changes computational returns under a fixed budget. 1. Sampling Unit 1.1 Independent Attempts Let’s start with the simplest approach. Have one agent modify code in its own environment, submit an evaluation, and preserve its best valid version; then replicate this process N times and select the highest-scoring result. This is best-of-N applied to agents. Each sample is already a workflow containing multiple tool calls and rounds of iteration, so sampling occurs between these complete attempts. This workflow has two important prerequisites: the task must be able to score candidate results, and the final goal must be to select one high-scoring result. A GPU kernel can first be checked for correctness and then compared by latency; formal proofs can be validated by a proof checker for valid progress. A verifier allows multiple exploration paths to compete on the same scale, giving additional samples a chance to translate into a better final result. The benefit of independent sampling is straightforward: with one more attempt, the existing best result is still retained. The difficulty is that every agent has to complete its own exploration from its own starting point. If some implementation detail causes one agent to get stuck, it may repeatedly cause other agents to get stuck as well; successful local discoveries, meanwhile, remain in their respective contexts. In the paper’s main experiment, 50 independent agents already captured 81%–100% of the gains brought by 100 agents, with the marginal return from adding attempts rapidly diminishing. 1.2 Shared Discoveries If the bottleneck is the quality of an individual attempt, a natural change is to let several agents read one another’s discoveries. The paper primarily uses a flat team structure: each member maintains an independent sandbox and context, but they post to and browse the same forum, listing titles and reading full posts as needed. A forum post can contain a complete solution and automatically include the poster’s current best score; the initial prompt encourages everyone to share clear successes and failures, specific changes, and measured scores. What is shared here is experimental information that can be used further. If one member discovers a promising region of the parameter space, other members can improve on it in different directions; if one member confirms that a particular approach has failed, others can avoid duplicating the attempt. The paper does not preassign roles such as researcher or reviewer, nor does it have a manager assigning work. Members decide for themselves when to read and write, and the team’s score is the highest valid score submitted by any member. Figure 1: Members with independent contexts exchange reusable discoveries through a shared forum At this point, the boundary of an individual attempt changes: originally, one sample was one agent’s workflow; now, one sample is a small team’s workflow. To determine whether communication actually helps, we should compare the best result from a k-person team with the best result from k independent agents while keeping the number of people and the time the same. On the paper’s graph node classification task, a ten-person team scored an average of 79.0, while the expected best score of ten independent agents was 69.1; eight of the twelve tasks benefited from this comparison. Sharing discoveries can indeed improve the quality of a single team attempt on some tasks. 2. Exploration Boundaries 2.1 Separate Teams If sharing is useful, should every agent then join one forum? That would change something else: the entire budget would amount to only one team attempt. Good discoveries would spread faster, members would be more likely to continue searching in the same direction, and previously independent paths would gradually merge. The paper’s communication analysis reveals this two-sided nature. In one groundwater contamination source inversion team, after one member posted a new best configuration, six of the other eight members resubmitted that configuration within ten minutes, and eight of the final nine members converged on the same score. Reusing results quickly raises the team’s common starting point, but it also pulls subsequent exploration toward the same region. To preserve both information diffusion and path diversity, we need boundaries on diffusion. Parallel Agent Teams divides the total budget B into multiple small teams, with k members per team and a sample count of N = B/k. Teams share a forum internally but operate independently from one another, and the best result from each team is selected again at the end. With 100 agents, for example, there are three organizational options: 100 independent agents: each attempt is inexpensive, allowing for 100 samples. One 100-person team: all members collaborate, producing only one team attempt. Ten ten-person teams: each attempt includes team collaboration while preserving ten independent team paths. Independent agents and a single large team are the two endpoints of the same budget-allocation problem; parallel small teams allow “how strong each attempt is” and “how many attempts there are” to be adjusted together. The original paper’s overview diagram compares all three approaches within the same workflow: the left side varies the sampling unit, while the right side examines the results as the budget expands. Figure 2: Parallel small teams preserve both team collaboration and selection across teams 2.2 Mean and Variance At this point, we still need to explain what may look like an unfavorable trade-off: a ten-person team has only one tenth as many samples as independent agents, so why can it still perform better? We can view each organizational approach as a score distribution. The mean μ describes the level a single attempt typically achieves, while the standard deviation σ describes the variation across attempts; the budget determines how many samples can be drawn from that distribution. Assuming independent samples and an approximately normal score distribution, the paper derives the following relationship. Here, a_N is the expected maximum of N standard normal samples: V_k(B) = μ_k + σ_k × a_(B/k) The first term is the average score of a single k-person team attempt, and the second is the gain obtained by selecting the best among multiple independent attempts. When N = 1, a_1 = 0, so the variance of a large team cannot produce a return through selection across teams. Increasing the number of teams gives the high-score tail more opportunities to be sampled. For large N, a_N grows approximately as √(2 ln N), so the returns from sampling gradually slow. This growth pattern explains why reducing the number of samples tenfold may still be worthwhile. In a normal model with the same standard deviation, if the budget is 100 and each team costs 10, a team mean only about 0.97σ higher is enough to offset the loss from reducing the sample count from 100 to 10. The team trades fewer samples for a higher starting point per attempt; if that improvement is large enough, the final selected result will be better. Figure 3: Teams raise the mean of individual attempts, while cross-team sampling continues to exploit high-score variance The left side of the figure shows the score distributions for a single agent and a ten-person team, while the right side compares the normal-fit predictions with the empirically measured selection curves. The paper used this model on twelve tasks to predict the direction of the gains under a 100-agent budget, and nine predictions were correct. When failed attempts form a long tail and scores cluster near the upper limit, the mean and standard deviation are insufficient to describe the high-score tail that actually determines the benefit of selection. This model is useful for explaining trade-offs, but actual configurations still need to be selected using each task’s real distribution. 3. Budget Returns 3.1 Long-Horizon Tasks The main experiments used Qwen3.8-27B, with agents running in the Codex harness and independent sandboxes. The twelve tasks came from six benchmarks and covered areas including GPU kernel optimization, formal proof, machine learning, geophysics, and structural design. Each agent generally worked for twelve hours, while the Browser Exploit task lasted six hours. Most tasks allowed agents to submit evaluations during the run and preserve their best results; the two MLS-Bench tasks scored the final files after the deadline, with agents performing their own local checks during the run. For each task, the authors primarily collected about 100 independent agent runs and 10–11 ten-person team runs, then calculated the expected maximum when selecting a given number of samples from the pool. Different budget points on the curves come from subset statistics of the same set of runs, so the overall shape and differences between tasks are what matter when reading the graphs. Figure 4: Under a 100-agent budget, parallel ten-person teams score higher on ten of twelve tasks Among the ten tasks where ten ten-person teams beat independent agents under a 100-agent budget, the most dramatic change occurred in Carleson formal proofs: the highest score rose from 19.0 to 54.8, with the score corresponding to the percentage of proofs completed. In this type of task, individual attempts tend to stall at a low level, while team discoveries help members enter new solvable regions; sampling across multiple teams then further expands the opportunity to find more complete results. KDA Chunk Forward also improved from 11.83 to 16.54, a gain of roughly 40%. Two other tasks illustrate the other side of the trade-off. In Active Learning, independent agents performed slightly better; in Kimi-Linear Decode, independent agents scored 2.36 compared with 2.16 for teams, a lead of about 8%. In the latter task, many attempts failed to produce a valid kernel for a long time, leaving little effective progress for teams to exchange; more independent attempts were therefore more valuable. Whether adding members improves the quality of a single attempt depends on whether they have anything to share and whether sharing opens up room for further improvement. 3.2 Range of Growth When aggregating across tasks, the paper first normalizes by the expected best score of ten independent agents and then averages across tasks. Compared with independent agents, parallel small teams are about 10% better with a total budget of ten agents and about 27% better with a budget of 100 agents. This comparison uses normalized results across different tasks while retaining each task’s original scoring scale. Figure 5: As the budget grows, the average advantage of parallel small teams increases while marginal returns decline The curve on the right deserves a closer look: the additional computational returns of both methods decline, but the small-team approach spends more time in a more productive growth range. When the total budget increases from 10 to 20, independent agents go from ten samples to twenty, while the team approach goes from one team to two and gains the benefit of selecting across teams for the first time. In this budget range, the paper measured an average improvement of 1.27% per additional agent for the team approach, compared with 0.64% for independent agents. This adds a tunable dimension to test-time scaling: once the curve for independent sampling has flattened, compute can be invested in improving the sampling unit and then sampling better units in parallel. Small teams are one approach validated by the paper; their benefits come both from communication among members and from the differences that remain between teams. 4. Team Design 4.1 Choosing Team Size With a fixed budget, additional members and additional teams compete for the same compute. On four tasks, the paper organized 100 agents as independent agents, five-person teams, ten-person teams, fifty-person teams, and a hundred-person team. The best configuration always fell between the two endpoints: ten-person teams performed best on three tasks, while graph node classification reached its highest score of 100 with two fifty-person teams. Figure 6: Under a fixed 100-agent budget, the best team size lies between independent agents and one large team Compared with a 100-person team, ten ten-person teams scored 8%–30% higher across these four tasks. When the large team grew from 64 to 100 members, performance declined on all four tasks. The large team had only 1–5 runs at each size, however, and the forums for 64 and 100 members showed only the latest 100 titles and unread posts, while small teams could see the complete index. This result treats “more people entering the same collaboration domain” as an option that must be evaluated carefully and makes communication-interface visibility part of the scale experiment. Ten people was the paper’s main experimental configuration. In scale sweeps with smaller budgets, some tasks worked better with three to five people, while the optimal size for some tasks increased with the budget. What should really be tuned is the size of the collaboration domain: it must be large enough to turn local discoveries into stronger results while leaving enough budget to form multiple team paths. 4.2 Information Paths Beyond size, how information flows also changes the quality of an individual attempt. The paper created a centrally orchestrated comparison: one orchestrator assigned tasks to nine workers and collected their reports; workers could not communicate directly and had to wait for assignments when they had no task. This design replaced direct peer-to-peer reuse with forwarding through a central node. Figure 7: A centrally orchestrated team trails a flat team with direct communication on multiple tasks On groundwater contamination source inversion, graph node classification, and KDA Chunk Forward, this orchestrated approach performed below both the flat team and independent agents at every measured budget point. On the groundwater task, the selected best of ten orchestrated teams scored 68.4, still below the flat team’s average of 76.5. Waiting for assignments and single-node forwarding make it harder to immediately turn effective discoveries into another member’s next attempt. This comparison also changes the number of people effectively solving the problem: the orchestrator’s workspace is not scored, whereas all ten members of the flat team can submit. The result therefore reflects the gains and losses of the entire organizational approach, including scheduling overhead, communication paths, and the number of members who can submit. For optimization tasks run in parallel on the same problem, directly sharing actionable discoveries already provides a lightweight coordination path; introducing a central role requires first explaining what additional constraint it resolves. In the forum implementation, this lightweight path consists of only three actions: post a discovery, scan titles, and read a post. Members are encouraged to check for similar attempts before starting a new experiment and to clearly document the exact changes and scores. The effective medium of collaboration is information that can change a peer’s next action; failed experiments are just as important as successful solutions. 5. Practical Costs 5.1 Pricing Basis Using the same model, the same number of people, and the same amount of time is suitable for isolating the effect of organizational structure; choosing an actual system also requires comparing bills. The paper allocated the GPU time for self-hosted Qwen3.8-27B inference and compared it with pricing based on GPT-5.6-sol’s recorded token usage. On the eight tasks with at least five GPT-5.6-sol runs, using the cost of five GPT-5.6-sol agents as the budget, parallel small teams won on four tasks. Figure 8: Under the paper’s inference-cost basis, small-model teams outperform stronger models on some tasks Formal proofs provide a concrete example: a single GPT-5.6-sol agent performed better than one ten-person Qwen3.8-27B team; once the budget increased to the cost of five GPT-5.6-sol agents, it was possible to run eight Qwen3.8-27B teams, which scored 52.0 compared with 47.9 for the control. The former had stronger per-attempt capability, while the latter bought more team attempts with room for exploration at that budget. This comparison uses a specific inference-cost calculation: Qwen is priced according to its share of an H200 service, GPT is priced by token, and unreported cache-write costs are estimated using an upper bound. If Qwen is also priced by token, the paper reports that the number of wins rises to six. Meanwhile, Qwen team members use 1.0–2.2 times as many tokens as independent agents on a task. Production evaluation also needs to include candidate execution, verifiers, infrastructure concurrency, and deadlines in the actual budget; this is especially important for kernel tasks, where each agent occupies dedicated H100 evaluation resources. 5.2 Model Conditions A stronger model changes the score distribution of team attempts. The paper repeated the comparison with DeepSeek V4.1 Flash on eight tasks. On easier tasks, a single agent could already achieve a very high score, multiple independent agents quickly approached the ceiling, and the room for collaboration correspondingly shrank; on tasks that remained difficult, teams were more likely to provide valuable improvements to the mean. Figure 9: Stronger models approach the ceiling quickly on easier tasks, and the value of small teams varies by task Groundwater inversion and KDA Chunk Forward still show clear advantages for teams in the figure, while the differences in graph node classification, formal proofs, and structural design are smaller. This supplementary experiment has two important conditions: the MoE independent agents were cut off early by the harness’s turn limit after about eight hours, while teams ran for twelve hours; in KDA, one team’s score of 31.8 was clearly higher than the other runs, pulling up the high-budget end of the team curve. Time budgets should be handled in the same way. In the supplementary analysis of graph node classification, the same set of independent agents continued from the sixth hour to the twelfth, yet their average score rose only from 52.0 to 52.7; two agents working for six hours each made better use of the same twelve agent-hours. The two-person team’s average of 57.4 was close to the 57.6 achieved by selecting the best of two independent six-hour agents, so at this size the main benefit came from the second attempt. The six-hour results were themselves extracted from runs originally scheduled for twelve hours, and agents genuinely planned with a six-hour deadline may adopt a different pace. Putting these conditions back into the design problem makes the selection process clearer: first measure the score distributions for a single run, independent selection under the same budget, and small teams; confirm whether collaboration improves an individual attempt; then decide how many team paths to preserve. If the final result lacks a reliable score, identifying the best team itself becomes a new system responsibility and requires additional work on the evaluation problem. 6. Conclusion The most valuable insight from Parallel Agent Teams is that it separates collaboration from sampling into two levels: sharing discoveries within a team raises the quality of an individual attempt, while preserving differences across teams lets selection extract the value of exploration. One shared domain deepens local progress, while multiple shared domains ensure that the budget still covers different directions. Test-time scaling can therefore adjust the model, session duration, team size, and sample count simultaneously. On verifiable long-horizon tasks, the paper shows that groups of small teams can provide better budget returns in many scenarios; the two losses, the variation across models, and the small-sample scale experiments establish the task conditions that must be kept in mind when selecting a configuration. The genuinely useful next question is: how can the information shared by each team be made more actionable while still allowing different teams to keep exploring differences worth preserving? Together, these two factors determine how much progress the next block of compute can buy. References • Test-Time Compute Scales Better with Parallel Agent Teams: https://arxiv.org/abs/2611.09798

Agent PAT 测试时计算多智能体 Benchmark