Introduction

July 2026

AgentStream: How Well Do Self-Evolving LLM Agents Perform Under Streaming Tasks?

Dong Yan^1,2,3,⋆ Jian Liang^1,3,‡ Dapeng Hu^2,‡ Ran He^1,3
Nicholas Jing Yuan² Qi Zhang² Tieniu Tan^1,3,4

¹ School of Artificial Intelligence, University of Chinese Academy of Sciences ² Microsoft
³ Institute of Automation, Chinese Academy of Sciences ⁴ Nanjing University

^†^†footnotetext: ^⋆ Work done during an internship at Microsoft.^‡ Corresponding authors: liangjian92@gmail.com, dapenghu@microsoft.com.

Abstract

Large language model (LLM) agents can self-evolve by continually improving from their own accumulated experience. However, existing studies predominantly adopt independent evaluation. Consequently, the behavior of self-evolving agents in realistic streaming settings, where agents adapt to diverse and complex task streams, remains poorly understood. To address this gap, we introduce AgentStream, a unified framework that evaluates self-evolving agents spanning diverse evolution components by organizing agentic benchmarks into a configurable task stream and instantiating the Isolated, Sequential, and Interleaved streaming scenarios at test time, which progressively vary the scope and domain composition of the stream. Over these scenarios, we combinatorially evaluate five representative self-evolving methods across three frontier foundation models, disentangling how model capability, method architecture, and streaming scenario jointly shape self-evolution. Our results show that self-evolution reliability varies across streaming scenarios, the benefit of self-evolution is gated by model capability and non-monotonic in model strength, and no single method dominates across models and scenarios. These findings offer concrete guidance for selecting self-evolving methods across models and streaming scenarios. Overall, we advocate that self-evolving agents should be evaluated under realistic task streams rather than isolated single-task settings.

Introduction

Large language model (LLM) agents are shifting from static, deploy-once systems toward adaptive architectures that continuously learn from their own accumulated experience during deployment, a paradigm broadly termed self-evolving [1, 2, 3]. Depending on which component of the agent is updated, recent methods evolve the prompt context, structured memory, reusable skill libraries, or an integrated harness [4, 5, 6, 7, 8, 9, 10, 11]. Such self-evolution is expected to produce increasingly capable agents over time, making it essential to understand how agent capabilities evolve across the diverse task streams encountered in realistic deployment.

However, as illustrated in Figure 1, existing agentic benchmarks and self-evolving studies predominantly adopt independent evaluation, where each task is solved in isolation and performance is aggregated without any cross-task state [12, 13, 14, 15, 16, 4, 6]. While a few studies move toward streaming evaluation [17, 18, 9], they stream each benchmark independently under a single evolution component, leaving how different evolution components transfer cross-domain experience systematically unexamined. Consequently, it remains unclear whether the improvements reported for self-evolving agents persist once they are deployed in realistic streaming settings, where tasks may span diverse domains and arrive without clear task boundaries or supervision. Answering this question requires an evaluation that considers the foundation model, the self-evolving method, and the structure of the task stream jointly rather than any one of them in isolation, which the prevailing independent evaluation is inherently unable to provide.

To this end, we propose AgentStream, a unified streaming evaluation framework that organizes tasks from multiple benchmarks into a configurable stream, ranging from within-domain to cross-domain composition, and evaluates self-evolving methods whose evolution components span context, memory, skill, and integrated harness. AgentStream performs a combinatorial evaluation across models, self-evolving methods, and streaming scenarios, enabling us to decouple the contributions of model capability and method architecture under different stream structures. Concretely, we instantiate three test-time streaming scenarios that progressively vary the scope and domain composition of the stream: Isolated, where each benchmark evolves in its own stream; Sequential, where the agent processes the benchmarks in a fixed order, transferring its evolution state across benchmarks; and Interleaved, where tasks from all benchmarks are shuffled into one unified stream. Across these scenarios, we evaluate five representative self-evolving methods, ACE [4], A-Mem [6], ReasoningBank [9], AutoSkill [8], and Harness [5, 11], across three frontier foundation models, GPT-5.4 [19], Gemini 3.1 Pro [20], and Claude Opus 4.7 [21], over six agentic benchmarks covering diverse capabilities, including AppWorld [12], BFCL [13], BrowseComp-Plus [22], HLE [15], SWE-bench-Verified [14], and Tau2 [16].

Our study reveals three findings. First, self-evolution is not uniformly beneficial, and its reliability varies with the streaming scenario. Isolated is the most reliable, while Interleaved generally outperforms Sequential despite its more heavily mixed stream. Second, not all models benefit from self-evolution, as its gain is gated by model capability. The weakest model exhibits negative evolution gains, and the benefit is non-monotonic in model strength, with a mid-capability model gaining more than a stronger one. Third, no single method dominates. Context-integrated methods favor Isolated while retrieval-based methods favor Interleaved, and the optimal method varies across models rather than transferring between them.

Overall, this work contributes AgentStream, the first framework that unifies agentic benchmarks into a configurable streaming evaluation and systematically assesses self-evolution along the streaming scenario, method, and model dimensions. Through a combinatorial analysis over these dimensions, we investigate how streaming scenario, model capability, and method architecture shape whether self-evolution improves or degrades performance. We further distill these observations into actionable guidance: applying self-evolution to sufficiently capable models, favoring context-integrated methods under within-domain streams and retrieval-based methods under cross-domain streams, and selecting the method per model rather than assuming a universal choice. We hope AgentStream encourages future research to evaluate self-evolving agents beyond isolated single-task settings and toward realistic task streams.

Refer to caption — Figure 1: Independent evaluation vs. our streaming evaluation framework. (a) The prevailing paradigm solves each task in isolation without cross-instance experience accumulation. (b) AgentStream: a stateful agent evolves through self-generated feedback over a configurable task stream, with foundation model, self-evolving method, and stream composition as modular design dimensions.

Related Work

Learning with Streaming Data

Learning from streaming data mainly involves test-time learning and continual learning. Test-time learning adapts a model to each incoming instance or distribution shift on the fly [23, 24], whereas continual learning targets sequential task streams while resisting catastrophic forgetting [25, 26]. With the advent of LLMs and agents, test-time learning proceeds at two levels [27, 28]. Parameter adaptation methods operate within individual test instances, either by directly updating model weights through data selection and in-place learning [29, 30, 31, 32, 33], or by applying reinforcement learning to iteratively refine model behavior [34, 35, 36, 37, 38, 39, 40]. Cross-instance accumulation methods instead build reusable experience that transfers across test tasks, including evolving libraries [41, 18], temporary skills [42, 43], cached plan templates [44], and consolidated memory systems [45, 46]. In parallel, continual learning for LLM agents has been explored through gradient-free inference-time updates [47, 48], parameter-level methods that mitigate forgetting [49, 50], and experience-centric approaches that reuse accumulated knowledge across tasks [51, 52, 53, 54].

Self-Evolving Agents

Self-evolving agents can autonomously improve their capabilities by learning from their own experience [1, 2]. Existing approaches primarily differ in the component of the agent system that undergoes evolution. Memory-based methods focus on the accumulation, retrieval, and reorganization of historical experience, enabling agents to selectively retain useful knowledge while pruning irrelevant information over successive tasks [7, 55, 6, 56, 4]. Skill-based methods operate at a higher level of abstraction, discovering and composing reusable capability modules into growing libraries that compound across diverse task domains [57, 8, 58, 59, 10, 60, 61]. Harness-level methods evolve the scaffolding infrastructure surrounding the base model, including system prompts, tool configurations, memory modules, and skill libraries [62, 63, 5, 11, 64, 65, 66, 67]. Beyond individual agents, architecture-level methods evolve the multi-agent topology itself, dynamically reconfiguring agent roles, communication structures, and coordination protocols [68, 69, 70]. Mechanistically, these methods are driven by reinforcement learning [7, 57, 64], evolutionary algorithms [71, 72, 65], trajectory distillation [8, 73, 74], or gradient-analogy optimization that treats textual feedback as differentiable signals [75, 69, 61]. However, existing work evaluates on isolated benchmarks without systematically comparing how different self-evolving methods and models behave under a controlled setting, while recent analyses further reveal that self-evolution can degrade or fail to transfer across domains [76, 1, 3]. Our work provides a unified streaming evaluation framework, analyzing roles of self-evolving methods and models under different stream structures.

Agentic Benchmarks

Agentic benchmarks have been developed to evaluate LLM agents across diverse and complex environments. Interactive web and application benchmarks require agents to complete long-horizon tasks by navigating stateful interfaces and executing actions that alter the environment state [77, 78, 79, 12, 80]. Tool-use benchmarks assess structured function calling, measuring whether agents can select, compose, and invoke external APIs with correct arguments [13, 81, 16, 82]. Software engineering benchmarks evaluate agents on repository-level tasks such as resolving real-world code issues under executable test suites [14, 83, 84]. Skill-oriented benchmarks measure how well agents acquire and reuse modular capabilities across heterogeneous tasks [85, 86]. Knowledge-intensive reasoning benchmarks stress deep retrieval and expert-level problem solving over frontier knowledge [15, 87, 88]. Economically grounded benchmarks further evaluate agents on real-world professional tasks of high practical value [89, 90]. However, these benchmarks predominantly evaluate agents on each task in isolation, and although a few adopt streaming evaluation [17, 18, 91], they remain limited to single-benchmark streams and evaluate only a single evolving component. In contrast, our framework unifies agentic benchmarks into more complex streaming scenarios covering both within-domain and cross-domain adaptation, and evaluates diverse evolving components including prompt, memory, skill, and harness.

The AgentStream Framework

To study how LLM-based agents improve through experience accumulation over realistic task streams, we design AgentStream, an evaluation framework centered on self-evolution, in which an agent processes a stream of tasks and, after each attempt, distills the interaction trajectory into persistent experience (e.g., refined context, memory entries, reusable skills, or revised harness). Building on this framework, we instantiate the problem setup and three streaming scenarios that systematically vary the scope and composition of the task stream, as illustrated in Figure 2.

Problem Setup

We consider a test-time learning setting [9, 17] in which a task stream $\mathcal{Q}=\{q_{1},q_{2},\ldots,q_{N}\}$ is presented to the agent, with each task revealed only after the preceding one is completed. The agent is parameterized by a foundation model $\mathcal{M}$ and equipped with a self-evolving method that maintains an evolution state $S$ . This state aggregates the experience distilled from the first $t$ interactions, initialized as $S_{0}=\varnothing$ . For each task $q_{t}$ , the agent performs a multi-step interaction conditioned on its current evolution state, generating an execution history $h_{t}=\{(a_{t,i},o_{t,i})\}_{i=1}^{L_{t}}$ of action-observation pairs over $L_{t}$ steps and a final solution $y_{t}=\mathcal{M}(q_{t},h_{t}\mid S_{t-1})$ . The agent subsequently updates $S_{t}=\textsc{Evolve}(S_{t-1},\,\tau_{t})$ by reflecting on the interaction trajectory $\tau_{t}=(q_{t},h_{t},y_{t},r_{t})$ , where $r_{t}$ denotes the self-generated feedback. Notably, no ground-truth labels are accessible at test time, and the evolution relies entirely on feedback intrinsic to the interaction, such as execution outcomes and reflective self-evaluation. We quantify the benefit of self-evolution through the evolution gain:

\Delta=\mathrm{Perf}(\mathcal{M},\,\mathcal{Q},\,S)\;-\;\mathrm{Perf}(\mathcal{M},\,\mathcal{Q},\,\varnothing),

(1)

where $\mathrm{Perf}(\mathcal{M},\mathcal{Q},\varnothing)$ denotes the baseline in which the same model solves each task with $S_{t}=\varnothing$ for all $t$ . A positive $\Delta$ indicates net improvement from self-evolution, while a negative $\Delta$ signals that accumulated experience introduces interference.

Streaming Scenarios

Given a set of $K$ benchmarks $\{\mathcal{B}_{1},\ldots,\mathcal{B}_{K}\}$ , each contributing a task subset $\mathcal{Q}^{(k)}$ , we instantiate three evaluation settings that systematically vary how the task stream $\mathcal{Q}$ is composed and how the evolution state $S_{t}$ is scoped across benchmarks.

Isolated.

Each benchmark $\mathcal{B}_{k}$ is assigned an independent agent instance with its own evolution state $S_{t}^{(k)}$ . The task stream for each agent instance is simply $\mathcal{Q}^{(k)}$ , and no experience transfers across benchmarks. This setting isolates intra-domain evolution and measures how effectively a self-evolving method accumulates useful experience within a single task domain.

Sequential.

A single agent processes all benchmarks in a fixed order $\mathcal{B}_{1}\rightarrow\mathcal{B}_{2}\rightarrow\cdots\rightarrow\mathcal{B}_{K}$ , with its evolution state $S_{t}$ retained across benchmark boundaries. The resulting task stream is the concatenation $\mathcal{Q}=\mathcal{Q}^{(1)}\oplus\mathcal{Q}^{(2)}\oplus\cdots\oplus\mathcal{Q}^{(K)}$ . This setting evaluates whether experience acquired in earlier domains facilitates or interferes with performance on later domains, testing forward transfer in a sequential curriculum.

Interleaved.

A single agent receives a unified stream $\mathcal{Q}=\mathrm{shuffle}(\mathcal{Q}^{(1)}\cup\cdots\cup\mathcal{Q}^{(K)})$ in which tasks from all benchmarks are interleaved in randomized order. The agent maintains one shared evolution state $S_{t}$ that is updated by tasks from all benchmarks indiscriminately. This setting tests the agent’s ability to retrieve domain-relevant experience while suppressing cross-domain interference under maximal task diversity.

Rather than being ordered by expected difficulty, the three streaming scenarios are designed to decouple distinct aspects of self-evolution, each capturing a challenge that self-evolving agents may encounter in deployment. Isolated removes cross-domain effects entirely, providing a controlled measurement of within-domain learning. Sequential introduces ordered domain shifts and tests whether accumulated experience enables forward transfer. Interleaved mixes tasks from all domains within a single stream, requiring the agent to retrieve relevant experience and suppress irrelevant interference without explicit domain boundaries. Crossing these three streaming scenarios with the self-evolving methods and frontier foundation models enables a controlled analysis of how each factor contributes to self-evolution.

Experimental Settings

Models.

We evaluate three frontier foundation models spanning different families and scales: GPT-5.4-medium [19], Gemini 3.1 Pro-medium [20], and Claude Opus 4.7-high [21]. This diversity allows us to analyze how model capacity influences test-time evolution across self-evolving methods.

Tasks.

Our evaluation suite comprises six diverse benchmarks covering a broad spectrum of agentic capabilities:

•

AppWorld [12]: interactive coding tasks requiring multi-app workflow execution with API understanding and dynamic environment interaction.
•

BFCL [13]: multi-step function calling evaluation across diverse domains, testing context-dependent tool use with missing parameters and long-context scenarios.
•

BrowseComp-Plus [22]: deep-research tasks requiring iterative web retrieval and complex information synthesis over a controlled document corpus.
•

HLE [15]: expert-level academic reasoning across dozens of disciplines, designed to challenge frontier models on questions resistant to memorization.
•

SWE-bench Verified [14]: real-world software engineering tasks requiring codebase understanding, fault localization, and patch generation.
•

Tau2 [16]: conversational agent tasks in dual-control environments where both agent and user take actions in a shared system, testing coordination and communication.

Self-Evolving Methods.

We select five representative methods that span context, memory, skill, and integrated harness evolution, collectively representing the principal evolution components: (1) ACE [4]: evolves agent context through modular generation, reflection, and curation of prompts, accumulating structured strategies that scale with long-context models. (2) A-Mem [6]: dynamically organizes agent memories using Zettelkasten-style indexing and linking, continuously refining contextual representations as new experience is integrated. (3) ReasoningBank [9]: distills generalizable reasoning strategies from both successful and failed trajectories into structured memory items. (4) AutoSkill [8]: extracts reusable skills from interaction experience through a lifecycle of extraction, structured representation, iterative refinement, and versioned maintenance. (5) Harness: inspired by [5, 11], we implement a harness evolution method that jointly maintains and updates system prompts, skills, and experience memory through reflection and revision.

Implementation Details.

Our evaluation infrastructure is built on Exgentic [92], a framework that standardizes communication between heterogeneous agent interfaces and benchmarks. All self-evolving methods are adapted to operate within this framework under a test-time setting. For text embedding, we adopt the all-MiniLM-L6-v2 model across all experiments. We sample $N=50$ tasks from each benchmark, where AppWorld uses the test-challenge split, BFCL uses the multi-turn base split, and Tau2 uses the telecom domain. Task-level performance is evaluated by the native scoring pipeline of each benchmark. All judge models and user simulator models required by the benchmarks are unified to GPT-5.4. To account for ordering effects, we run three random seeds that shuffle task order while keeping the task set fixed. In the Sequential setting, benchmarks are presented in the order AppWorld $\rightarrow$ BFCL $\rightarrow$ BrowseComp+ $\rightarrow$ HLE $\rightarrow$ SWE $\rightarrow$ Tau2. Across all three streaming scenarios, the within-benchmark task exposure order is held constant to ensure comparability.

Table 1: Main results (%) of self-evolving methods across three streaming scenarios, averaged over three random seeds. Red background indicates scores below the model’s vanilla baseline, and blue marks the best scenario within each self-evolving method.

Methods	Mode	AppWorld	BFCL	BrowseComp+	HLE	SWE	Tau2	Avg
GPT-5.4	Vanilla	44.6	66.0	50.0	2.0	62.0	50.0	45.8
ACE [4]	Isolated	39.1 $\pm$ 5.9	68.0 $\pm$ 12.2	46.7 $\pm$ 2.3	5.3 $\pm$ 1.2	60.0 $\pm$ 3.5	63.3 $\pm$ 6.1	47.1
	Sequential	38.9 $\pm$ 5.9	64.0 $\pm$ 5.3	48.7 $\pm$ 6.1	7.3 $\pm$ 1.2	62.7 $\pm$ 5.0	45.3 $\pm$ 22.3	44.5
	Interleaved	32.9 $\pm$ 1.4	58.7 $\pm$ 3.1	44.7 $\pm$ 2.3	6.0 $\pm$ 2.0	63.3 $\pm$ 5.0	48.7 $\pm$ 3.1	42.4
A-Mem [6]	Isolated	41.2 $\pm$ 9.5	68.7 $\pm$ 2.3	52.0 $\pm$ 7.2	8.7 $\pm$ 3.1	65.3 $\pm$ 1.2	50.0 $\pm$ 36.2	47.7
	Sequential	39.9 $\pm$ 4.5	66.7 $\pm$ 6.4	50.7 $\pm$ 3.1	10.0 $\pm$ 2.0	60.0 $\pm$ 5.3	62.7 $\pm$ 18.9	48.3
	Interleaved	37.9 $\pm$ 4.8	65.3 $\pm$ 3.1	52.7 $\pm$ 4.2	8.7 $\pm$ 2.3	62.0 $\pm$ 3.5	76.0 $\pm$ 21.6	50.4
ReasoningBank [9]	Isolated	40.9 $\pm$ 1.8	62.7 $\pm$ 7.0	45.3 $\pm$ 2.3	10.0 $\pm$ 2.0	59.3 $\pm$ 5.0	46.7 $\pm$ 12.2	44.2
	Sequential	42.9 $\pm$ 0.6	62.7 $\pm$ 2.3	47.3 $\pm$ 3.1	14.7 $\pm$ 4.6	62.7 $\pm$ 4.6	36.0 $\pm$ 14.4	44.4
	Interleaved	38.6 $\pm$ 3.5	63.3 $\pm$ 5.0	48.7 $\pm$ 4.2	12.0 $\pm$ 2.0	56.7 $\pm$ 6.4	53.3 $\pm$ 4.2	45.4
AutoSkill [8]	Isolated	40.1 $\pm$ 2.8	70.7 $\pm$ 1.2	48.0 $\pm$ 3.5	4.7 $\pm$ 1.2	57.3 $\pm$ 7.0	36.0 $\pm$ 14.0	42.8
	Sequential	38.2 $\pm$ 1.6	72.0 $\pm$ 3.5	43.3 $\pm$ 5.0	6.7 $\pm$ 1.2	58.0 $\pm$ 4.0	44.0 $\pm$ 13.1	43.7
	Interleaved	40.0 $\pm$ 1.7	72.0 $\pm$ 5.3	46.0 $\pm$ 2.0	4.7 $\pm$ 1.2	60.7 $\pm$ 1.2	40.0 $\pm$ 8.7	43.9
Harness [5, 11]	Isolated	37.4 $\pm$ 2.3	64.0 $\pm$ 8.7	46.0 $\pm$ 2.0	7.3 $\pm$ 5.8	61.3 $\pm$ 2.3	57.3 $\pm$ 12.2	45.6
	Sequential	35.1 $\pm$ 1.6	66.7 $\pm$ 2.3	45.3 $\pm$ 1.2	6.7 $\pm$ 1.2	57.3 $\pm$ 6.1	54.0 $\pm$ 5.3	44.2
	Interleaved	37.0 $\pm$ 4.9	65.3 $\pm$ 5.0	50.0 $\pm$ 5.3	6.7 $\pm$ 1.2	57.3 $\pm$ 2.3	46.0 $\pm$ 5.3	43.7
Gemini 3.1 Pro	Vanilla	41.8	58.0	34.0	52.0	64.0	90.0	56.6
ACE [4]	Isolated	43.2 $\pm$ 1.9	62.7 $\pm$ 13.3	44.0 $\pm$ 5.3	50.0 $\pm$ 0.0	64.0 $\pm$ 0.0	90.7 $\pm$ 9.5	59.1
	Sequential	41.2 $\pm$ 3.2	72.0 $\pm$ 9.2	43.3 $\pm$ 5.0	52.0 $\pm$ 0.0	68.0 $\pm$ 0.0	95.3 $\pm$ 3.1	62.0
	Interleaved	40.7 $\pm$ 2.0	52.7 $\pm$ 13.0	44.0 $\pm$ 6.0	50.7 $\pm$ 1.2	64.7 $\pm$ 1.2	96.7 $\pm$ 3.1	58.3
A-Mem [6]	Isolated	41.2 $\pm$ 6.7	54.0 $\pm$ 25.0	50.0 $\pm$ 4.0	52.0 $\pm$ 2.0	62.0 $\pm$ 5.3	92.7 $\pm$ 4.6	58.7
	Sequential	40.4 $\pm$ 3.5	54.7 $\pm$ 2.3	46.0 $\pm$ 2.0	48.7 $\pm$ 4.2	60.7 $\pm$ 1.2	91.3 $\pm$ 2.3	57.0
	Interleaved	40.4 $\pm$ 2.6	56.7 $\pm$ 16.7	47.3 $\pm$ 7.6	50.7 $\pm$ 1.2	60.7 $\pm$ 2.3	90.7 $\pm$ 6.1	57.8
ReasoningBank [9]	Isolated	42.8 $\pm$ 3.1	54.7 $\pm$ 4.6	50.0 $\pm$ 5.3	48.7 $\pm$ 4.2	62.0 $\pm$ 4.0	95.3 $\pm$ 2.3	58.9
	Sequential	42.0 $\pm$ 0.8	56.0 $\pm$ 2.0	44.0 $\pm$ 2.0	51.3 $\pm$ 1.2	66.0 $\pm$ 5.3	94.7 $\pm$ 3.1	59.0
	Interleaved	44.7 $\pm$ 1.9	48.7 $\pm$ 3.1	52.7 $\pm$ 4.2	52.7 $\pm$ 2.3	67.3 $\pm$ 3.1	94.7 $\pm$ 2.3	60.1
AutoSkill [8]	Isolated	43.1 $\pm$ 2.3	62.7 $\pm$ 3.1	46.7 $\pm$ 4.2	51.3 $\pm$ 3.1	64.7 $\pm$ 5.0	83.3 $\pm$ 5.0	58.6
	Sequential	45.5 $\pm$ 1.5	64.7 $\pm$ 5.8	41.3 $\pm$ 4.2	50.0 $\pm$ 3.5	62.7 $\pm$ 3.1	86.7 $\pm$ 5.0	58.5
	Interleaved	44.7 $\pm$ 1.2	65.3 $\pm$ 1.2	41.3 $\pm$ 6.1	50.0 $\pm$ 2.0	66.0 $\pm$ 2.0	87.3 $\pm$ 2.3	59.1
Harness [5, 11]	Isolated	39.9 $\pm$ 5.0	74.7 $\pm$ 3.1	42.7 $\pm$ 4.6	52.0 $\pm$ 3.5	68.0 $\pm$ 4.0	90.0 $\pm$ 5.3	61.2
	Sequential	41.9 $\pm$ 1.0	54.7 $\pm$ 23.1	35.3 $\pm$ 2.3	50.7 $\pm$ 2.3	64.7 $\pm$ 4.2	91.3 $\pm$ 4.2	56.4
	Interleaved	40.9 $\pm$ 2.0	74.7 $\pm$ 4.2	38.0 $\pm$ 16.4	51.3 $\pm$ 2.3	63.3 $\pm$ 3.1	90.7 $\pm$ 6.1	59.8
Claude Opus 4.7	Vanilla	41.2	86.0	70.0	38.0	68.0	80.0	63.9
ACE [4]	Isolated	47.5 $\pm$ 4.2	85.3 $\pm$ 1.2	72.0 $\pm$ 2.0	33.3 $\pm$ 4.2	70.7 $\pm$ 2.3	93.3 $\pm$ 3.1	67.0
	Sequential	47.5 $\pm$ 0.9	82.7 $\pm$ 3.1	68.7 $\pm$ 1.2	38.7 $\pm$ 2.3	70.7 $\pm$ 4.2	91.3 $\pm$ 5.0	66.6
	Interleaved	46.7 $\pm$ 0.8	80.7 $\pm$ 4.2	68.0 $\pm$ 0.0	34.0 $\pm$ 7.2	70.0 $\pm$ 3.5	72.0 $\pm$ 8.7	61.9
A-Mem [6]	Isolated	48.7 $\pm$ 2.5	85.3 $\pm$ 1.2	71.3 $\pm$ 1.2	27.3 $\pm$ 1.2	71.3 $\pm$ 6.4	90.7 $\pm$ 7.6	65.8
	Sequential	48.1 $\pm$ 0.4	82.7 $\pm$ 1.2	68.0 $\pm$ 4.0	32.7 $\pm$ 3.1	69.3 $\pm$ 4.2	96.0 $\pm$ 3.5	66.1
	Interleaved	49.3 $\pm$ 0.9	81.3 $\pm$ 1.2	67.3 $\pm$ 2.3	32.0 $\pm$ 2.0	72.7 $\pm$ 6.1	86.0 $\pm$ 6.9	64.8
ReasoningBank [9]	Isolated	44.0 $\pm$ 1.1	84.0 $\pm$ 0.0	70.0 $\pm$ 2.0	36.0 $\pm$ 2.0	69.3 $\pm$ 4.2	90.0 $\pm$ 0.0	65.6
	Sequential	41.8 $\pm$ 1.4	85.3 $\pm$ 1.2	71.3 $\pm$ 3.1	35.3 $\pm$ 3.1	66.7 $\pm$ 3.1	85.3 $\pm$ 1.2	64.3
	Interleaved	45.1 $\pm$ 0.9	84.7 $\pm$ 1.2	70.0 $\pm$ 3.5	39.3 $\pm$ 3.1	68.7 $\pm$ 1.2	88.7 $\pm$ 3.1	66.1
AutoSkill [8]	Isolated	41.9 $\pm$ 1.3	86.0 $\pm$ 0.0	73.3 $\pm$ 1.2	36.7 $\pm$ 1.2	68.0 $\pm$ 5.3	82.0 $\pm$ 3.5	64.7
	Sequential	41.8 $\pm$ 0.5	86.0 $\pm$ 0.0	72.7 $\pm$ 2.3	37.3 $\pm$ 2.3	63.3 $\pm$ 1.2	86.7 $\pm$ 7.6	64.6
	Interleaved	40.6 $\pm$ 2.4	86.7 $\pm$ 1.2	73.3 $\pm$ 1.2	38.0 $\pm$ 3.5	67.3 $\pm$ 3.1	86.7 $\pm$ 2.3	65.4
Harness [5, 11]	Isolated	44.8 $\pm$ 1.0	84.0 $\pm$ 2.0	68.7 $\pm$ 1.2	34.0 $\pm$ 5.3	69.3 $\pm$ 4.2	90.7 $\pm$ 3.1	65.3
	Sequential	42.6 $\pm$ 0.8	82.7 $\pm$ 2.3	71.3 $\pm$ 4.2	36.0 $\pm$ 2.0	72.0 $\pm$ 0.0	74.0 $\pm$ 28.0	63.1
	Interleaved	46.1 $\pm$ 2.1	85.3 $\pm$ 1.2	69.3 $\pm$ 1.2	37.3 $\pm$ 1.2	69.3 $\pm$ 2.3	87.3 $\pm$ 9.2	65.8

Results

Table 1 reports the main results across models, self-evolving methods and streaming scenarios, averaged over three random seeds. Per-seed results are provided in Tables 11, 12 and 13.

Does Self-Evolution Help at All, and Under Which Scenario?

Marginalizing over both models and methods in Table 2, we find that self-evolution is not uniformly beneficial and underperforms the vanilla baseline in a substantial share of configurations. Given this variability, we read Table 1 along the streaming-scenario axis alone, asking under which scenario self-evolution most reliably attains a positive evolution gain.

Table 2: Positive rate and average evolution gain (%) over vanilla baseline for each streaming scenario, aggregated over all model-method configurations. Top-1 rate is the percentage of configurations for which the scenario attains the highest accuracy. Blue marks the best scenario per metric.

Scenario	Positive ( $>$ Vanilla)	Negative ( $<$ Vanilla)	Gain	Top-1 rate
Isolated	34/45	11/45	+1.37 $\pm$ 0.80	38%
Sequential	28/45	16/45	+0.75 $\pm$ 0.48	29%
Interleaved	28/45	17/45	+0.90 $\pm$ 0.34	33%

Isolated is the most reliable streaming scenario for self-evolution.

As shown in Table 2, Isolated achieves the highest positive rate of 75.7% and the largest average evolution gain of +1.37%, substantially outperforming both Sequential at 62.3% positive rate and +0.75% gain, and Interleaved at 62.3% and +0.90%. This advantage is further validated by its highest Top-1 rate of 38%. When all tasks come from a single benchmark, the experience stream remains distributionally coherent. Each solved task contributes directly relevant knowledge for subsequent tasks, and no filtering or retrieval gating is needed to avoid cross-domain interference. This makes Isolated the lowest-risk streaming setting regardless of method architecture.

The two cross-domain scenarios do not follow the expected difficulty ordering.

A natural hypothesis is that Interleaved, which exposes the agent to tasks from all domains in mixed order, should pose the greatest challenge due to maximal cross-domain interference. However, the empirical results contradict this expectation. Although both scenarios achieve comparable positive rates of 62.3% in Table 2, Interleaved attains a higher average evolution gain of +0.90% against +0.75% for Sequential, together with a higher Top-1 rate of 33% versus 29%. The advantage of Interleaved becomes more apparent in direct pairwise comparison. Across all 15 configurations in Table 1, Interleaved achieves higher average accuracy than Sequential in 10 cases while Sequential leads in only 5. This pattern holds across models, with Interleaved prevailing in 4 out of 5 configurations on Gemini 3.1 Pro and 3 out of 5 on Claude Opus 4.7. On GPT-5.4, where both scenarios yield negative evolution gains, Interleaved still leads in 3 out of 5 configurations. Beyond its higher evolution gain, Interleaved also exhibits a smaller standard deviation than Sequential, indicating that its advantage is not only larger but also more stable. Together, these results suggest that the diversity of interleaved streams compensates for cross-domain noise more effectively than the ordered domain transitions in Sequential.

When Does Self-Evolution Help? The Role of Model Capability

Different models may respond differently to self-evolution. Figure 3 provides the marginal view averaged across all five methods, and Table 3 offers a finer-grained breakdown under the three streaming scenarios.

Figure 3: Average accuracy for each model under the three streaming scenarios.

Table 3: Positive rate and average evolution gain (%) over vanilla baseline for each model under each streaming scenario. Positive denotes the number of methods, out of 5, exceeding vanilla baseline.

	Isolated		Sequential		Interleaved
Model	Positive	Gain	Positive	Gain	Positive	Gain
GPT-5.4	2/5	$-$ 0.35	1/5	$-$ 0.78	1/5	$-$ 0.62
Gemini 3.1 Pro	5/5	+2.71	4/5	+1.98	5/5	+2.41
Claude Opus 4.7	5/5	+1.75	4/5	+1.05	4/5	+0.90

The benefit of self-evolution is gated by model capability.

As shown in Table 3, GPT-5.4 exhibits negative average evolution gains under all three streaming scenarios, ranging from $-$ 0.35% to $-$ 0.78%, with only 4 out of 15 configurations exceeding its vanilla baseline. In contrast, Gemini 3.1 Pro exceeds the vanilla baseline in 14 of 15 configurations with gains between +1.98% and +2.71%, and Claude Opus 4.7 in 13 of 15 with gains between +0.90% and +1.75%. This disparity persists across Isolated, Sequential, and Interleaved scenarios, as confirmed by per-seed results in Tables 11, 12 and 13. These results suggest that self-evolution relies on a bootstrap loop in which successful task completions generate high-quality experience that benefits subsequent tasks. When the base model’s solve rate is low, the experience stream is dominated by failed or partially correct trajectories, from which the agent cannot reliably extract transferable knowledge. Above a sufficient capability threshold, the model solves enough early tasks to seed the experience buffer with useful patterns, potentially initiating a virtuous cycle of accumulation and transfer that persists across all three streaming scenarios.

The evolution gain is non-monotonic in base model strength across streaming scenarios.

Between the two models that benefit from self-evolution, Gemini 3.1 Pro obtains a larger average evolution gain of +2.37% than Claude Opus 4.7 at +1.23%, despite Claude being the stronger model on the vanilla baseline. As shown in Table 3, this pattern holds across all three streaming scenarios and is corroborated in Tables 11, 12 and 13. Moreover, the magnitude of this difference varies with streaming complexity. The gap in evolution gain between Gemini 3.1 Pro and Claude Opus 4.7 is 0.96% under Isolated and 0.93% under Sequential, but widens to 1.51% under Interleaved. As the streaming scenario introduces more cross-domain mixing, the mid-capability model benefits increasingly more than the stronger model. Under Interleaved, the mixed task stream exposes the agent to diverse cross-domain experience, providing substantial learning signal for a model that still has broad room to improve across multiple benchmarks. A model that already achieves high baseline accuracy on most benchmarks extracts less additional value from this diversity.

How Does Model Capability Shape the Choice of Self-Evolving Method?

Having established that model capability gates when self-evolution helps, we now turn to the interaction between model and self-evolving method, asking how model capability shapes the choice of self-evolving method. Table 4 reports the accuracy of each method under each model, averaged across the three streaming scenarios.

Method sensitivity decreases with model capability.

As shown in Table 4, the spread between the best and worst method contracts monotonically with model strength. On GPT-5.4, the spread is 5.3% with only 1 of 5 methods exceeding the vanilla baseline, and the worst method losing 2.3%. On Gemini 3.1 Pro, the spread narrows to 2.0% with all 5 methods above baseline. On Claude Opus 4.7, it further contracts to 0.9%, with all methods clustered in a narrow band above the vanilla baseline. Results in Tables 11, 12 and 13 confirm this pattern, with GPT-5.4 exhibiting spreads of 7.0%, 5.6%, and 7.7% against 2.3%, 3.3%, and 2.2% for Claude Opus 4.7. The trend follows directly from the capability-gating effect identified in Section 5.2. On a weaker model, method architecture determines whether the agent can extract usable signal from a largely unsuccessful experience stream, making method choice decisive. On a stronger model, all methods reliably generate positive transfer, leaving less room for method choice to affect the outcome.

Table 4: Average accuracy (%) of each self-evolving method under each model, averaged across three streaming scenarios. Red background marks performance below the model’s vanilla baseline, and blue marks the best method within each model.

Model	Vanilla	ACE	A-Mem	ReasoningBank	AutoSkill	Harness
GPT-5.4	45.8	44.6	48.8	44.7	43.5	44.5
Gemini 3.1 Pro	56.6	59.8	57.8	59.4	58.8	59.2
Claude Opus 4.7	63.9	65.2	65.6	65.3	64.9	64.7

Method choice governs whether self-evolution equalizes or amplifies inter-model gaps.

Because method sensitivity concentrates on the weakest model, method choice determines the aggregate effect of self-evolution on inter-model performance gaps. Averaging across all five methods, the gap between GPT-5.4 and Claude Opus 4.7 widens from 18.1% at vanilla to 20.0% after evolution, as evolution harms the weakest model on average while consistently benefiting the strongest. However, pairing each model with its optimal method narrows the gap to 16.8%, as A-Mem [6] lifts GPT-5.4 to 48.8% while still reaching 65.6% on Claude Opus 4.7. Critically, the optimal method is not portable across models. A-Mem is the best method on GPT-5.4 but ranks lowest on Gemini 3.1 Pro, while ACE [4] leads on Gemini yet ranks third on the other two models. Only ReasoningBank [9] remains competitive across all three models.

How Do Self-Evolving Methods Interact with Streaming Scenarios?

Table 5: Average evolution gain (%) of self-evolving methods across three streaming scenarios, aggregated over all models. The last two columns indicate the number of configurations (out of 9) in which Isolated or Interleaved attains the higher performance.

Method	Isolated	Sequential	Interleaved	Isolated Higher	Interleaved Higher
ACE	+2.28	+2.26	$-$ 1.26	7	2
Harness	+1.91	$-$ 0.86	+1.01	5	3
A-Mem	+1.93	+1.71	+2.22	5	4
ReasoningBank	+0.78	+0.46	+1.79	2	7
AutoSkill	$-$ 0.07	+0.19	+0.72	2	7

Table 6: Average accuracy (%) of each self-evolving method across three streaming scenarios and three models. Blue marks the best scenario within each self-evolving method.

	GPT-5.4			Gemini 3.1 Pro			Claude Opus 4.7
Method	Isolated	Sequential	Interleaved	Isolated	Sequential	Interleaved	Isolated	Sequential	Interleaved
ACE	47.1	44.5	42.4	59.1	62.0	58.3	67.0	66.6	61.9
Harness	45.6	44.2	43.7	61.2	56.4	59.8	65.2	63.1	65.8
A-Mem	47.6	48.3	50.4	58.7	57.0	57.7	65.8	66.1	64.8
ReasoningBank	44.2	44.4	45.4	58.9	59.0	60.1	65.5	64.3	66.1
AutoSkill	42.8	43.7	43.9	58.6	58.5	59.1	64.7	64.6	65.4

Turning from the model to the streaming scenario, we ask how self-evolving methods interact with the streaming structure. A single design choice splits self-evolving methods along the scenario axis: how tightly each binds experience to the execution context. Context-integrated methods such as ACE [4] and Harness [5, 11] fold experience directly into the agent prompt, whereas retrieval-based methods such as ReasoningBank [9], AutoSkill [8], and A-Mem [6] keep it in an external store and inject only the entries retrieved for the current task. As shown in Table 5, the two families diverge along this axis. Context-integrated methods perform best under Isolated, with ACE attaining the highest average evolution gain of +2.28%, whereas retrieval-based methods each peak under Interleaved, with A-Mem reaching +2.22%. Table 6 corroborates this at the per-model level, where ACE and Harness reach their peak accuracy under Isolated for the majority of model configurations while ReasoningBank and AutoSkill consistently peak under Interleaved.

This interaction between method and streaming scenario can be explained by how each method couples stored knowledge with the execution context. For context-integrated methods such as ACE and Harness, under Isolated, this tight coupling converts experience into precise domain-specific strategies that transfer reliably across tasks within the same distribution. When the stream spans multiple domains in Interleaved, the gain of ACE drops sharply from +2.28% to $-$ 1.26%, and Harness decreases from +1.91% to +1.01%, indicating that tightly coupled experience is susceptible to cross-domain interference. For retrieval-based methods such as ReasoningBank, AutoSkill, and A-Mem, this gating mechanism suppresses cross-domain interference under Interleaved by activating only task-relevant experience, while the diversity of the interleaved tasks simultaneously drives the consolidation of transferable patterns across domains. Under Sequential, these methods maintain moderate gains of +0.46%, +0.19%, and +1.71% respectively, as the retrieval gate filters out experience from earlier domains that is no longer relevant to the current one.

Conclusion

We introduce AgentStream, a unified framework that organizes agentic benchmarks into a configurable task stream and evaluates self-evolving agents under three streaming scenarios across multiple models and methods. Our analysis reveals that self-evolution reliability varies across streaming scenarios, the benefit of self-evolution is gated by model capability and non-monotonic in model strength, and no single method dominates across models and streaming scenarios. These results provide practical guidance for deploying self-evolving agents and highlight the value of evaluating them under realistic streaming settings.

Limitations

Because a given model performs unevenly across benchmarks and our evaluation is instantiated within the Exgentic [92] framework, the notion of model capability strength used throughout this work is grounded in the empirical observations under our specific experimental setup rather than a universally valid ranking of the models. Different agent frameworks, prompting strategies, or benchmark selections may alter the relative ordering, and our conclusions regarding capability gating should therefore be interpreted within this scope. In addition, we instantiate the framework with three streaming scenarios and six agentic benchmarks, which cover representative but not exhaustive stream compositions and task domains. Broader coverage of streaming scenarios, benchmarks, models, and self-evolving methods is a direction for future work, and AgentStream is designed to be extensible along all of these axes.

References

[1] H. Gao, J. Geng, W. Hua, M. Hu, X. Juan, H. Liu, S. Liu, J. Qiu, X. Qi, Q. Ren, Y. Wu, H. WANG, H. Xiao, Y. Zhou, S. Zhang, J. Zhang, J. Xiang, Y. Fang, Q. Zhao, D. Liu, C. Qian, Z. Wang, M. Hu, H. Wang, Q. Wu, H. Ji, and M. Wang (2026) A survey of self-evolving agents: what, when, how, and where to evolve on the path to artificial super intelligence. Transactions on Machine Learning Research. External Links: ISSN 2835-8856 Cited by: §1, §2.2.
[2] M. Lin, H. Lu, Z. Shi, B. He, R. Mao, Z. Zhang, Z. Wu, X. Tang, H. Liu, Z. Dai, X. Zhang, S. Wang, B. Dumoulin, and J. Pei (2026) Position: agentic evolution is the path to evolving llms. arXiv preprint arXiv:2602.00359. Cited by: §1, §2.2.
[3] J. Fang, Y. Peng, X. Zhang, Y. Wang, X. Yi, G. Zhang, Y. Xu, B. Wu, S. Liu, Z. Li, et al. (2025) A comprehensive survey of self-evolving ai agents: a new paradigm bridging foundation models and lifelong agentic systems. arXiv preprint arXiv:2508.07407. Cited by: §1, §2.2.
[4] Q. Zhang, C. Hu, S. Upasani, B. Ma, F. Hong, V. Kamanuru, J. Rainton, C. Wu, M. Ji, H. Li, U. Thakker, J. Zou, and K. Olukotun (2026) Agentic context engineering: evolving contexts for self-improving language models. In Proc. ICLR, Cited by: Appendix A, §B.1, §1, §1, §1, §2.2, §4, Table 1, Table 1, Table 1, §5.3, §5.4.
[5] M. Lin, J. Wu, Z. Wang, Z. Shi, Y. Sang, B. He, Z. Liu, T. Wei, Z. Wu, Z. Zhang, D. Wang, X. Zhang, B. Dumoulin, C. Xie, Y. Zhou, S. Wang, and H. Lu (2026) Harness updating is not harness benefit: disentangling evolution capabilities in self-evolving llm agents. arXiv preprint arXiv:2605.30621. Cited by: Appendix A, §B.1, §1, §1, §2.2, §4, Table 1, Table 1, Table 1, §5.4.
[6] W. Xu, Z. Liang, K. Mei, H. Gao, J. Tan, and Y. Zhang (2025) A-mem: agentic memory for llm agents. In Proc. NeurIPS, Cited by: Appendix A, §1, §1, §1, §2.2, §4, Table 1, Table 1, Table 1, §5.3, §5.4.
[7] S. Zhang, J. Wang, R. Zhou, J. Liao, Y. Feng, Z. Li, Y. Zheng, W. Zhang, Y. Wen, Z. Li, F. Xiong, Y. Qi, B. Tang, and M. Wen (2026) MemRL: self-evolving agents via runtime reinforcement learning on episodic memory. arXiv preprint arXiv:2601.03192. Cited by: §1, §2.2.
[8] Y. Yang, J. Li, Q. Pan, B. Zhan, Y. Cai, L. Du, J. Zhou, K. Chen, Q. Chen, X. Li, B. Zhang, and L. He (2026) AutoSkill: experience-driven lifelong learning via skill self-evolution. arXiv preprint arXiv:2603.01145. Cited by: Appendix A, §B.1, §1, §1, §2.2, §4, Table 1, Table 1, Table 1, §5.4.
[9] S. Ouyang, J. Yan, I. Hsu, Y. Chen, K. Jiang, Z. Wang, R. Han, L. T. Le, S. Daruki, X. Tang, V. Tirumalashetty, G. Lee, M. Rofouei, H. Lin, J. Han, C. Lee, and T. Pfister (2026) ReasoningBank: scaling agent self-evolving with reasoning memory. In Proc. ICLR, Cited by: Appendix A, §B.1, §1, §1, §1, §3.1, §4, Table 1, Table 1, Table 1, §5.3, §5.4.
[10] H. Zhou, S. Guo, A. Liu, Z. Yu, Z. Gong, B. Zhao, Z. Chen, M. Zhang, Y. Chen, J. Li, R. Yang, Q. Liu, X. Yu, J. Zhou, N. Wang, C. Sun, and J. Wang (2026) Memento-skills: let agents design agents. arXiv preprint arXiv:2603.18743. Cited by: §1, §2.2.
[11] J. Lin, S. Liu, C. Pan, L. Lin, S. Dou, Z. Xi, X. Huang, H. Yan, Z. Han, T. Gui, et al. (2026) Agentic harness engineering: observability-driven automatic evolution of coding-agent harnesses. arXiv preprint arXiv:2604.25850. Cited by: Appendix A, §B.1, §1, §1, §2.2, §4, Table 1, Table 1, Table 1, §5.4.
[12] H. Trivedi, T. Khot, M. Hartmann, R. Manku, V. Dong, E. Li, S. Gupta, A. Sabharwal, and N. Balasubramanian (2024) AppWorld: a controllable world of apps and people for benchmarking interactive coding agents. In Proc. ACL, Cited by: §1, §1, §2.3, 1st item.
[13] S. G. Patil, H. Mao, F. Yan, C. C. Ji, V. Suresh, I. Stoica, and J. E. Gonzalez (2025) The berkeley function calling leaderboard (BFCL): from tool use to agentic evaluation of large language models. In Proc. ICML, Cited by: §1, §1, §2.3, 2nd item.
[14] C. E. Jimenez, J. Yang, A. Wettig, S. Yao, K. Pei, O. Press, and K. Narasimhan (2024) Swe-bench: can language models resolve real-world github issues?. In Proc. ICLR, Cited by: §1, §1, §2.3, 5th item.
[15] L. Phan, A. Gatti, Z. Han, N. Li, J. Hu, H. Zhang, C. B. C. Zhang, M. Shaaban, J. Ling, S. Shi, et al. (2025) Humanity’s last exam. arXiv preprint arXiv:2501.14249. Cited by: §1, §1, §2.3, 4th item.
[16] V. Barres, H. Dong, S. Ray, X. Si, and K. Narasimhan (2025) $\tau^{2}$ -Bench: Evaluating Conversational Agents in a Dual-Control Environment. arXiv preprint arXiv:2506.07982. Cited by: §1, §1, §2.3, 6th item.
[17] C. Wu, Z. R. Tam, C. Lin, Y. Chen, and H. Lee (2024) Streambench: towards benchmarking continuous improvement of language agents. In Proc. NeurIPS, Cited by: §1, §2.3, §3.1.
[18] T. Wei, N. Sachdeva, B. Coleman, Z. He, Y. Bei, X. Ning, M. Ai, Y. Li, J. He, E. H. Chi, C. Wang, S. Chen, F. Pereira, W. Kang, and D. Z. Cheng (2026) Evo-memory: benchmarking llm agent test-time learning with self-evolving memory. arXiv preprint arXiv:2511.20857. Cited by: §1, §2.1, §2.3.
[19] A. Singh, A. Fry, A. Perelman, A. Tart, A. Ganesh, A. El-Kishky, A. McLaughlin, A. Low, A. Ostrow, A. Ananthram, et al. (2025) Openai gpt-5 system card. arXiv preprint arXiv:2601.03267. Cited by: §1, §4.
[20] Google DeepMind (2026) Gemini 3.1 Pro model card. External Links: Link Cited by: §1, §4.
[21] Anthropic (2026) Introducing Claude Opus 4.7. External Links: Link Cited by: §1, §4.
[22] Z. Chen, X. Ma, S. Zhuang, P. Nie, K. Zou, A. Liu, J. Green, K. Patel, R. Meng, M. Su, S. Sharifymoghaddam, Y. Li, H. Hong, X. Shi, X. Liu, N. Thakur, C. Zhang, L. Gao, W. Chen, and J. Lin (2026) BrowseComp-plus: a more fair and transparent evaluation benchmark of deep-research agent. In Proc. ACL, Cited by: §1, 3rd item.
[23] Y. Sun, X. Wang, Z. Liu, J. Miller, A. Efros, and M. Hardt (2020) Test-time training with self-supervision for generalization under distribution shifts. In Proc. ICML, Cited by: §2.1.
[24] J. Liang, D. Hu, and J. Feng (2020) Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation. In Proc. ICML, Cited by: §2.1.
[25] J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, et al. (2017) Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114 (13), pp. 3521–3526. Cited by: §2.1.
[26] D. Lopez-Paz and M. Ranzato (2017) Gradient episodic memory for continual learning. In Proc. NeurIPS, Cited by: §2.1.
[27] C. Snell, J. Lee, K. Xu, and A. Kumar (2025) Scaling llm test-time compute optimally can be more effective than scaling model parameters. In Proc. ICLR, Cited by: §2.1.
[28] M. Hardt and Y. Sun (2024) Test-time training on nearest neighbors for large language models. In Proc. ICLR, Cited by: §2.1.
[29] J. Hübotter, S. Bongni, I. Hakimi, and A. Krause (2025) Efficiently learning at test-time: active fine-tuning of llms. In Proc. ICLR, Cited by: §2.1.
[30] E. Akyürek, M. Damani, A. Zweiger, L. Qiu, H. Guo, J. Pari, Y. Kim, and J. Andreas (2025) The surprising effectiveness of test-time training for few-shot learning. In Proc. ICML, Cited by: §2.1.
[31] G. Feng, S. Luo, K. Hua, G. Zhang, W. Huang, D. He, and T. Cai (2026) In-place test-time training. In Proc. ICLR, Cited by: §2.1.
[32] A. Chen, Z. Liu, J. Zhang, A. Prabhakar, Z. Liu, S. Heinecke, S. Savarese, V. Zhong, and C. Xiong (2026) Test-time adaptation for llm agents via environment interaction. In Proc. ICLR, Cited by: §2.1.
[33] J. Hu, Z. Zhang, G. Chen, X. Wen, C. Shuai, W. Luo, B. Xiao, Y. Li, and M. Tan (2025) Test-time learning for large language models. In Proc. ICML, Cited by: §2.1.
[34] Y. Zuo, K. Zhang, L. Sheng, S. Qu, G. Cui, X. Zhu, H. Li, Y. Zhang, X. Long, E. Hua, B. Qi, Y. Sun, Z. Ma, L. Yuan, N. Ding, and B. Zhou (2025) TTRL: test-time reinforcement learning. In Proc. NeurIPS, Cited by: §2.1.
[35] J. Hübotter, L. Diaz-Bone, I. Hakimi, A. Krause, and M. Hardt (2025) Learning on the job: test-time curricula for targeted reinforcement learning. arXiv preprint arXiv:2510.04786. Cited by: §2.1.
[36] M. Yuksekgonul, D. Koceja, X. Li, F. Bianchi, J. McCaleb, X. Wang, J. Kautz, Y. Choi, J. Zou, C. Guestrin, and Y. Sun (2026) Learning to discover at test time. In Proc. ICML, Cited by: §2.1.
[37] Z. Hu, Y. Hu, J. Liu, S. S. Li, Y. Wang, Z. Xu, S. Ng, A. T. Luu, X. Xu, B. Hooi, C. Breazeal, and H. W. Park (2026) Collaborative multi-agent test-time reinforcement learning for reasoning. arXiv preprint arXiv:2601.09667. Cited by: §2.1.
[38] D. Yan, J. Liang, Y. Wang, S. Lu, R. He, and T. Tan (2026) What if consensus lies? selective-complementary reinforcement learning at test time. In Proc. ACL, Cited by: §2.1.
[39] H. He, Z. Rong, L. Zhao, Y. Zhao, L. Yang, and H. Zhang (2026) TTSR: test-time self-reflection for continual reasoning improvement. arXiv preprint arXiv:2603.03297. Cited by: §2.1.
[40] C. Yang, Z. Xiang, Y. Tang, Z. Teng, C. Huang, F. Long, Y. Liu, and J. Su (2026) TTCS: test-time curriculum synthesis for self-evolving. arXiv preprint arXiv:2601.22628. Cited by: §2.1.
[41] W. Xu, A. Sordoni, C. Singh, Z. Gero, M. Galley, X. Yuan, and J. Gao (2026) Test-time learning with an evolving library. arXiv preprint arXiv:2605.14477. Cited by: §2.1.
[42] J. Wang, C. Zhou, Z. Fu, J. Wang, W. Liu, W. Zhang, and J. Lin (2026) Skills on the fly: test-time adaptive skill synthesis for llm agents. arXiv preprint arXiv:2605.16986. Cited by: §2.1.
[43] J. Wang, Z. Tao, H. Zeng, Z. Yang, H. Zamani, and H. Yu (2026) TARSE: test-time adaptation via retrieval of skills and experience for reasoning agents. arXiv preprint arXiv:2603.01241. Cited by: §2.1.
[44] Q. Zhang, M. Wornow, G. Wan, and K. Olukotun (2025) Agentic plan caching: test-time memory for fast and cost-efficient llm agents. In Proc. NeurIPS, Cited by: §2.1.
[45] Y. Cheng, Y. Hu, J. Zhou, Y. Zhang, Y. Chen, H. Zhou, M. Chen, Z. Zhang, K. Shao, Y. Xie, and Z. Yin (2026) TAME: a trustworthy test-time evolution of agent memory with systematic benchmarking. arXiv preprint arXiv:2602.03224. Cited by: §2.1.
[46] E. C. Acikgoz, C. Qian, H. Ji, D. Hakkani-Tür, and G. Tur (2025) Self-improving llm agents at test-time. arXiv preprint arXiv:2510.07841. Cited by: §2.1.
[47] Y. Li, Z. Lin, A. Deng, X. Zhang, Y. He, S. Ji, T. Cao, and B. Hooi (2026) Just-in-time reinforcement learning: continual learning in llm agents without gradient updates. In Proc. ICML, Cited by: §2.1.
[48] S. Rajesh, P. Holur, M. Y. Turali, C. Duan, and V. Roychowdhury (2026) Panini: continual learning in token space via structured memory. In Proc. ICML, Cited by: §2.1.
[49] Z. Wu, X. Lou, X. Ma, Y. Li, W. Liu, W. Zhang, J. Wang, and Z. Zhang (2026) Agent-dice: disentangling knowledge updates via geometric consensus for agent continual learning. arXiv preprint arXiv:2601.03641. Cited by: §2.1.
[50] Y. Lu, Y. He, J. Chen, and H. Zha (2026) MSSR: memory-aware adaptive replay for continual llm fine-tuning. arXiv preprint arXiv:2603.09892. Cited by: §2.1.
[51] Y. Xiong, S. Hu, and J. Clune (2026) Learning to continually learn via meta-learning agentic memory designs. arXiv preprint arXiv:2602.07755. Cited by: §2.1.
[52] G. Jiang, Z. Su, X. Qu, and Y. R. Fung (2026) Xskill: continual learning from experience and skills in multimodal agents. In Proc. ICML, Cited by: §2.1.
[53] T. Ye, L. Dong, Q. Dong, X. Wu, S. Huang, and F. Wei (2026) Online experiential learning for language models. arXiv preprint arXiv:2603.16856. Cited by: §2.1.
[54] W. Yang, D. Cao, J. Pang, M. Weng, and Y. Liu (2026) Adaptive collaboration with humans: metacognitive policy optimization for multi-agent llms with continual learning. In Proc. ICLR, Cited by: §2.1.
[55] H. Zhang, Q. Long, J. Bao, T. Feng, W. Zhang, H. Yue, and W. Wang (2026) MemSkill: learning and evolving memory skills for self-evolving agents. arXiv preprint arXiv:2602.02474. Cited by: §2.2.
[56] P. Chhikara, D. Khant, S. Aryan, T. Singh, and D. Yadav (2025) Mem0: building production-ready ai agents with scalable long-term memory. arXiv preprint arXiv:2504.19413. Cited by: §2.2.
[57] P. Xia, J. Chen, H. Wang, J. Liu, K. Zeng, Y. Wang, S. Han, Y. Zhou, X. Zhao, H. Chen, Z. Zheng, C. Xie, and H. Yao (2026) SkillRL: evolving agents via recursive skill-augmented reinforcement learning. arXiv preprint arXiv:2602.08234. Cited by: §2.2.
[58] S. Ouyang, J. Yan, Y. Chen, R. Han, Z. Wang, B. D. Mishra, R. Meng, C. Li, Y. Jiao, K. Zha, M. Shen, V. Tirumalashetty, G. Lee, J. Han, T. Pfister, and C. Lee (2026) SkillOS: learning skill curation for self-evolving agents. arXiv preprint arXiv:2605.06614. Cited by: §2.2.
[59] S. Alzubi, N. Provenzano, J. Bingham, W. Chen, and T. Vu (2026) EvoSkill: automated skill discovery for multi-agent systems. arXiv preprint arXiv:2603.02766. Cited by: §2.2.
[60] Z. Yan, D. Song, H. Zhang, W. Liang, Y. Zhang, Y. Dai, L. He, P. S. Yu, R. Xu, X. Li, and L. Sun (2026) OpenSkill: open-world self-evolution for llm agents. arXiv preprint arXiv:2606.06741. Cited by: §2.2.
[61] Y. Yang, Z. Gong, W. Huang, Q. Yang, Z. Zhou, Z. Huang, Y. Li, X. Gao, Q. Dai, B. Liu, K. Qiu, Y. Yang, D. Chen, X. Yang, and C. Luo (2026) SkillOpt: executive strategy for self-evolving agent skills. arXiv preprint arXiv:2605.23904. Cited by: §2.2.
[62] Y. Lee, R. Nair, Q. Zhang, K. Lee, O. Khattab, and C. Finn (2026) Meta-harness: end-to-end optimization of model harnesses. arXiv preprint arXiv:2603.28052. Cited by: §2.2.
[63] X. Guo, J. Kuang, L. Pan, Y. Li, Y. Li, H. Zheng, Y. Shen, D. Yin, and X. Sun (2026) EvoConfig: self-evolving multi-agent systems for efficient autonomous environment configuration. arXiv preprint arXiv:2601.16489. Cited by: §2.2.
[64] D. Zhang, X. Liu, L. Cheng, Y. Wang, K. Murray, and H. Wei (2026) SELAUR: self evolving llm agent via uncertainty-aware rewards. In Proc. PAKDD, Cited by: §2.2.
[65] G. Xu, Z. Qi, H. Su, W. Ye, H. Lakkaraju, S. M. Kakade, and Y. Du (2026) Self-improving language models with bidirectional evolutionary search. arXiv preprint arXiv:2605.28814. Cited by: §2.2.
[66] E. C. Acikgoz, C. Qian, J. Hübotter, H. Ji, D. Hakkani-Tür, and G. Tur (2026) Tool-r0: self-evolving llm agents for tool-learning from zero data. arXiv preprint arXiv:2602.21320. Cited by: §2.2.
[67] L. Sheng, W. Ma, R. Hong, X. Wang, A. Zhang, and T. Chua (2026) Reinforcing chain-of-thought reasoning with self-evolving rubrics. arXiv preprint arXiv:2602.10885. Cited by: §2.2.
[68] Y. Wang, J. Zhao, H. Xie, H. Ma, Y. Lei, S. Liu, X. Song, Z. Zhang, and H. Zhang (2026) MetaGen: self-evolving roles and topologies for multi-agent llm reasoning. arXiv preprint arXiv:2601.19290. Cited by: §2.2.
[69] Y. Hu, Y. Cai, Y. Du, X. Zhu, X. Liu, Z. Yu, Y. Hou, S. Tang, and S. Chen (2025) Self-evolving multi-agent collaboration networks for software development. In Proc. ICLR, Cited by: §2.2.
[70] S. Liu, J. Fang, H. Zhou, Y. Wang, and Z. Meng (2026) SEW: self-evolving agentic workflows for automated code generation. arXiv preprint arXiv:2505.18646. Cited by: §2.2.
[71] S. Yang, S. C. Han, X. Ma, Y. Li, M. R. G. Madani, and E. Hovy (2026) EvoTool: self-evolving tool-use policy optimization in llm agents via blame-aware mutation and diversity-aware selection. arXiv preprint arXiv:2603.04900. Cited by: §2.2.
[72] A. Novikov, N. Vũ, M. Eisenberger, E. Dupont, P. Huang, A. Z. Wagner, S. Shirobokov, B. Kozlovskii, F. J. R. Ruiz, A. Mehrabian, M. P. Kumar, A. See, S. Chaudhuri, G. Holland, A. Davies, S. Nowozin, P. Kohli, and M. Balog (2025) AlphaEvolve: a coding agent for scientific and algorithmic discovery. arXiv preprint arXiv:2506.13131. Cited by: §2.2.
[73] Y. He, J. Liu, Y. Liu, Y. Li, T. Cao, Z. Hu, X. Xu, and B. Hooi (2026) EvoTest: evolutionary test-time learning for self-improving agentic systems. In Proc. ICLR, Cited by: §2.2.
[74] Y. Cai, Y. Hao, J. Zhou, H. Yan, Z. Lei, R. Zhen, Z. Han, Y. Yang, J. Li, Q. Pan, T. Huai, Q. Chen, X. Li, K. Chen, B. Zhang, X. Qiu, and L. He (2026) Building self-evolving agents via experience-driven lifelong learning: a framework and benchmark. arXiv preprint arXiv:2508.19005. Cited by: §2.2.
[75] M. Yuksekgonul, F. Bianchi, J. Boen, S. Liu, P. Lu, Z. Huang, C. Guestrin, and J. Zou (2025) Optimizing generative ai by backpropagating language model feedback. Nature 639, pp. 609–616. Cited by: §2.2.
[76] S. Shao, Q. Ren, C. Qian, B. Wei, D. Guo, J. Yang, X. Song, L. Zhang, W. Zhang, D. Liu, and J. Shao (2026) Your agent may misevolve: emergent risks in self-evolving llm agents. In Proc. ICLR, Cited by: §2.2.
[77] S. Yao, H. Chen, J. Yang, and K. Narasimhan (2022) WebShop: towards scalable real-world web interaction with grounded language agents. In Proc. NeurIPS, Cited by: §2.3.
[78] S. Zhou, F. F. Xu, H. Zhu, X. Zhou, R. Lo, A. Sridhar, X. Cheng, T. Ou, Y. Bisk, D. Fried, U. Alon, and G. Neubig (2024) WebArena: a realistic web environment for building autonomous agents. In Proc. ICLR, Cited by: §2.3.
[79] H. He, W. Yao, K. Ma, W. Yu, Y. Dai, H. Zhang, Z. Lan, and D. Yu (2024) WebVoyager: building an end-to-end web agent with large multimodal models. In Proc. ACL, Cited by: §2.3.
[80] T. Xie, D. Zhang, J. Chen, X. Li, S. Zhao, R. Cao, T. J. Hua, Z. Cheng, D. Shin, F. Lei, Y. Liu, Y. Xu, S. Zhou, S. Savarese, C. Xiong, V. Zhong, and T. Yu (2024) OSWorld: benchmarking multimodal agents for open-ended tasks in real computer environments. In Proc. NeurIPS, Cited by: §2.3.
[81] C. Bandi, R. Dumitru, B. Hertzberg, D. Agarwal, G. Boo, T. Polakam, S. Hassaan, J. Da, H. Kim, V. Gupta, M. Sharma, A. Park, M. Dimakis, E. G. H. Montoya, D. Rambado, I. Salazar, R. Cruz, M. Rezaei, C. Rane, B. Levin, D. Y. Zhang, B. Kenstler, and B. Liu (2026) MCP-atlas: a large-scale benchmark for tool-use competency with real mcp servers. arXiv preprint arXiv:2602.00933. Cited by: §2.3.
[82] J. Li, W. Zhao, J. Zhao, W. Zeng, H. Wu, X. Wang, R. Ge, Y. Cao, Y. Huang, W. Liu, et al. (2026) The tool decathlon: benchmarking language agents for diverse, realistic, and long-horizon task execution. In Proc. ICLR, Cited by: §2.3.
[83] M. A. Merrill, A. G. Shaw, N. Carlini, B. Li, H. Raj, I. Bercovich, L. Shi, J. Y. Shin, T. Walshe, E. K. Buchanan, et al. (2026) Terminal-bench: benchmarking agents on hard, realistic tasks in command line interfaces. In Proc. ICLR, Cited by: §2.3.
[84] Z. Wang, T. Shi, J. He, M. Cai, J. Zhang, and D. Song (2026) CyberGym: evaluating AI agents’ real-world cybersecurity capabilities at scale. In Proc. ICLR, Cited by: §2.3.
[85] X. Li, Y. Liu, W. Chen, B. You, Z. Di, Y. He, S. Zheng, K. W. Choe, J. Sun, S. Wang, et al. (2026) SkillsBench: benchmarking how well agent skills work across diverse tasks. arXiv preprint arXiv:2602.12670. Cited by: §2.3.
[86] Z. Zhang, K. Shi, S. Huang, A. Nie, Y. Zeng, Y. Zhao, Z. Fang, Q. Su, H. Qiu, W. Yang, et al. (2026) SkillFlow: benchmarking lifelong skill discovery and evolution for autonomous agents. arXiv preprint arXiv:2604.17308. Cited by: §2.3.
[87] G. Mialon, C. Fourrier, T. Wolf, Y. LeCun, and T. Scialom (2024) Gaia: a benchmark for general ai assistants. In Proc. ICLR, Cited by: §2.3.
[88] J. Wei, Z. Sun, S. Papay, S. McKinney, J. Han, I. Fulford, H. W. Chung, A. T. Passos, W. Fedus, and A. Glaese (2025) Browsecomp: a simple yet challenging benchmark for browsing agents. arXiv preprint arXiv:2504.12516. Cited by: §2.3.
[89] T. Patwardhan, R. Dias, E. Proehl, G. Kim, M. Wang, O. Watkins, S. P. Fishman, M. Aljubeh, P. Thacker, L. Fauconnet, et al. (2025) Gdpval: evaluating ai model performance on real-world economically valuable tasks. arXiv preprint arXiv:2510.04374. Cited by: §2.3.
[90] A. Bigeard, L. Nashold, R. Krishnan, and S. Wu (2025) Finance agent benchmark: benchmarking llms on real-world financial research tasks. arXiv preprint arXiv:2508.00828. Cited by: §2.3.
[91] Z. Z. Wang, J. Mao, D. Fried, and G. Neubig (2025) Agent workflow memory. In Proc. ICML, Cited by: §2.3.
[92] E. Bandel, A. Yehudai, L. Eden, Y. Sagron, Y. Perlitz, E. Venezian, N. Razinkov, N. Ergas, S. S. Ifergan, S. Shlomov, et al. (2026) General agent evaluation. In Proc. ICML, Cited by: Appendix E, §4, §7.

Appendix A Cost Analysis

A practical concern for deploying self-evolving agents is whether performance gains justify the additional cost. Tables 7 and 8 report the per-task cost, relative cost overhead, and average agent steps for GPT-5.4 and Gemini 3.1 Pro respectively, where all costs are reported using LiteLLM’s pricing data¹¹ 1 Model prices.. Figure 4 summarizes these results at the method level, averaged across three streaming scenarios.

Self-evolution does not necessarily increase inference cost. On Gemini 3.1 Pro, four methods operate below the vanilla baseline cost, reducing it to 64% for ReasoningBank [9], 71% for Harness [5, 11], 82% for AutoSkill [8], and 84% for A-Mem [6]. These methods simultaneously reduce the average number of agent steps, suggesting that accumulated experience helps the agent reach solutions more efficiently. On GPT-5.4, the pattern reverses. Most methods incur higher cost, with A-Mem reaching 577% of the baseline and ACE [4] at 266%, while ReasoningBank remains the only cost-reducing method at 92% of the baseline.

The cost-performance tradeoff is governed by the model. On Gemini 3.1 Pro, ACE and AutoSkill both achieve evolution gains above +3% with moderate or reduced cost overhead. ReasoningBank provides +2.3% evolution gain while reducing cost to 64% of the baseline, making it the most cost-efficient method on this model. On GPT-5.4, only A-Mem achieves a positive evolution gain of +3.2%, but at 577% of the baseline cost. All other methods on GPT-5.4 incur negative evolution gains regardless of their cost overhead. This asymmetry echoes the finding in Section 5.2 that weaker models struggle to convert accumulated experience into performance improvements, and further shows that this limitation extends to cost efficiency.

Figure 4: Cost overhead and evolution gain of each self-evolving method averaged across three streaming scenarios based on a single evaluation.

Table 7: Per-task cost ($), relative cost overhead, evolution gain (%), and average agent steps for GPT-5.4 based on a single evaluation. The vanilla baseline operates at $0.297 per task with 9.5 steps and 45.8% accuracy.

Method	Mode	Cost	Relative	$\Delta$ Acc.	Steps	$\Delta$ Steps (%)
ACE	Isolated	0.504	1.70 $\times$	$-$ 1.6	11.3	+18.9
	Sequential	0.977	3.29 $\times$	$-$ 7.3	10.4	+9.5
	Interleaved	0.890	3.00 $\times$	$-$ 2.4	10.5	+10.5
Harness	Isolated	0.385	1.30 $\times$	+2.5	10.2	+7.4
	Sequential	0.460	1.55 $\times$	$-$ 2.7	10.5	+10.5
	Interleaved	0.390	1.31 $\times$	$-$ 1.7	10.1	+6.3
A-Mem	Isolated	1.419	4.78 $\times$	$-$ 2.2	8.1	$-$ 14.7
	Sequential	1.827	6.15 $\times$	+6.2	8.6	$-$ 9.5
	Interleaved	1.893	6.37 $\times$	+5.6	8.8	$-$ 7.4
ReasoningBank	Isolated	0.291	0.98 $\times$	+1.4	8.7	$-$ 8.4
	Sequential	0.270	0.91 $\times$	$\pm$ 0.0	8.1	$-$ 14.7
	Interleaved	0.258	0.87 $\times$	$-$ 3.7	8.2	$-$ 13.7
AutoSkill	Isolated	0.298	1.00 $\times$	$-$ 2.3	9.1	$-$ 4.2
	Sequential	0.324	1.09 $\times$	+0.3	10.2	+7.4
	Interleaved	0.309	1.04 $\times$	$-$ 1.8	9.4	$-$ 1.1

Table 8: Per-task cost ($), relative cost overhead, evolution gain (%), and average agent steps for Gemini 3.1 Pro based on a single evaluation. The vanilla baseline operates at $4.035 per task with 21.1 steps and 56.6% accuracy.

Method	Mode	Cost	Relative	$\Delta$ Acc.	Steps	$\Delta$ Steps (%)
ACE	Isolated	4.239	1.05 $\times$	+4.9	22.9	+8.5
	Sequential	3.499	0.87 $\times$	+6.6	21.0	$-$ 0.5
	Interleaved	5.791	1.44 $\times$	$-$ 1.7	23.3	+10.4
Harness	Isolated	3.791	0.94 $\times$	+3.3	21.1	+0.0
	Sequential	2.461	0.61 $\times$	$-$ 4.3	18.9	$-$ 10.4
	Interleaved	2.310	0.57 $\times$	$-$ 0.2	19.3	$-$ 8.5
A-Mem	Isolated	3.508	0.87 $\times$	+2.7	17.0	$-$ 19.4
	Sequential	3.659	0.91 $\times$	$-$ 1.8	17.5	$-$ 17.1
	Interleaved	2.945	0.73 $\times$	+3.4	16.2	$-$ 23.2
ReasoningBank	Isolated	2.613	0.65 $\times$	+2.5	19.7	$-$ 6.6
	Sequential	2.680	0.66 $\times$	+1.6	19.2	$-$ 9.0
	Interleaved	2.492	0.62 $\times$	+2.9	18.8	$-$ 10.9
AutoSkill	Isolated	2.809	0.70 $\times$	+2.2	20.0	$-$ 5.2
	Sequential	3.421	0.85 $\times$	+5.0	21.0	$-$ 0.5
	Interleaved	3.656	0.91 $\times$	+2.4	22.0	+4.3

Appendix B Model Evolution Behavior Analysis

Beyond aggregate performance, we analyze how the three models differ in their self-evolution behavior under the Interleaved stream, focusing on the evolution state each model accumulates and how that state is updated as tasks are processed.

Evolution State Accumulation

Table 9: Evolution state accumulated by each model over the Interleaved stream.

Method	Metric	GPT-5.4	Gemini 3.1 Pro	Claude Opus 4.7
ACE	playbook bullets	552	209	1251
ReasoningBank	memory items	894	585	898
A-Mem	evolutions	261	248	261
AutoSkill	skills	176	117	160
Harness	skills	181	137	463
Harness	prompt+memory (chars)	8,711	29,172	9,362

As shown in Table 9, the three models accumulate evolution states of markedly different sizes and structures. Claude Opus 4.7 generates the largest states across all methods, averaging 1251 ACE [4] playbook bullets and 463 Harness [5, 11] skills, while Gemini 3.1 Pro consistently produces the most compact states with only 209 ACE bullets and 117 AutoSkill [8] skills. GPT-5.4 generates a moderate number of items with relatively concise per-item content, balancing breadth of coverage with manageable state complexity. The contrast extends beyond volume to architectural preference: under Harness, Claude Opus 4.7 allocates the vast majority of its evolution budget to the skill library while maintaining only 9K characters of system prompt and memory, whereas Gemini 3.1 Pro distributes 29K characters across its system prompt and memory fields, favoring centralized guidance over distributed skill entries. A similar pattern holds for memory-based methods. Under ReasoningBank [9], Claude Opus 4.7 and GPT-5.4 each distill approximately 900 memory items while Gemini produces only 585, confirming that Gemini extracts fewer but more consolidated experience entries from each interaction.

Update Dynamics

Table 10 summarizes how the models update their Harness state, which provides the most informative view since it jointly maintains system prompt, memory, and skill library. The three models adopt distinct update strategies. GPT-5.4 is largely append-only, adding skills but rarely editing them, with only 5 edits on average over 300 tasks and no modification to the system prompt. Gemini 3.1 Pro spreads its updates more evenly, revising the memory and system prompt 267 times while also refining skills through 66 edits. Claude Opus 4.7 performs the most intensive refinement, revising on 96% of tasks while concentrating almost entirely on the skill library rather than the memory or system prompt.

Table 10: Evolution behavior on Harness state over the Interleaved stream.

	GPT-5.4	Gemini 3.1 Pro	Claude Opus 4.7
Revision rate	65%	69%	96%
Skills added	181	147	492
Skills edited	5	66	460
Memory/prompt edits	21	267	103

Appendix C Detailed Results across Random Seeds

The main results in Table 1 are averaged over three random seeds that share the same task set but differ in the global task arrival order. Tables 11, 12 and 13 provide the complete per-seed results.

Table 11: Main results (%) of self-evolving methods across three streaming scenarios under seed 42. Red background indicates scores below the model’s vanilla baseline, and blue marks the best scenario within each self-evolving method.

Methods	Mode	AppWorld	BFCL	BrowseComp+	HLE	SWE	Tau2	Avg
GPT-5.4	Vanilla	44.6	66.0	50.0	2.0	62.0	50.0	45.8
ACE	Isolated	33.1	54.0	44.0	6.0	58.0	70.0	44.2
	Sequential	33.1	58.0	54.0	8.0	58.0	20.0	38.5
	Interleaved	32.3	62.0	46.0	4.0	64.0	52.0	43.4
A-Mem	Isolated	41.8	70.0	60.0	12.0	66.0	12.0	43.6
	Sequential	41.8	64.0	48.0	10.0	64.0	84.0	52.0
	Interleaved	32.4	68.0	56.0	10.0	60.0	82.0	51.4
ReasoningBank	Isolated	42.9	70.0	48.0	8.0	54.0	60.0	47.2
	Sequential	42.9	60.0	48.0	12.0	60.0	52.0	45.8
	Interleaved	36.6	58.0	44.0	12.0	52.0	50.0	42.1
AutoSkill	Isolated	36.8	72.0	52.0	6.0	64.0	30.0	43.5
	Sequential	36.3	76.0	48.0	6.0	54.0	56.0	46.1
	Interleaved	42.0	66.0	46.0	4.0	60.0	46.0	44.0
Harness	Isolated	37.9	70.0	48.0	14.0	60.0	60.0	48.3
	Sequential	36.4	68.0	46.0	8.0	52.0	48.0	43.1
	Interleaved	40.6	66.0	46.0	8.0	60.0	44.0	44.1
Gemini 3.1 Pro	Vanilla	41.8	58.0	34.0	52.0	64.0	90.0	56.6
ACE	Isolated	41.2	74.0	46.0	50.0	64.0	94.0	61.5
	Sequential	40.9	74.0	48.0	52.0	68.0	96.0	63.2
	Interleaved	41.6	40.0	38.0	52.0	64.0	94.0	54.9
A-Mem	Isolated	33.6	62.0	54.0	50.0	66.0	90.0	59.3
	Sequential	36.9	52.0	46.0	44.0	60.0	90.0	54.8
	Interleaved	37.9	62.0	56.0	50.0	62.0	92.0	60.0
ReasoningBank	Isolated	44.4	60.0	44.0	50.0	62.0	94.0	59.1
	Sequential	41.3	58.0	42.0	52.0	64.0	92.0	58.2
	Interleaved	46.7	46.0	54.0	50.0	68.0	92.0	59.5
AutoSkill	Isolated	44.7	66.0	48.0	52.0	64.0	78.0	58.8
	Sequential	45.5	68.0	46.0	52.0	66.0	92.0	61.6
	Interleaved	45.8	64.0	40.0	52.0	66.0	86.0	59.0
Harness	Isolated	45.5	72.0	40.0	50.0	64.0	88.0	59.9
	Sequential	41.8	28.0	34.0	52.0	68.0	90.0	52.3
	Interleaved	38.6	70.0	20.0	54.0	64.0	92.0	56.4
Claude Opus 4.7	Vanilla	41.2	86.0	70.0	38.0	68.0	80.0	63.9
ACE	Isolated	48.3	86.0	70.0	32.0	68.0	90.0	65.7
	Sequential	48.5	82.0	68.0	36.0	66.0	86.0	64.4
	Interleaved	47.5	82.0	68.0	26.0	68.0	78.0	61.6
A-Mem	Isolated	46.2	86.0	70.0	28.0	64.0	94.0	64.7
	Sequential	48.0	82.0	68.0	36.0	74.0	98.0	67.7
	Interleaved	49.0	80.0	70.0	30.0	78.0	90.0	66.2
ReasoningBank	Isolated	43.9	84.0	68.0	34.0	68.0	90.0	64.6
	Sequential	40.2	86.0	68.0	32.0	66.0	84.0	62.7
	Interleaved	44.1	84.0	68.0	36.0	68.0	92.0	65.4
AutoSkill	Isolated	42.6	86.0	74.0	36.0	72.0	78.0	64.8
	Sequential	41.2	86.0	74.0	36.0	64.0	78.0	63.2
	Interleaved	42.9	86.0	74.0	40.0	70.0	88.0	66.8
Harness	Isolated	43.8	84.0	70.0	28.0	66.0	88.0	63.3
	Sequential	42.5	84.0	68.0	34.0	72.0	94.0	65.7
	Interleaved	45.9	84.0	68.0	38.0	68.0	82.0	64.3

Table 12: Main results (%) of self-evolving methods across three streaming scenarios under seed 44. Red background indicates scores below the model’s vanilla baseline, and blue marks the best scenario within each self-evolving method.

Methods	Mode	AppWorld	BFCL	BrowseComp+	HLE	SWE	Tau2	Avg
GPT-5.4	Vanilla	44.6	66.0	50.0	2.0	62.0	50.0	45.8
ACE	Isolated	44.8	76.0	48.0	6.0	64.0	62.0	50.1
	Sequential	44.8	68.0	42.0	6.0	62.0	54.0	46.1
	Interleaved	34.5	58.0	46.0	8.0	58.0	48.0	42.1
A-Mem	Isolated	31.4	66.0	50.0	6.0	66.0	54.0	45.6
	Sequential	34.8	62.0	54.0	12.0	62.0	48.0	45.5
	Interleaved	41.2	62.0	54.0	6.0	66.0	94.0	53.9
ReasoningBank	Isolated	39.7	56.0	44.0	12.0	60.0	44.0	42.6
	Sequential	42.3	64.0	50.0	12.0	60.0	24.0	42.1
	Interleaved	36.6	68.0	50.0	14.0	54.0	58.0	46.8
AutoSkill	Isolated	41.4	70.0	46.0	4.0	58.0	52.0	45.2
	Sequential	39.2	70.0	44.0	6.0	58.0	46.0	43.9
	Interleaved	38.7	76.0	44.0	4.0	60.0	44.0	44.5
Harness	Isolated	34.9	54.0	44.0	4.0	64.0	44.0	40.8
	Sequential	33.4	68.0	46.0	6.0	56.0	56.0	44.2
	Interleaved	39.1	60.0	56.0	6.0	56.0	42.0	43.2
Gemini 3.1 Pro	Vanilla	41.8	58.0	34.0	52.0	64.0	90.0	56.6
ACE	Isolated	43.6	66.0	38.0	50.0	64.0	80.0	56.9
	Sequential	44.6	80.0	44.0	52.0	68.0	92.0	63.4
	Interleaved	38.4	52.0	50.0	50.0	66.0	100.0	59.4
A-Mem	Isolated	46.3	26.0	46.0	54.0	64.0	90.0	54.4
	Sequential	40.5	56.0	44.0	50.0	62.0	94.0	57.8
	Interleaved	43.0	38.0	44.0	50.0	62.0	84.0	53.5
ReasoningBank	Isolated	39.2	52.0	52.0	52.0	66.0	94.0	59.2
	Sequential	41.9	56.0	46.0	50.0	62.0	94.0	58.3
	Interleaved	44.5	48.0	56.0	54.0	64.0	96.0	60.4
AutoSkill	Isolated	44.1	60.0	50.0	48.0	70.0	88.0	60.0
	Sequential	44.1	68.0	38.0	46.0	62.0	86.0	57.4
	Interleaved	43.5	66.0	48.0	48.0	68.0	90.0	60.6
Harness	Isolated	36.0	74.0	40.0	56.0	72.0	86.0	60.7
	Sequential	43.0	68.0	38.0	52.0	66.0	96.0	60.5
	Interleaved	41.9	76.0	42.0	50.0	66.0	84.0	60.0
Claude Opus 4.7	Vanilla	41.2	86.0	70.0	38.0	68.0	80.0	63.9
ACE	Isolated	42.9	86.0	74.0	30.0	72.0	96.0	66.8
	Sequential	47.1	80.0	70.0	40.0	74.0	92.0	67.2
	Interleaved	45.9	84.0	68.0	36.0	74.0	76.0	64.0
A-Mem	Isolated	48.8	84.0	72.0	26.0	76.0	82.0	64.8
	Sequential	47.8	82.0	64.0	32.0	68.0	92.0	64.3
	Interleaved	50.3	82.0	66.0	32.0	66.0	78.0	62.4
ReasoningBank	Isolated	43.0	84.0	70.0	36.0	66.0	90.0	64.8
	Sequential	43.0	86.0	74.0	38.0	64.0	86.0	65.2
	Interleaved	45.6	86.0	74.0	42.0	68.0	86.0	66.9
AutoSkill	Isolated	42.8	86.0	74.0	36.0	62.0	84.0	64.1
	Sequential	42.1	86.0	74.0	36.0	62.0	90.0	65.0
	Interleaved	40.8	88.0	74.0	40.0	68.0	84.0	65.8
Harness	Isolated	44.9	82.0	68.0	38.0	74.0	90.0	66.1
	Sequential	41.9	80.0	70.0	36.0	72.0	42.0	57.0
	Interleaved	44.1	86.0	70.0	36.0	72.0	82.0	65.0

Table 13: Main results (%) of self-evolving methods across three streaming scenarios under seed 46. Red background indicates scores below the model’s vanilla baseline, and blue marks the best scenario within each self-evolving method.

Methods	Mode	AppWorld	BFCL	BrowseComp+	HLE	SWE	Tau2	Avg
GPT-5.4	Vanilla	44.6	66.0	50.0	2.0	62.0	50.0	45.8
ACE	Isolated	39.3	74.0	48.0	4.0	58.0	58.0	46.9
	Sequential	38.7	66.0	50.0	8.0	68.0	62.0	48.8
	Interleaved	31.9	56.0	42.0	6.0	68.0	46.0	41.7
A-Mem	Isolated	50.3	70.0	46.0	8.0	64.0	84.0	53.7
	Sequential	43.1	74.0	50.0	8.0	54.0	56.0	47.5
	Interleaved	40.0	66.0	48.0	10.0	60.0	52.0	46.0
ReasoningBank	Isolated	40.0	62.0	44.0	10.0	64.0	36.0	42.7
	Sequential	43.4	64.0	44.0	20.0	68.0	32.0	45.2
	Interleaved	42.6	64.0	52.0	10.0	64.0	52.0	47.4
AutoSkill	Isolated	42.0	70.0	46.0	4.0	50.0	26.0	39.7
	Sequential	39.0	70.0	38.0	8.0	62.0	30.0	41.2
	Interleaved	39.4	74.0	48.0	6.0	62.0	30.0	43.2
Harness	Isolated	39.5	68.0	46.0	4.0	60.0	68.0	47.6
	Sequential	35.6	64.0	44.0	6.0	64.0	58.0	45.3
	Interleaved	31.4	70.0	48.0	6.0	56.0	52.0	43.9
Gemini 3.1 Pro	Vanilla	41.8	58.0	34.0	52.0	64.0	90.0	56.6
ACE	Isolated	44.9	48.0	48.0	50.0	64.0	98.0	58.8
	Sequential	38.2	62.0	38.0	52.0	68.0	98.0	59.4
	Interleaved	42.1	66.0	44.0	50.0	64.0	96.0	60.4
A-Mem	Isolated	43.7	74.0	50.0	52.0	56.0	98.0	62.3
	Sequential	43.8	56.0	48.0	52.0	60.0	90.0	58.3
	Interleaved	40.2	70.0	42.0	52.0	58.0	96.0	59.7
ReasoningBank	Isolated	44.7	52.0	54.0	44.0	58.0	98.0	58.5
	Sequential	42.8	54.0	44.0	52.0	72.0	98.0	60.5
	Interleaved	43.0	52.0	48.0	54.0	70.0	96.0	60.5
AutoSkill	Isolated	40.4	62.0	42.0	54.0	60.0	84.0	57.1
	Sequential	47.0	58.0	40.0	52.0	60.0	82.0	56.5
	Interleaved	44.9	66.0	36.0	50.0	64.0	86.0	57.8
Harness	Isolated	38.3	78.0	48.0	50.0	68.0	96.0	63.1
	Sequential	41.0	68.0	34.0	48.0	60.0	88.0	56.5
	Interleaved	42.3	78.0	52.0	50.0	60.0	96.0	63.1
Claude Opus 4.7	Vanilla	41.2	86.0	70.0	38.0	68.0	80.0	63.9
ACE	Isolated	51.2	84.0	72.0	38.0	72.0	94.0	68.5
	Sequential	46.9	86.0	68.0	40.0	72.0	96.0	68.2
	Interleaved	46.6	76.0	68.0	40.0	68.0	62.0	60.1
A-Mem	Isolated	51.2	86.0	72.0	28.0	74.0	96.0	67.9
	Sequential	48.6	84.0	72.0	30.0	66.0	98.0	66.4
	Interleaved	48.7	82.0	66.0	34.0	74.0	90.0	65.8
ReasoningBank	Isolated	45.1	84.0	72.0	38.0	74.0	90.0	67.2
	Sequential	42.1	84.0	72.0	36.0	70.0	86.0	65.0
	Interleaved	45.7	84.0	68.0	40.0	70.0	88.0	66.0
AutoSkill	Isolated	40.4	86.0	72.0	38.0	70.0	84.0	65.1
	Sequential	42.1	86.0	70.0	40.0	64.0	92.0	65.7
	Interleaved	38.1	86.0	72.0	34.0	64.0	88.0	63.7
Harness	Isolated	45.8	86.0	68.0	36.0	68.0	94.0	66.3
	Sequential	43.5	84.0	76.0	38.0	72.0	86.0	66.6
	Interleaved	48.2	86.0	70.0	38.0	68.0	98.0	68.0

Appendix D Cumulative Accuracy Dynamics

To characterize the dynamics of self-evolution, we report the cumulative accuracy over three streaming scenarios for each self-evolving method and model in Figures 5 to 13.

Appendix E Method Prompts

For reproducibility, we provide the full set of prompts used by self-evolving methods. All prompts are adapted to our test-time setting, where no ground-truth labels are available, and to the Exgentic [92] framework, in which the agent completes tasks through multi-step tool use rather than single-turn question answering.

Figure 5: Cumulative accuracy dynamics of ACE on GPT-5.4 over three streaming scenarios.

Figure 6: Cumulative accuracy dynamics of ACE on Gemini 3.1 Pro over three streaming scenarios.

Figure 7: Cumulative accuracy dynamics of ACE on Claude Opus 4.7 over three streaming scenarios.

Figure 8: Cumulative accuracy dynamics of A-Mem on GPT-5.4 over three streaming scenarios.

Figure 9: Cumulative accuracy dynamics of A-Mem on Gemini 3.1 Pro over three streaming scenarios.

Figure 10: Cumulative accuracy dynamics of A-Mem on Claude Opus 4.7 over three streaming scenarios.

Figure 11: Cumulative accuracy dynamics of ReasoningBank on GPT-5.4 over three streaming scenarios.

Figure 12: Cumulative accuracy dynamics of ReasoningBank on Gemini 3.1 Pro over three streaming scenarios.

Figure 13: Cumulative accuracy dynamics of ReasoningBank on Claude Opus 4.7 over three streaming scenarios.

Figure 14: Cumulative accuracy dynamics of AutoSkill on GPT-5.4 over three streaming scenarios.

Figure 15: Cumulative accuracy dynamics of AutoSkill on Gemini 3.1 Pro over three streaming scenarios.

Figure 16: Cumulative accuracy dynamics of AutoSkill on Claude Opus 4.7 over three streaming scenarios.

Figure 17: Cumulative accuracy dynamics of Harness on GPT-5.4 over three streaming scenarios.

Figure 18: Cumulative accuracy dynamics of Harness on Gemini 3.1 Pro over three streaming scenarios.

Figure 19: Cumulative accuracy dynamics of Harness on Claude Opus 4.7 over three streaming scenarios.