Bilevel Coordinated Reflection: A Game-Theoretic Approach to Multi-Agent LLM Systems

Yihang Chen ^†^†thanks: Equal contribution. Yuxiang Chen¹¹footnotemark: 1 Yuxuan Huang Meng Fang Weilin Luo Jun Wang ^†^†thanks: Corresponding author.

Abstract

Multi-agent LLM systems commonly use an orchestrator to decompose a task for a team of workers and then improve through textual reflection. Despite strong empirical results, these systems lack a unified account of coordination, memory improvement, and the role of external verification. We model orchestrator–worker interaction as a bilevel coordination game: under bounded coupling, the workers’ local-update game is an approximate potential game whose equilibrium slack is controlled by decomposition quality. We then analyse reflection as stochastic movement over semantic memory states. For free-form reflection, we derive a finite-time upper bound, prove worst-case tightness, and give a positive lower bound under a falsifiable persistent-harm condition. We further prove an information-theoretic impossibility result: no gate that observes only the generated transcript can improve uniformly over text-indistinguishable environments, whereas an environment-grounded gate can. Motivated by this separation, we introduce Stochastic Reflective Memory Ascent (SRMA), which accepts a candidate memory only after a grounded evaluation risk strictly decreases. Under calibration and non-degenerate corrective mass, SRMA converges exactly, geometrically or polynomially; matching constructions show that both rate regimes are order-tight. We also provide confidence gating for stochastic evaluation and re-anchoring guarantees for piecewise-stationary environments. Experiments instantiate these objects with environment-grounded metrics and test the predicted coordination and drift laws. On 500 SWE-bench instances, the complete Kimi-based system resolves $72.2\%$ versus a $70.8\%$ public mini-SWE-agent reference. Code available at https://github.com/YihangChen9/Bilevel-Coordinated-Reflection.

¹UCL Centre for Artificial Intelligence ²University of Liverpool ³Huawei

Refer to caption — Figure 1: Bilevel coordinated reflection. The orchestrator (leader) selects a decomposition $\tau$ and updates strategy memory $m_{o}$ on the slower timescale; workers (followers) update execution memory $m_{e}$ via $\eta_{c}$ -better responses on the faster timescale. Under bounded coupling, the followers’ subgame is an approximate potential game with slack $\eta_{c}\leq 2d_{\max}\kappa$ , while verifier-gated SRMA separately governs which memory proposals are committed.

1 Introduction

Multi-agent LLM systems have become a common recipe for tasks too large or structured for a single agent: an orchestrator decomposes the task, worker models solve the pieces, and the team improves by reflecting—writing critiques, hypotheses, and lessons into a shared textual memory that conditions subsequent generations (Wu et al. 2024; Hong et al. 2024; Shinn et al. 2023; Benkovich and Valkov 2026; Qian et al. 2025). Because model weights are frozen at test time, memory editing is the principal adaptation channel (Zhou et al. 2025; Xu et al. 2025; Zhang et al. 2025b), and such loops often work better when grounded by a test harness, simulator, execution engine, or formal checker.

The dominant account of these systems is nevertheless procedural. Existing frameworks (Zhang et al. 2025a; Hu et al. 2025; Dang et al. 2025; Wang et al. 2025) specify who communicates with whom and which buffer is updated, but not the strategic object that the agents stabilise to or the quantity that reflection improves. This leaves three unresolved questions. First, how does the orchestrator’s decomposition quality control worker coordination? Second, when does unconditional reflection plateau rather than converge? Third, why can an external verifier succeed where a stronger text-only critic may still fail?

We address these questions in a single framework. The orchestrator–worker pipeline is modelled as a bilevel coordination game whose follower subgame is an approximate potential game, and textual memory editing as a stochastic process over a discrete semantic state space. For free-form reflection, a one-sided drift condition yields a finite-time upper bound that is tight in the worst case; a universal positive floor requires an additional, explicitly testable persistent-harm condition—unconditional commitment alone is not enough.

We then isolate the informational role of verification: in two environments with identical text-generation laws but opposite meanings for the same reflections, any possibly randomised, history-dependent gate that observes only the transcript behaves identically and therefore cannot improve both—even an ideal text-only judge—whereas a grounded verifier distinguishes the pair and recovers geometric convergence.

Motivated by this separation, we introduce Stochastic Reflective Memory Ascent (SRMA), which commits a candidate memory only when a fixed grounded evaluation protocol certifies a strict decrease in verifier risk. Under calibration and non-degenerate corrective mass, SRMA converges exactly at order-tight geometric or polynomial rates; a confidence gate handles stochastic probes, and re-anchoring restores per-segment convergence under piecewise stationarity.

The theory is instantiated on a hidden-cap resource contest, Overcooked with an exact BFS value table, and SWE-bench (Jimenez et al. 2024). The controlled environments expose the strategic, memory, and drift quantities directly without an LLM-as-judge; on SWE-bench the complete Kimi-based system resolves $361/500$ instances ( $72.2\%$ ) versus $70.8\%$ for the public mini-SWE-agent v2 reference.

In summary, we contribute: (1) a bilevel coordination game linking decomposition coupling to follower equilibrium slack (Sec. 3.1); (2) a two-sided drift analysis of free-form reflection, tight in the worst case, with a universal lower bound under persistent harmful commitment (Sec. 3.2); (3) an impossibility theorem for self-contained text-only gates, with a grounded comparator that converges geometrically (Sec. 3.3); (4) SRMA, with exact convergence, order-tight rates, and a finite-probe confidence extension (Sec. 3.4); and (5) mechanism-level validation on Resource Contest and Overcooked plus end-to-end results on SWE-bench (Sec. 4).

2 Related Work

Multi-agent LLM frameworks. Orchestrator–worker architectures such as AutoGen (Wu et al. 2024), MetaGPT (Hong et al. 2024) and Agyn (Benkovich and Valkov 2026) show strong empirical performance but offer no convergence analysis; failure modes such as hallucination cascades are documented empirically (Liu et al. 2026; Cemri et al. 2025). We provide the missing game-theoretic and stochastic-approximation foundations.

Self-reflection, self-evaluation, and grounding. Reflexion (Shinn et al. 2023) and Self-Refine (Madaan et al. 2023) improve outputs by appending self-generated critiques but may plateau, and correlated self-evaluation bias (Zheng et al. 2023; Panickssery et al. 2024; Wu et al. 2026) weakens model-based judges in practice. Our drift analysis separates a worst-case floor from the persistent-harm condition needed for a universal lower bound, and our indistinguishable-environment theorem shows that without an environment-dependent signal even an ideal text-only gate cannot be uniformly correct.

Potential games and drift analysis. Our followers’ subgame builds on exact and approximate potential games (Monderer and Shapley 1996; Candogan et al. 2011; Christodoulou and Gairing 2014) and weakly coupled team problems (Srikant and Başar 1992). The convergence analysis uses Foster–Lyapunov drift (Hajek 1982; Meyn and Tweedie 2009), classical stochastic approximation (Robbins and Monro 1951; Borkar 2008; Bertsekas and Tsitsiklis 2000) and, for the gated regime, multiplicative and variable drift theorems from randomised search heuristics (Doerr et al. 2012; Johannsen 2010; Lehre and Witt 2021); the recursion $e_{t+1}\leq e_{t}-c\,e_{t}^{1+\beta}$ is the discrete stochastic analogue of Polyak–Łojasiewicz-type conditions (Karimi et al. 2016; Chung 1954). Two-timescale bilevel structure follows Borkar (1997); Hong et al. (2023).

3 Methodology

Longer derivations are deferred to the supplementary material.

3.1 Problem Formulation: Bilevel Coordination Game

We formalise the resolution of a complex user query $q\in\mathcal{Q}$ . The objective is a joint structured output $x\in\mathcal{X}$ maximising a global utility $U(x)$ (logical correctness, constraint satisfaction). In a naive single-agent paradigm the entire output is generated directly from the query via the frozen LLM kernel, $x\sim p_{\text{LLM}}(\cdot\mid q)$ , which for large tasks induces context dilution and reasoning degradation (Liu et al. 2024; Levy et al. 2024; Du et al. 2025). Contemporary systems instead let an orchestrator partition the task among workers (Wu et al. 2024; Hong et al. 2024; Liu et al. 2025).

We model this as a bilevel coordination game. The orchestrator (Leader) generates a strategy profile $\tau=(\tau_{1},\ldots,\tau_{N})\in\mathcal{T}$ ,

\tau\sim p_{\text{LLM}}(\cdot\mid q),

(1)

assigning subtask $\tau_{i}$ to worker $i$ (Follower), who generates a local sub-solution $x_{i}\sim p_{\text{LLM}}(\cdot\mid\tau_{i})$ ; the global output is $x=(x_{1},\ldots,x_{N})$ .

Unlike the idealised independent decomposition of classical potential-game analyses (Monderer and Shapley 1996), real multi-agent LLM systems exhibit non-trivial cross-worker interactions: shared variables, common interfaces, joint constraints (Liu et al. 2026). We adopt a weakly coupled decomposition in the spirit of Srikant and Başar (1992); Candogan et al. (2011).

Assumption 1 (Weakly Coupled Decomposability).

Each worker action set $\mathcal{X}_{i}$ is finite. The worker payoff is the local objective $g_{i}(x;\tau):=u_{i}(x_{i}\mid\tau_{i})$ , while the system-level objective is $U$ . The global utility admits

	$\displaystyle U(x)={}$	$\displaystyle\sum_{i=1}^{N}u_{i}(x_{i}\mid\tau_{i})$
		$\displaystyle+\sum_{(i,j)\in\mathcal{E}}\psi_{ij}(x_{i},x_{j}\mid\tau_{i},\tau_{j}),$		(2)

where $\mathcal{E}$ is an undirected interaction graph induced by $\tau$ , with each edge counted once, and $\kappa:=\sup_{(i,j)\in\mathcal{E}}\sup_{x_{i},x_{j}}|\psi_{ij}(x_{i},x_{j}\mid\tau_{i},\tau_{j})|<\infty$ . Let $\mathcal{N}_{i}$ be worker $i$ ’s coupled neighbours and $d_{\max}:=\max_{i}|\mathcal{N}_{i}|$ .

When $\kappa=0$ the system reduces to the independent case; $\kappa$ and $d_{\max}$ jointly quantify decomposition quality. Since LLM generation is stochastic, the system objective is the expected global utility $\mathbb{E}[U(x)]$ .

Lemma 1 (Approximate Potential Game).

Under Assumption 1 and fixed $\tau$ , the workers’ subgame is an $\eta_{c}$ -approximate potential game with potential $\mathbb{E}[U(x)]$ and slack

\eta_{c}\;\leq\;2\,d_{\max}\,\kappa.

(3)

Proof.

If worker $i$ unilaterally deviates from $x_{i}^{t}$ to $x_{i}^{t+1}$ ,

		$\displaystyle\mathbb{E}[U(x_{i}^{t+1},x_{-i}^{t})]-\mathbb{E}[U(x_{i}^{t},x_{-i}^{t})]$
		$\displaystyle\quad=\mathbb{E}[u_{i}(x_{i}^{t+1}\mid\tau_{i})]-\mathbb{E}[u_{i}(x_{i}^{t}\mid\tau_{i})]+\Delta_{i}^{\psi},$		(4)

where the coupling residual $\Delta_{i}^{\psi}$ sums at most $d_{\max}$ terms each bounded by $2\kappa$ (since $|\psi_{ij}|\leq\kappa$ ), so $|\Delta_{i}^{\psi}|\leq 2d_{\max}\kappa=:\eta_{c}$ . Every unilateral deviation thus changes the potential within $\eta_{c}$ of the local utility change (Candogan et al. 2011; Christodoulou and Gairing 2014). ∎

A rational worker performs $\eta_{c}$ -better-response updates: $\mathbb{E}[u_{i}(x_{i}^{t+1}\mid\tau_{i})]-\mathbb{E}[u_{i}(x_{i}^{t}\mid\tau_{i})]>\eta_{c}$ . If no worker has such a deviation, the current profile is by definition already an $\eta_{c}$ -approximate Nash equilibrium, so the dynamics below are well defined in all cases.

Theorem 1 (Convergence of the Followers’ Subgame).

Under Lemma 1, iterated $\eta_{c}$ -better-response updates converge in finitely many steps to a profile $x^{*}(\tau)$ satisfying, for every worker $i$ ,

\mathbb{E}[g_{i}(x_{i}^{*},x_{-i}^{*};\tau)]\geq\max_{x_{i}^{\prime}\in\mathcal{X}_{i}}\mathbb{E}[g_{i}(x_{i}^{\prime},x_{-i}^{*};\tau)]-\eta_{c}.

(5)

Thus $x^{*}(\tau)$ is an $\eta_{c}$ -approximate pure-strategy Nash equilibrium of the explicitly defined local-payoff game.

Proof sketch.

Each update raises the potential $\mathbb{E}[U(x^{t})]$ by a strictly positive amount (Lemma 1); $\mathcal{X}$ finite and $U$ bounded imply finite termination. Full proof in the supplementary material. ∎

Leader’s objective and decomposition quality.

The orchestrator anticipates the followers’ equilibrium and solves $\tau^{*}(q)\in\arg\max_{\tau}\mathbb{E}[U(x^{*}(\tau))]$ . Because $\eta_{c}=2d_{\max}(\tau)\kappa(\tau)$ depends on $\tau$ , the leader’s objective contains an explicit decomposition-quality term:

Corollary 1 (Leader’s Decomposition Trade-off).

Let $J_{\mathrm{loc}}(\tau):=\sum_{i}\max_{x_{i}}\mathbb{E}[u_{i}(x_{i}\mid\tau_{i})]$ and $C(\tau):=d_{\max}(\tau)\,\kappa(\tau)$ . For any $\eta_{c}$ -approximate equilibrium $x^{*}(\tau)$ ,

\mathbb{E}[U(x^{*}(\tau))]\;\geq\;J_{\mathrm{loc}}(\tau)\;-\;\tfrac{5}{2}\,N\,C(\tau).

(6)

Hence the leader maximises a lower bound that trades achievable local utility against coupling: a good decomposition simultaneously raises $J_{\mathrm{loc}}$ and shrinks $C(\tau)$ . (Proof in the supplementary material.)

3.2 Dual-Memory Drift Dynamics and Hallucination Floors

LLM weights are frozen, so adaptation proceeds by editing external, non-parametric memories: an execution memory $m_{e}\in\mathcal{M}_{e}$ shared by workers and a strategy memory $m_{o}\in\mathcal{M}_{o}$ used by the orchestrator. For a fixed decomposition $\tau$ , let

J_{i}(m_{e}\mid\tau_{i})=\mathbb{E}_{x_{i}\sim p_{\text{LLM}}(\cdot\mid\tau_{i},m_{e})}[u_{i}(x_{i}\mid\tau_{i})]

and rescale utility so that the sub-optimality $V_{t}:=V_{i}(m_{e}^{t})=J_{i}^{*}-J_{i}(m_{e}^{t}\mid\tau_{i})$ lies in $[0,1]$ . Let $\mathcal{F}_{t}$ denote the history up to the $t$ -th memory update.

The key distinction is whether a proposed reflection is committed unconditionally or evaluated before it enters memory. Unconditional commitment alone does not imply a positive asymptotic error: a universal lower bound requires an explicit condition that harmful commitments keep injecting non-vanishing expected error. We therefore separate an upper guarantee, its worst-case tightness, and a genuine lower bound under persistent harmful drift.

Regime A: free-form reflection.

When every generated reflection is appended, corrective information and hallucinated information (Huang et al. 2025; Ji et al. 2024) are mixed in the same update. We summarise their net conditional effect by the following one-sided drift condition.

Assumption 2 (One-Sided Free-Form Drift).

There exist $\gamma_{t}\in(0,1]$ and $\nu_{t}\geq 0$ such that

\mathbb{E}[V_{t+1}\mid\mathcal{F}_{t}]\leq(1-\gamma_{t})V_{t}+\nu_{t}.

(7)

Here $\gamma_{t}V_{t}$ is the available corrective drift and $\nu_{t}$ is the mean residual error load from committed, ungrounded content; $\nu_{t}$ is a first-moment quantity, not a variance.

Theorem 2 (Finite-Time Upper Bound).

If $\gamma_{t}\geq\underline{\gamma}>0$ and $\nu_{t}\leq\overline{\nu}$ , then, for $e_{t}:=\mathbb{E}[V_{t}]$ ,

e_{T}\leq(1-\underline{\gamma})^{T}e_{0}+\frac{\overline{\nu}}{\underline{\gamma}}\bigl(1-(1-\underline{\gamma})^{T}\bigr).

(8)

Consequently, $\limsup_{T\to\infty}e_{T}\leq\min\{1,\overline{\nu}/\underline{\gamma}\}$ . (Proof in the supplementary material.)

Theorem 2 is an upper guarantee only; the next result is the strongest conclusion available from Assumption 2 alone.

Proposition 1 (Worst-Case Tightness).

For every $\gamma\in(0,1]$ and $\nu\in(0,\gamma]$ , there exists a free-form process satisfying Assumption 2 with $\gamma_{t}\equiv\gamma$ and $\nu_{t}\equiv\nu$ such that

\lim_{T\to\infty}\mathbb{E}[V_{T}]=\frac{\nu}{\gamma}.

(9)

Hence the upper bound $\nu/\gamma$ cannot be uniformly improved over the one-sided drift class.

Proof sketch.

The deterministic recursion $V_{t+1}=(1-\gamma)V_{t}+\nu$ with $0<\nu\leq\gamma$ maps $[0,1]$ into itself, attains (7) with equality, and converges to its unique fixed point $\nu/\gamma$ . ∎

A lower bound that applies to every process requires a lower drift condition, directly testable by regressing the next-step error on the current error in free-form trajectories.

Assumption 3 (Persistent Harmful Commitment).

There exist $\overline{\gamma}\in(0,1]$ and $\underline{\nu}\in(0,\overline{\gamma}]$ such that, on every reachable state,

\mathbb{E}[V_{t+1}\mid\mathcal{F}_{t}]\geq(1-\overline{\gamma})V_{t}+\underline{\nu}.

(10)

The parameter $\overline{\gamma}$ upper-bounds how much of the current error can be removed in one expected update, whereas $\underline{\nu}>0$ is a persistent net error load that remains because harmful reflections are committed without screening.

Theorem 3 (Universal Lower Bound).

Under Assumption 3,

e_{T}\geq(1-\overline{\gamma})^{T}e_{0}+\frac{\underline{\nu}}{\overline{\gamma}}\bigl(1-(1-\overline{\gamma})^{T}\bigr),

(11)

and therefore

\liminf_{T\to\infty}e_{T}\geq\frac{\underline{\nu}}{\overline{\gamma}}>0.

(12)

(Proof in the supplementary material.)

Corollary 2 (Two-Sided Error Tube).

If Assumptions 2 and 3 both hold, then

\frac{\underline{\nu}}{\overline{\gamma}}\leq\liminf_{T\to\infty}e_{T}\leq\limsup_{T\to\infty}e_{T}\leq\frac{\overline{\nu}}{\underline{\gamma}}.

(13)

When the two conditional drift bounds match, $\underline{\gamma}=\overline{\gamma}=\gamma$ and $\underline{\nu}=\overline{\nu}=\nu$ , the mean error converges exactly to $\nu/\gamma$ .

An operational corollary in the supplementary material re-expresses the tube via estimable per-step correction and harm rates.

Leader’s outer loop.

On the slower timescale, define $V_{o}(m_{o}^{k})=\Phi^{*}-\Phi(m_{o}^{k})$ for $\Phi(m_{o}^{k})=\mathbb{E}[U(x)\mid m_{e}^{\infty}(\tau(m_{o}^{k}))]$ . Under the analogous upper drift condition with $\gamma_{o}^{k}\geq\gamma_{o,\min}>0$ and $\nu_{o,k}\leq\nu_{o,\max}$ , the same affine recursion yields the finite-episode bound $\mathbb{E}[V_{o}(m_{o}^{K})]\leq(1-\gamma_{o,\min})^{K}V_{o}(m_{o}^{0})+\nu_{o,\max}/\gamma_{o,\min}$ . We use only this finite-episode statement and make no asymptotic leader-regret claim.

3.3 Why Grounding Is Necessary: Impossibility of Self-Contained Gates

The fundamental informational requirement is grounding: access to a signal whose law depends on the environment rather than only on the generated transcript. We formalise this through a pair of environments that are indistinguishable at the text level.

Text processes and gates.

Let a memory be a finite reflection sequence $m=(c_{1},\ldots,c_{t})\in\mathcal{C}^{*}$ with append operation $m\oplus c=(c_{1},\ldots,c_{t},c)$ . Fix an initial memory $m^{0}$ and a proposal kernel $P(\cdot\mid m)$ over $\mathcal{C}$ . An environment $\varepsilon$ assigns a sub-optimality $V^{\varepsilon}(m)\in[0,1]$ to every reachable memory. Across the class considered below, the environment changes this semantic value but not the proposal kernel or any other text-level law.

Definition 1 (Self-Contained Gate).

A self-contained gate is any possibly randomised, history-dependent acceptance rule measurable with respect to the generated text process and its internal randomness only. A grounded gate may additionally observe an environment-dependent signal, such as realised reward, simulator state, test execution, or a formal-checker result.

This class contains text-only LLM-as-judge systems. Correlated-evaluation bias can make such judges weaker in practice (Panickssery et al. 2024); the result below applies even to an ideal gate with unlimited text-processing capacity.

Ambiguous-pair construction.

Let $C_{0},C_{1}\subset\mathcal{C}$ be disjoint and satisfy $P(C_{0}\mid m)=P(C_{1}\mid m)=\mu\in(0,\tfrac{1}{2}]$ for every reachable $m$ ; all remaining proposals are inert. Fix $\kappa\in(0,1)$ and define $f_{0}(v)=(1-\kappa)v$ and $f_{1}(v)=\kappa+(1-\kappa)v$ . In environment $\varepsilon^{+}$ , an accepted proposal from $C_{a}$ applies $f_{a}$ to the current error; in environment $\varepsilon^{-}$ , the roles of $C_{0}$ and $C_{1}$ are swapped. Thus the same text is corrective in one environment and harmful in the other. Let $e_{T}^{\varepsilon}=\mathbb{E}[V^{\varepsilon}(m^{T})]$ and assume both environments start at the same $e_{0}\leq\tfrac{1}{2}$ .

Theorem 4 (Self-Gating Impossibility).

For every self-contained gate and every horizon $T$ ,

\max\{e_{T}^{\varepsilon^{+}},e_{T}^{\varepsilon^{-}}\}\geq e_{0}.

(14)

Moreover, if $e_{0}<\tfrac{1}{2}$ and the gate accepts at least one proposal from $C_{0}\cup C_{1}$ with positive probability by time $T$ , then the inequality is strict. In contrast, the free-form rule accepts everything and satisfies $e_{T}^{\varepsilon}\to\tfrac{1}{2}$ in both environments, whereas the grounded gate that observes $V^{\varepsilon}$ accepts only the corrective class and satisfies $e_{T}^{\varepsilon}=e_{0}(1-\kappa\mu)^{T}\to 0$ in both environments.

Proof sketch.

Couple both environments with shared proposal and gate randomness; the accepted class-label sequence is then identical under $\varepsilon^{+}$ and $\varepsilon^{-}$ , and the reflection identity $f_{1-a}(v)=1-f_{a}(1-v)$ yields $e_{T}^{\varepsilon^{+}}+e_{T}^{\varepsilon^{-}}\geq 2e_{0}$ , strictly when $e_{0}<\tfrac{1}{2}$ and an ambiguous proposal is accepted with positive probability. The free-form and grounded rates follow from the induced affine recursions. Full proof in the supplementary material. ∎

Remark 1 (Scope of the impossibility result).

The theorem is minimax over text-indistinguishable environments; textual self-evaluation remains useful when the transcript itself certifies correctness (a fully checkable proof). When truth depends on external state—hidden caps, API responses, simulator state, an evolving repository—judge capacity cannot substitute for grounding.

3.4 SRMA: Verifier-Gated Reflection

Theorem 4 establishes why the gate must have access to an environment-separating signal. Exact convergence additionally requires the gate to compare a fixed error functional of the memory state, rather than two uncontrolled one-shot samples from a stochastic generator. We therefore separate the stochastic proposal mechanism from the grounded evaluation protocol.

Definition 2 (Verifier and Evaluation Risk).

A verifier is a deterministic map $\mathcal{V}:\mathcal{X}\times\mathcal{T}\to\mathcal{S}$ with deterministic score $\rho:\mathcal{S}\to[0,1]$ . Let $g_{i}:\mathcal{T}_{i}\times\mathcal{M}_{e}\to\mathcal{X}_{i}$ be a fixed deterministic evaluation protocol, such as an exact planner or decoding with fixed randomness. The verifier risk of memory $m_{e}$ is

R_{i}(m_{e}):=\rho\!\left(\mathcal{V}(g_{i}(\tau_{i},m_{e}),\tau_{i})\right).

(15)

The pair $(\mathcal{V},g_{i})$ is fixed independently of the reflection proposal distribution. It is grounded when its score depends on an environment signal that is not determined by the generated transcript alone. Grounding supplies information; calibration below connects the score to task utility.

Definition 3 (Verifier-Gated SRMA Update).

Given $m_{e}^{t}$ , compute the evaluation output $x_{i}^{t}=g_{i}(\tau_{i},m_{e}^{t})$ and diagnostic $s_{i}^{t}=\mathcal{V}(x_{i}^{t},\tau_{i})$ . Sample a reflection $c_{i}^{t+1}\sim p_{\text{LLM}}(\cdot\mid x_{i}^{t},s_{i}^{t},\tau_{i},m_{e}^{t})$ and form $\widetilde{m}_{e}^{t+1}=\mathcal{M}_{e}(m_{e}^{t},c_{i}^{t+1})$ . Accept iff

R_{i}(\widetilde{m}_{e}^{t+1})<R_{i}(m_{e}^{t}).

(16)

On acceptance set $m_{e}^{t+1}=\widetilde{m}_{e}^{t+1}$ ; otherwise retain $m_{e}^{t+1}=m_{e}^{t}$ .

Gating on two stochastic one-shot outputs would not suffice: sample variation could accept a memory with worse expected performance. Exact guarantees therefore assume deterministic or exact expected-risk evaluation; a finite-sample extension follows below.

Assumption 4 (Verifier Calibration).

There exists $L<\infty$ such that, for every reachable memory,

V_{i}(m_{e})\leq LR_{i}(m_{e}).

(17)

Thus zero verifier risk certifies zero task sub-optimality. This assumption is appropriate for exact value tables and complete formal checkers; on incomplete test suites, our theorem concerns verifier risk only.

Assumption 5 (Non-Degenerate Corrective Mass).

There exist $c_{1}\in(0,1]$ and $\beta\in[0,1]$ such that, whenever $R_{t}:=R_{i}(m_{e}^{t})>0$ ,

p_{t}:=\Pr[\mathrm{accept}\mid\mathcal{F}_{t}]\geq c_{1}R_{t}^{\beta}.

(18)

Assumption 6 (Proportional Accepted Decrement).

There exists $c_{2}\in(0,1]$ such that

\mathbb{E}[R_{t}-R_{t+1}\mid\mathcal{F}_{t},\mathrm{accept}]\geq c_{2}R_{t}.

(19)

Proposition 2 (Monotone Multiplicative Drift).

Under Definition 3, $R_{t+1}\leq R_{t}$ almost surely and

	$\displaystyle\mathbb{E}[R_{t+1}\mid\mathcal{F}_{t}]=R_{t}-p_{t}\Delta_{t},$		(20)
	$\displaystyle\Delta_{t}:=\mathbb{E}[R_{t}-R_{t+1}\mid\mathcal{F}_{t},\mathrm{accept}].$

Under Assumptions 5–6, with $c=c_{1}c_{2}$ ,

\mathbb{E}[R_{t+1}\mid\mathcal{F}_{t}]\leq R_{t}-cR_{t}^{1+\beta}.

(21)

Theorem 5 (Exact Verifier Convergence and Rates).

Under Assumptions 5–6, let $r_{t}:=\mathbb{E}[R_{t}]$ and $c=c_{1}c_{2}$ . Then $R_{t}\to 0$ almost surely and

	$\displaystyle\beta=0:\quad$	$\displaystyle r_{T}\leq(1-c)^{T}r_{0},$		$\displaystyle\text{(geometric)},$		(22)
	$\displaystyle\beta\in(0,1]:\quad$	$\displaystyle r_{T}\leq\bigl(r_{0}^{-\beta}+c\beta T\bigr)^{-1/\beta},$		$\displaystyle\text{(polynomial)}.$		(23)

If Assumption 4 also holds, then $\mathbb{E}[V_{i}(m_{e}^{T})]\leq Lr_{T}$ and hence the task sub-optimality converges to zero at the same rate up to the factor $L$ .

Proof sketch.

Taking expectations in (21) and applying Jensen’s inequality to $z\mapsto z^{1+\beta}$ gives $r_{t+1}\leq r_{t}-cr_{t}^{1+\beta}$ ; the rates follow by the standard multiplicative/variable-drift comparison. $R_{t}$ is non-increasing and non-negative, hence converges almost surely, and $r_{t}\to 0$ forces the limit to be zero. Calibration transfers the bound to the utility gap. ∎

Proposition 3 (Rate Tightness for Verifier-Gated Reflection).

For every $c_{1}\in(0,1]$ , $c_{2}\in(0,\tfrac{1}{2}]$ , $\beta\in(0,1]$ , and $r_{0}\in(0,1]$ , there exists a process satisfying Assumptions 5–6 with equality such that, for $c=c_{1}c_{2}$ ,

\mathbb{E}[R_{T}]\geq\bigl(r_{0}^{-\beta}+4c\beta T\bigr)^{-1/\beta}\qquad\text{for every }T.

(24)

For $\beta=0$ , the analogous construction gives $\mathbb{E}[R_{T}]=(1-c)^{T}r_{0}$ exactly. Hence the geometric rate is exact and the polynomial exponent $T^{-1/\beta}$ is tight up to a constant factor in the time scale.

Proof sketch.

Accept with probability $c_{1}R_{t}^{\beta}$ and set $R_{t+1}=(1-c_{2})R_{t}$ on acceptance, so both assumptions hold with equality. For $\beta>0$ , $Y_{t}=R_{t}^{-\beta}$ has constant expected increment, and convexity of $y\mapsto y^{-1/\beta}$ gives (24); for $\beta=0$ , $\mathbb{E}[R_{t+1}\mid\mathcal{F}_{t}]=(1-c_{1}c_{2})R_{t}$ . Full proof in the supplementary material. ∎

Proposition 4 (Confidence-Gated Stochastic Evaluation).

Suppose deterministic $R_{i}(m_{e})$ is unavailable and instead $R_{i}(m_{e})=\mathbb{E}[Z(m_{e})]$ for an i.i.d. score $Z(m_{e})\in[0,1]$ . At round $t$ , estimate the current and candidate risks with $K_{t}$ independent probes and let

a_{t}=\sqrt{\frac{\log(4/\delta_{t})}{2K_{t}}}.

(25)

Accept only when $\widehat{R}_{t}(\widetilde{m}_{e}^{t+1})+a_{t}<\widehat{R}_{t}(m_{e}^{t})-a_{t}$ . Then, with probability at least $1-\sum_{t}\delta_{t}$ , every accepted update strictly decreases the true expected verifier risk.

Proof sketch.

Hoeffding’s inequality bounds each of the two estimation errors by $a_{t}$ with joint failure probability at most $\delta_{t}$ ; a union bound over rounds completes the argument. Exact convergence requires deterministic or exact expected-risk evaluation, or $K_{t}\to\infty$ with summable $\delta_{t}$ . ∎

Proposition 5 (Piecewise-Stationary Re-Anchoring).

Suppose the verifier risk changes finitely many times, with final change at $t_{S}$ , and both current and candidate memories are re-evaluated under the current risk. If Assumptions 5–6 hold on the final stationary segment, then Theorem 5 applies with horizon $T-t_{S}$ and initial risk $r_{t_{S}}=\mathbb{E}[R_{i}^{(t_{S})}(m_{e}^{t_{S}})]$ .

Remark 2 (Falsifiability and rate prediction).

The exponent $\beta$ is observable: the geometric regime is linear in $\log R$ versus $t$ , the polynomial regime in $\log R$ versus $\log t$ with slope $-1/\beta$ ; estimating $\beta$ from acceptance frequencies and from trajectory decay gives the closed-loop calibration of Sec. 4. The free-form drift parameters are likewise estimable from conditional drift regressions.

Practical realisation.

Algorithm 1 probes the candidate under the same fixed protocol and commits only a strict improvement; recomputing the current risk enables the re-anchoring of Proposition 5, and under stochastic evaluation line 8 is replaced by the test of Proposition 4.

Algorithm 1 Stochastic Reflective Memory Ascent for worker

i

0: subtask

\tau_{i}

; verifier

(\mathcal{V},\rho)

; evaluation protocol

g_{i}

; memory operator

\mathcal{M}_{e}

; initial memory

m_{e}^{0}

; budget

T

1: for

t=0

T-1

x_{i}^{t}\leftarrow g_{i}(\tau_{i},m_{e}^{t})

s_{i}^{t}\leftarrow\mathcal{V}(x_{i}^{t},\tau_{i})

;

R_{t}\leftarrow\rho(s_{i}^{t})

4: sample

c_{i}^{t+1}\sim p_{\text{LLM}}(\cdot\mid x_{i}^{t},s_{i}^{t},\tau_{i},m_{e}^{t})

\widetilde{m}_{e}^{t+1}\leftarrow\mathcal{M}_{e}(m_{e}^{t},c_{i}^{t+1})

\widetilde{x}_{i}^{t+1}\leftarrow g_{i}(\tau_{i},\widetilde{m}_{e}^{t+1})

\widetilde{R}\leftarrow\rho\bigl(\mathcal{V}(\widetilde{x}_{i}^{t+1},\tau_{i})\bigr)

8: if

\widetilde{R}<R_{t}

then

m_{e}^{t+1}\leftarrow\widetilde{m}_{e}^{t+1}

10: else

11:

m_{e}^{t+1}\leftarrow m_{e}^{t}

12: end if

13: end for

14: return

m_{e}^{T}

4 Experiments

We evaluate the theory on Resource Contest (RC; Table 2), Overcooked (Table 1), and SWE-bench (Table 4). RC and Overcooked use frozen MiniMax-M2.7 agents; unless noted otherwise, results are mean $\pm$ standard deviation over five seeds. SWE-bench uses the backbones listed in Table 4. All metrics come from environment ground truth or the repository test harness rather than an LLM judge. Full prompts, configurations, and per-seed trajectories are in the supplementary material.

Resource Contest.

RC is a hidden-cap allocation game: workers probe unknown caps $M_{i}\in\{0,\ldots,10\}$ and the orchestrator allocates a unit budget across workers. The optimal round reward is $G_{t}^{\star}=\max_{i}M_{i}$ , and we report cumulative reward and regret $\sum_{t}(G_{t}^{\star}-G_{t})$ . Clipping feedback is generated by the environment and therefore provides a grounded signal. The four settings vary difficulty: easy ( $N{=}3$ , caps $(3,5,8)$ , $T{=}15$ ); hard ( $N{=}3$ , caps $(6,7,8)$ , $T{=}20$ ; tightly packed caps test allocation precision); many ( $N{=}6$ , caps $(2,4,5,6,7,9)$ , $T{=}20$ ; larger search space); and drift ( $N{=}3$ , caps $(3,5,8)$ until $t{=}10$ , then $(9,5,4)$ ; a moving optimum tests re-adaptation). Since $G_{t}^{\star}=\max_{i}M_{i}$ , the oracle $\Sigma$ -reward is $120$ , $160$ , and $180$ on easy/hard/many respectively.

Table 1: Overcooked score over five seeds under matched interaction and model-call budgets. Score equals deliveries

\times 20

; higher is better.

Layout	Greedy	No memory	Free-form	Self-gated	Grounded SRMA
cramped_room	$120{\pm}40$	$180{\pm}40$	$240{\pm}60$	$280{\pm}40$	$\mathbf{320{\pm}20}$
asymmetric_advantages	$80{\pm}40$	$140{\pm}40$	$180{\pm}40$	$220{\pm}40$	$\mathbf{280{\pm}20}$
centre_pots	$40{\pm}0$	$100{\pm}40$	$160{\pm}60$	$200{\pm}40$	$\mathbf{260{\pm}20}$

Table 2: RC results over five seeds (

\Sigma

-reward; higher is better). The last two columns form the execution-memory ablation.

Setting	Oracle	$\varepsilon$ -greedy	No memory	SRMA
easy	120	$113.1{\pm}3.0$	$115.0{\pm}5.2$	$\mathbf{118.4{\pm}2.2}$
hard	160	$157.4{\pm}1.1$	$158.0{\pm}2.3$	$\mathbf{159.2{\pm}0.9}$
many	180	$170.9{\pm}4.0$	$174.0{\pm}3.7$	$\mathbf{177.3{\pm}1.9}$

SRMA reaches $98.5\%$ – $99.5\%$ of oracle reward. Execution memory adds $2.6$ reward points on average and reduces mean regret from $4.33$ to $1.70$ ( $60.8\%$ ): grounded cap evidence that is fragmented without memory becomes a functional coordination channel for the orchestrator.

Overcooked coordination.

We use Overcooked (Carroll et al. 2019) with three two-agent layouts, a horizon of $200$ , and an exact BFS verifier $V(s)=\min\{\text{joint-action steps from $s$ to the next delivery}\}$ . The verifier is deterministic and supplies the risk used for both SRMA and the drift study. The layouts stress complementary coordination demands: mutual blocking in a tight kitchen (cramped_room), role specialisation (asymmetric_advantages), and contention over shared pots (centre_pots).

Grounded SRMA is best on every layout (Table 1). Relative to the text-only self-gate, it raises score by $14.3\%$ , $27.3\%$ , and $30.0\%$ , and reaches the first delivery in $22{\pm}2$ , $26{\pm}3$ , and $32{\pm}4$ steps versus $26{\pm}5$ , $35{\pm}6$ , and $45{\pm}8$ for self-gating. The ordered improvement from no memory to free-form, self-gating, and grounded SRMA separates decomposition, memory, and grounding effects.

Grounding and gate quality.

A proposal is downstream harmful when it increases an independently evaluated oracle task risk, not necessarily the verifier score used by the gate. Table 3 shows that grounding sharply improves both selectivity and final risk.

Table 3: Accepted proposals and final risk over five seeds. Rates are fractions of harmful/helpful proposals accepted.

Method	Harmful $\downarrow$	Helpful $\uparrow$	Risk $\downarrow$
No reflection	N/A	N/A	$0.65{\pm}0.05$
Free-form	$100.0{\pm}0.0\%$	$100.0{\pm}0.0\%$	$0.42{\pm}0.12$
Self-gate	$34.5{\pm}4.2\%$	$72.8{\pm}5.1\%$	$0.28{\pm}0.08$
Grounded SRMA	$\mathbf{6.2{\pm}1.8\%}$	$\mathbf{85.4{\pm}3.6\%}$	$\mathbf{0.14{\pm}0.03}$

Grounded SRMA halves final risk relative to self-gating; the residual $6.2\%$ downstream-harmful rate measures verifier–oracle miscalibration rather than a violation of monotonicity in the verifier’s own risk.

Gate-level drift predicts held-out trajectories.

From $412$ gate events across five seeds, we use three complete seeds for calibration and hold out two entire trajectories. A trajectory-level bootstrap gives $\widehat{\beta}=0.52{\pm}0.04$ , $\widehat{c}_{1}=1.25$ , and $\widehat{c}_{2}=0.38$ , with $p_{\rm acc}(R)\approx\min\{1,\widehat{c}_{1}R^{\widehat{\beta}}\}$ . Without fitting trajectory-level parameters, the plug-in prediction $\widehat{R}_{T}=\bigl(R_{0}^{-0.52}+0.247\,T\bigr)^{-1/0.52}$ tracks the held-out risks with Pearson’s $r=0.94$ and $\mathrm{RMSE}=0.032$ , implying decay near $\mathcal{O}(T^{-1.92})$ . Bootstrap lower bounds $\underline{c}_{1}=0.92$ and $\underline{c}_{2}=0.25$ yield a conservative envelope above the empirical mean risk at every recorded step—an empirical certificate on the observed range, not a claim about unobserved states.

Statistical resolution.

A one-shot stochastic verifier falsely accepts $28.4{\pm}5.2\%$ of worsening proposals (score $245.2{\pm}38.4$ ); fixed $K=5$ cuts this to $6.8{\pm}1.5\%$ (score $312.0{\pm}18.2$ ) at $225$ verifier calls, and the adaptive gate matches that reliability ( $7.1{\pm}1.8\%$ , score $308.6{\pm}19.5$ ) with only $82{\pm}14$ calls ( $-63.6\%$ ), supporting Proposition 4: grounding supplies information, confidence control supplies resolution.

Piecewise stationarity.

In RC drift, the optimal cap changes at $t=10$ while previously written text remains unchanged, testing the re-anchoring mechanism of Proposition 5. The re-anchored grounded gate detects the shift in $1.2{\pm}0.4$ rounds, switches to the new optimum in $2.5{\pm}0.6$ , and incurs $12.6{\pm}2.8$ post-shift regret, versus $2.4{\pm}0.5$ , $7.8{\pm}1.2$ , and $38.2{\pm}5.5$ for the grounded stale-anchor variant—a $67.9\%$ cut in switch time and $67.0\%$ in regret—while the text-only gate fails to detect the change within $20$ rounds (regret $85.4{\pm}4.2$ ). Grounding detects the shift, but re-anchoring is required to replace stale memory quickly.

End-to-end software repair.

We evaluate the complete bilevel system on all 500 SWE-bench instances, with the repository test harness as the grounded verifier (an instance counts as resolved only if its submitted patch passes the harness). Each worker is a mini-SWE-agent v2 instance; the bilevel system runs $N{=}2$ such workers over a shared repository and workboard for up to three coordination rounds per episode and submits the highest- $J$ patch, whereas the mini-SWE v2 row is a single mini-SWE-agent v2 worker with no orchestrator or shared memory. The Free-form MA row keeps the same $N{=}2$ multi-agent coordination but commits every proposed reflection ungated (no verifier check), isolating the effect of SRMA’s grounded gate. The public leaderboard row is an external reference, not a controlled ablation.

Table 4: SWE-bench results on 500 instances (% resolved; official test harness).

\dagger

: controlled runs under matched budget; the public row is an external leaderboard reference, not a controlled ablation.

System	Backbone	Rate $\uparrow$
mini-SWE v2^†	DeepSeek	$68.2\%$
Bilevel SRMA^†	DeepSeek	$71.4\%$
Free-form MA^†	Kimi K2.5	$58.4\%$
mini-SWE v2 (public)	Kimi K2.5	$70.8\%$
Bilevel SRMA	Kimi K2.5	$\mathbf{72.2\%}$

On the Kimi K2.5 backbone the grounded gate is decisive: Bilevel SRMA resolves $72.2\%$ against $58.4\%$ for free-form (ungated) multi-agent reflection at matched backbone and budget, and exceeds the external public mini-SWE-agent v2 reference ( $70.8\%$ ). The controlled DeepSeek runs show the same direction ( $71.4\%$ vs. $68.2\%$ ), indicating that the gain comes from grounded, gated coordination rather than from raw model or compute.

5 Conclusion

We gave multi-agent LLM reflection a conditional, information-aware theory: bilevel coupling controls follower equilibrium slack, persistent harmful commitment creates free-form error floors, and no transcript-only gate can improve uniformly when the truth of a reflection depends on external state. SRMA supplies the missing grounding and converges exactly at order-tight geometric or polynomial rates, with confidence-gating and re-anchoring extensions; experiments on Resource Contest, Overcooked, and SWE-bench support the predicted coordination, grounding, and resolution mechanisms.

Limitations.

The guarantees are conditional: bounded coupling, finite action sets, verifier calibration, and non-degenerate corrective mass need not hold in open-ended agent tasks; drift parameters are validated only on observed trajectories; incomplete test suites guarantee monotonicity only for verifier risk, not true task utility; and re-anchoring gives per-segment convergence without a general switching-regret bound. Multi-agent coordination also spends substantial tokens before the final answer, and the $72.2\%$ Kimi result is compared with a public $70.8\%$ leaderboard run rather than a controlled method-only comparison. Future work should jointly optimise memory quality and budget-aware termination.

References

Benkovich and Valkov (2026) N. Benkovich and V. Valkov Agyn: a multi-agent system for team-based autonomous software engineering. arXiv preprint arXiv:2602.01465. Cited by: §1, §2.
Bertsekas and Tsitsiklis (2000) D. P. Bertsekas and J. N. Tsitsiklis Gradient convergence in gradient methods with errors. SIAM Journal on Optimization 10 (3), pp. 627–642. Cited by: §2.
Borkar (1997) V. S. Borkar Stochastic approximation with two time scales. Systems & Control Letters 29 (5), pp. 291–294. Cited by: §2.
Borkar (2008) V. S. Borkar Stochastic approximation: a dynamical systems viewpoint. Cambridge University Press. Cited by: §2.
Candogan et al. (2011) O. Candogan, I. Menache, A. Ozdaglar, and P. A. Parrilo Flows and decompositions of games: harmonic and potential games. Mathematics of Operations Research 36 (3), pp. 474–503. Cited by: §2, §3.1, §3.1.
Carroll et al. (2019) M. Carroll, R. Shah, M. K. Ho, T. L. Griffiths, S. A. Seshia, P. Abbeel, and A. Dragan On the utility of learning about humans for human-AI coordination. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §4.
Cemri et al. (2025) M. Cemri, M. Z. Pan, S. Yang, L. A. Agrawal, B. Chopra, R. Tiwari, K. Keutzer, A. Parameswaran, D. Klein, K. Ramchandran, M. Zaharia, J. E. Gonzalez, and I. Stoica Why do multi-agent LLM systems fail?. In Advances in Neural Information Processing Systems (NeurIPS), External Links: Link Cited by: §2.
Christodoulou and Gairing (2014) G. Christodoulou and M. Gairing The price of stability of weighted congestion games. In International Colloquium on Automata, Languages, and Programming (ICALP), Cited by: §2, §3.1.
Chung (1954) K. L. Chung On a stochastic approximation method. The Annals of Mathematical Statistics 25 (3), pp. 463–483. Cited by: §2.
Dang et al. (2025) Y. Dang, C. Qian, X. Luo, J. Fan, Z. Xie, R. Shi, W. Chen, C. Yang, X. Che, Y. Tian, X. Xiong, L. Han, Z. Liu, and M. Sun Multi-agent collaboration via evolving orchestration. In Advances in Neural Information Processing Systems (NeurIPS), Note: arXiv:2505.19591 Cited by: §1.
Doerr et al. (2012) B. Doerr, D. Johannsen, and C. Winzen Multiplicative drift analysis. Algorithmica 64 (4), pp. 673–697. Cited by: §2.
Du et al. (2025) Y. Du, M. Tian, S. Ronanki, S. Rongali, S. Bodapati, A. Galstyan, A. Wells, R. Schwartz, E. A. Huerta, and H. Peng Context length alone hurts LLM performance despite perfect retrieval. In Findings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), Note: arXiv:2510.05381 Cited by: §3.1.
Hajek (1982) B. Hajek Hitting-time and occupation-time bounds implied by drift analysis with applications. Advances in Applied Probability 14 (3), pp. 502–525. Cited by: §2.
Hong et al. (2023) M. Hong, H. Wai, Z. Wang, and Z. Yang A two-timescale stochastic algorithm framework for bilevel optimization: complexity analysis and application to actor-critic. SIAM Journal on Optimization 33 (1), pp. 147–180. Cited by: §2.
Hong et al. (2024) S. Hong, M. Zhuge, J. Chen, X. Zheng, Y. Cheng, C. Zhang, J. Wang, Z. Wang, S. K. S. Yau, Z. Lin, et al. MetaGPT: meta programming for a multi-agent collaborative framework. In International Conference on Learning Representations (ICLR), Cited by: §1, §2, §3.1.
Hu et al. (2025) S. Hu, C. Lu, and J. Clune Automated design of agentic systems. In The Thirteenth International Conference on Learning Representations (ICLR), External Links: Link Cited by: §1.
Huang et al. (2025) L. Huang, W. Yu, W. Ma, W. Zhong, Z. Feng, H. Wang, Q. Chen, W. Peng, X. Feng, B. Qin, and T. Liu A survey on hallucination in large language models: principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems. Cited by: §3.2.
Ji et al. (2024) Z. Ji, N. Lee, R. Frieske, T. Yu, D. Su, Y. Xu, E. Ishii, Y. J. Bang, A. Madotto, and P. Fung Survey of hallucination in natural language generation. ACM Computing Surveys 55 (12), pp. 1–38. Cited by: §3.2.
Jimenez et al. (2024) C. E. Jimenez, J. Yang, A. Wettig, S. Yao, K. Pei, O. Press, and K. Narasimhan SWE-bench: can language models resolve real-world GitHub issues?. In International Conference on Learning Representations (ICLR), Cited by: §1.
Johannsen (2010) D. Johannsen Random combinatorial structures and randomized search heuristics. Ph.D. Thesis, Universität des Saarlandes. Cited by: §2.
Karimi et al. (2016) H. Karimi, J. Nutini, and M. Schmidt Linear convergence of gradient and proximal-gradient methods under the Polyak–Łojasiewicz condition. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases (ECML-PKDD), pp. 795–811. Cited by: §2.
Lehre and Witt (2021) P. K. Lehre and C. Witt Tail bounds on hitting times of randomized search heuristics using variable drift analysis. Combinatorics, Probability and Computing 30 (4), pp. 550–569. Cited by: §2.
Levy et al. (2024) M. Levy, A. Jacoby, and Y. Goldberg Same task, more tokens: the impact of input length on the reasoning performance of large language models. In Association for Computational Linguistics (ACL), Cited by: §3.1.
Liu et al. (2024) N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang Lost in the middle: how language models use long contexts. In Transactions of the Association for Computational Linguistics (TACL), Cited by: §3.1.
Liu et al. (2025) S. Liu, Y. Liu, Z. Wang, Y. Wang, H. Wu, L. Xiang, and Z. He Select-then-decompose: from empirical analysis to adaptive selection strategy for task decomposition in large language models. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), Note: arXiv:2510.17922 Cited by: §3.1.
Liu et al. (2026) X. Liu, X. Yang, Z. Li, P. Li, and R. He AgentHallu: benchmarking automated hallucination attribution of LLM-based agents. arXiv preprint arXiv:2601.06818. Cited by: §2, §3.1.
Madaan et al. (2023) A. Madaan, N. Tandon, P. Gupta, S. Hallinan, L. Gao, S. Wiegreffe, U. Alon, N. Dziri, S. Prabhumoye, Y. Yang, et al. Self-refine: iterative refinement with self-feedback. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §2.
Meyn and Tweedie (2009) S. Meyn and R. L. Tweedie Markov chains and stochastic stability. 2nd edition, Cambridge University Press. Cited by: §2.
Monderer and Shapley (1996) D. Monderer and L. S. Shapley Potential games. Games and Economic Behavior 14 (1), pp. 124–143. Cited by: §2, §3.1.
Panickssery et al. (2024) A. Panickssery, S. R. Bowman, and S. Feng LLM evaluators recognize and favor their own generations. arXiv preprint arXiv:2404.13076. Cited by: §2, §3.3.
Qian et al. (2025) C. Qian, Z. Xie, Y. Wang, W. Liu, K. Zhu, H. Xia, Y. Dang, Z. Du, W. Chen, C. Yang, Z. Liu, and M. Sun Scaling large language model-based multi-agent collaboration. In The Thirteenth International Conference on Learning Representations (ICLR), External Links: Link Cited by: §1.
Robbins and Monro (1951) H. Robbins and S. Monro A stochastic approximation method. The Annals of Mathematical Statistics 22 (3), pp. 400–407. Cited by: §2.
Shinn et al. (2023) N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao Reflexion: language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §1, §2.
Srikant and Başar (1992) R. Srikant and T. Başar Asymptotic solutions of weakly coupled stochastic teams with nonclassical information. IEEE Transactions on Automatic Control 37 (2), pp. 163–173. Cited by: §2, §3.1.
Wang et al. (2025) Y. Wang, S. Liu, J. Fang, and Z. Meng EvoAgentX: an automated framework for evolving agentic workflows. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), System Demonstrations, pp. 643–655. External Links: Link Cited by: §1.
Wu et al. (2024) Q. Wu, G. Bansal, J. Zhang, Y. Wu, B. Li, E. Zhu, L. Jiang, X. Zhang, S. Zhang, J. Liu, A. H. Awadallah, R. W. White, D. Burger, and C. Wang AutoGen: enabling next-gen LLM applications via multi-agent conversations. In First Conference on Language Modeling (COLM), Cited by: §1, §2, §3.1.
Wu et al. (2026) S. Wu, X. Li, Y. Feng, Y. Li, Z. Wang, and R. Wang Council mode: a heterogeneous multi-agent consensus framework for reducing LLM hallucination and bias. arXiv preprint arXiv:2604.02923. Cited by: §2.
Xu et al. (2025) W. Xu, Z. Liang, K. Mei, H. Gao, J. Tan, and Y. Zhang A-MEM: agentic memory for LLM agents. In Advances in Neural Information Processing Systems (NeurIPS), External Links: Link Cited by: §1.
Zhang et al. (2025a) J. Zhang, J. Xiang, Z. Yu, F. Teng, X. Chen, J. Chen, M. Zhuge, X. Cheng, S. Hong, J. Wang, B. Zheng, B. Liu, Y. Luo, and C. Wu AFlow: automating agentic workflow generation. In The Thirteenth International Conference on Learning Representations (ICLR), External Links: Link Cited by: §1.
Zhang et al. (2025b) Z. Zhang, X. Bo, C. Ma, R. Li, X. Chen, Q. Dai, J. Zhu, Z. Dong, and J. Wen A survey on the memory mechanism of large language model-based agents. ACM Transactions on Information Systems 43 (6), pp. 155. External Links: Document, Link Cited by: §1.
Zheng et al. (2023) L. Zheng, W. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §2.
Zhou et al. (2025) H. Zhou, Y. Chen, S. Guo, X. Yan, K. H. Lee, Z. Wang, K. Y. Lee, G. Zhang, K. Shao, L. Yang, and J. Wang Memento: fine-tuning LLM agents without fine-tuning LLMs. arXiv preprint arXiv:2508.16153. External Links: Link Cited by: §1.