Wuying-Browser-Agent: Real-World Centric Fundamental Long-Horizon Browser Agents

The harness exposes a structured browser action space with 24 atomic operations, summarized in Table 1. These actions span five functional categories, including navigation, interaction, extraction, file handling, and flow control. At each interaction step, the model emits a structured action over this tool space as its browser response $y_{t}$.

The harness parses each emitted call, validates its schema and parameters against the tool definitions, dispatches it to the corresponding executor in the AgentBay sandbox (Section 3.2), and returns structured feedback $e_{t}$ together with the updated browser observation $o_{t+1}$. Invalid calls are rejected with typed error messages that re-enter the context as feedback, allowing the policy to detect and correct its own malformed outputs rather than failing silently. This establishes a tightly coupled interaction process in which perception, decision, execution, and state update remain linked at every step.

A key function of the harness is task-level state management. Rather than exposing the model to an ever-growing raw interaction transcript, the harness maintains an efficient decision-oriented context that keeps the most relevant task state, action history, and environment feedback available at each step. It also manages intermediate artifacts such as extracted content and temporary files, enabling cross-step memory and result delivery for tasks whose outcome is a file or an aggregated report. Because this shared harness abstraction is used consistently in RUIC-SFT data construction, online RL rollouts, and evaluation, the policy experiences an identical interface across offline learning, online interaction, and deployment, allowing direct transfer between stages. More broadly, this design trains the policy to operate over a schema-constrained tool interface with explicit state tracking and feedback grounding, which helps preserve transfer to more general agentic settings beyond browser-specific benchmarks.

Simply increasing the volume of generic browser trajectories [52, 7] is unlikely to resolve either deficiency. General successful demonstrations teach the model how to act correctly when execution remains on the intended path, but provide little supervision on what should happen after the model has already deviated from that path. Likewise, complex UI widgets appear only sparsely in naturally collected browser trajectories, with heavily skewed type distributions, making it difficult for scale alone to induce robust interaction policies. In other words, the missing supervision is structural rather than merely quantitative. The reflection dataset explicitly exposes the model to trajectories with an error–awareness–reflection–correction structure, while the specialized UI dataset systematically concentrates training on interaction patterns that are under-represented in general web data.

From a learning perspective, the two datasets provide complementary supervision. The specialized UI dataset reduces uncertainty in action selection over complex controls by repeatedly exposing the model to structured interaction patterns for difficult widgets. The reflection dataset, in contrast, provides supervision specifically for off-trajectory recovery, which is largely absent from successful demonstrations alone. Together, they improve not only the probability of executing a difficult task correctly on the first attempt, but also the model’s ability to recover when deviations still occur.

Each reflection example is organized around an erroneous step $t^{*}$. In addition to the task context and the incorrect action itself, the example records the resulting error observation, a natural-language reflection explaining why the action was inappropriate under the current page state, and a correction strategy consisting of actions that return the agent to a viable execution path. The dataset covers a broad range of common browser-agent failure patterns, including navigation mistakes, interaction errors, state misinterpretation, and ineffective repeated behavior. All correction strategies are executed in a sandbox environment to verify that they genuinely recover the task.

Reflection examples are collected through a combination of automated mining and human-in-the-loop curation. Failed trajectories generated during bootstrapping are analyzed to localize likely error segments, after which annotators provide reflection reasoning and corrective continuations. We further expand coverage by constructing additional recovery examples around underrepresented failure patterns and by preserving matched successful and unsuccessful interaction traces when useful for future preference-based training.

Data quality is controlled through a combination of execution verification, annotation review, and distributional monitoring. In particular, reflection reasoning must be grounded in concrete state transitions, and correction sequences must be executable and demonstrably useful in restoring a viable task path.

As a result, the reflection dataset does not merely expose the model to failed trajectories; it teaches the policy to treat unexpected observations as evidence of possible off-path execution and to generate grounded corrective continuations, which are central to robust browser deployment.

Modern web applications widely employ complex UI controls whose interaction patterns differ substantially from those of ordinary text inputs and buttons. Examples include date pickers that require hierarchical year–month–day selection, cascaders that rely on progressive expansion and linked confirmation, and rich-text editors involving multi-step toolbar interactions. These controls are often implemented differently across frontend frameworks and websites, so sparse natural exposure is insufficient for the model to internalize reliable interaction strategies. This creates a systematic capability bottleneck in browser-agent deployment.

To address this issue, we construct a specialized UI component dataset through a four-stage pipeline. We first collect component scenes from real websites and establish mappings among component type, website, page, and concrete control instance using a combination of automated DOM-structure scanning and manual verification. We then design parameterized interaction tasks for each control type with progressively increasing difficulty, ranging from simple single-step selection tasks to compound operations and edge cases involving invalid inputs or exceptional control states. Next, human annotators operate the real controls and record action sequences, key operation nodes, and control state transitions such as panel expansion, highlighting changes, and linked updates. Finally, we augment the dataset through cross-website transfer of interaction strategies, generation of equivalent action-sequence variants, and injection of anomalous states such as loading delays, disabled controls, and validation failures. These anomalous UI states also create a natural bridge to the reflection dataset, allowing the two supervision sources to reinforce one another.

This concentrated supervision reduces the reliance on sparse natural exposure and equips the policy with reusable interaction patterns for UI structures that frequently trigger failures in real deployment.

Figure 4 provides an overview of the three data sources and their curriculum-based integration. Rather than training sequentially on each data source or using a fixed mixture from the beginning, we adopt a curriculum-based mixed training strategy with progressive ratio annealing [5]. The capability dependencies in browser-agent learning suggest a natural order: the model should first stabilize basic browser actions, then strengthen its ability to manipulate specialized UI components, and only afterward emphasize self-correction behavior. At the same time, purely sequential training risks catastrophic forgetting, especially when late-stage reflection-heavy training shifts the model toward overly cautious behavior. We therefore combine curriculum learning with progressive mixture control.

Let $D_{g}$, $D_{u}$, and $D_{r}$ denote the general browser task dataset, the specialized UI component dataset, and the reflection dataset, respectively. Rather than using a fixed mixture throughout training, we adopt a dynamic mixing strategy that changes over the course of training. As illustrated in Figure 4, training begins with a general-data-dominant distribution to stabilize basic browser operations, gradually increases the proportion of specialized UI data to strengthen complex interaction skills, and introduces reflection data only in the later stage to cultivate self-correction without destabilizing the execution prior. The transition schedule is selected based on validation behavior.

The three phases serve distinct purposes. Phase 1 is dominated by general browser trajectories, with a small amount of UI-specialized exposure, and is primarily used to stabilize foundational capabilities such as navigation, element grounding, clicking, typing, and action formatting. Phase 2 gradually shifts emphasis toward complex component interaction while retaining sufficient general data to preserve the browser-operation prior acquired in Phase 1. Phase 3 introduces reflection data while annealing the UI-specialized proportion to a lower level. This delayed introduction of $D_{r}$ is deliberate: preliminary experiments showed that introducing reflection supervision before the execution policy had stabilized led to overly conservative behavior, with the agent becoming excessively prone to backtracking and over-explaining even on straightforward tasks. By introducing reflection only after the execution prior and UI interaction skills have been sufficiently strengthened, we obtain a more balanced model that can both act decisively and recover when necessary. The phase boundaries and endpoint ratios are determined from preliminary tuning; Section 7.6 compares this schedule against fixed-ratio mixing and an aggressive early-reflection variant.

Under this schedule, the learning objective remains the standard supervised next-token prediction loss, with the effective training distribution changing over time:

The curriculum is therefore realized through dynamic sampling rather than by changing the optimization objective itself. This allows us to retain the optimization stability of conventional SFT while altering the effective supervision distribution in a capability-aware manner.

From a functional perspective, the three data sources contribute distinct but complementary signals. General data $D_{g}$ provides the foundational prior for standard browser operation. Specialized UI data $D_{u}$ improves recognition and manipulation of complex controls whose interaction logic cannot be reliably inferred from sparse natural coverage. Reflection data $D_{r}$ teaches the model to interpret unexpected observations as evidence of possible failure, attribute the error to plausible preceding actions, and generate corrective continuations. Because these capabilities are introduced through a progressively annealed mixture rather than abrupt dataset switching, the resulting model is less prone to forgetting previously acquired behavior and better able to balance direct task execution with recovery behavior.

In summary, RUIC-SFT provides a capability-structured initialization that extends beyond behavioral cloning to establish a structured credit prior for subsequent online optimization. By explicitly annotating step-level error-recovery pairs, the reflection dataset supplies dense pseudo-labels for what would otherwise be sparse terminal rewards in long-horizon tasks. Consequently, DAO-GRPO initializes from a policy with inherent causal attribution capability, allowing online RL to concentrate its optimization budget on refining branch-defining decisions rather than discovering recovery primitives from scratch. This structured handoff motivates the divergence-aware online refinement stage described next.

DAO-GRPO is built around a simple principle: online learning should optimize the browser policy using the same branching structure and step-specific context under which the agent actually acts. Figure 5 illustrates the system-level interaction loop of online optimization. For each sampled task $d\sim\mathcal{D}$, the current policy $\pi_{\theta}$ is served by the browser agent controller and interacts with multiple AgentBay browser-use sandbox instances in parallel. Each sandbox executes one browser rollout, producing a trajectory, and the resulting trajectory group is consumed by the DAO-GRPO optimizer for policy updates. The same lightweight browser interface introduced in Section 3.2 is used during rollout, enabling efficient interaction without repeatedly serializing full browser states.

For each sampled task, DAO-GRPO generates a grouped rollout

where each trajectory $\tau_{i}$ contains a sequence of step-specific decision contexts, policy responses, executable browser actions, and environment feedback. At step $t$, the reconstructed decision context $c_{i,t}$ may include the task instruction, the current structured browser state $S_{i,t}$, an optional screenshot $V_{i,t}$ when visual grounding is required, and the retained interaction history consisting of previous actions and environment feedback. For optimization, DAO-GRPO combines three quantities: a trajectory-level relative advantage $A_{i}$ estimated within the group, a step-level credit weight $w_{i,t}$ derived from cross-trajectory divergence analysis, and the step-specific decision context $c_{i,t}$ reconstructed by the harness. These are fused into a weighted response-level policy objective.

A first challenge in online browser learning is that task-level success provides only sparse supervision, while robust long-horizon behavior depends on intermediate progress signals. We therefore combine terminal judgment with potential-based reward shaping (PBRS), which provably preserves the optimal policy in the discounted infinite-horizon setting [22]; our finite-horizon grouped variant inherits this only as a practical approximation. Empirically, adding shaping improves the terminal success rate itself rather than merely inflating shaped returns (Table 5), indicating that the potential signal guides rather than hijacks learning.

For each trajectory $\tau_{i}$, we obtain a terminal reward $R_{i}^{\mathrm{term}}\in\{0,1\}$ indicating whether the terminal browser state satisfies the task specification. To densify this sparse signal, we augment it with a potential-based shaping term $r^{\mathrm{shape}}_{i,t}=\gamma\Phi(s_{i,t+1})-\Phi(s_{i,t})$, where $\Phi(\cdot)$ is a task-conditioned progress estimator and $\gamma$ is a discount factor. The boundary condition $\Phi(s_{T_{i}})=0$ ensures that shaping does not alter the terminal success criterion. The resulting trajectory return combines both signals:

Since trajectories are generated in groups for the same task, we normalize returns within each rollout group to obtain a trajectory-level relative advantage:

where $\mu_{R}$ and $\sigma_{R}$ are the group mean and standard deviation. This grouped normalization yields a task-matched preference signal that is less sensitive to reward-scale variation across tasks. Groups in which all trajectories receive identical returns are discarded, as they carry no relative optimization signal.

A second challenge is that robust browser execution often hinges on a small number of branch-defining decisions: multiple trajectories may share a long common prefix, yet diverge sharply once the agent chooses an incorrect page, element, or action sequence. Uniform trajectory-level weighting therefore fails to focus learning on the decisions that most strongly determine whether execution stays on path or deviates further.

To address this issue, DAO-GRPO estimates step importance from semantic divergences within each trajectory group. Given the $K$ trajectories sampled for the same task, an LLM-based divergence estimator compares them step by step and identifies branch points at which relatively successful and unsuccessful trajectories first differ in a behaviorally meaningful way. For each detected divergence, the estimator outputs (1) a divergence step index $t^{*}$, (2) a criticality score $c^{*}\in[0,1]$, and (3) a partition of trajectories into a favorable branch $\mathcal{B}^{+}$ and an unfavorable branch $\mathcal{B}^{-}$.

We emphasize that the divergence estimator is used as a semantic credit prior rather than as an exact oracle. The direction of optimization is still determined by the trajectory-level advantage $A_{i}$ in Eq. (6); divergence estimation only redistributes update magnitude across steps.

For trajectory $\tau_{i}$ and step $t$, we define the step credit weight as:

where $\alpha_{\mathrm{shared}}$ and $\alpha_{\mathrm{div}}$ are scaling constants, and $\delta^{+},\delta^{-}\in(0,1)$ are asymmetric decay factors satisfying $\delta^{+}>\delta^{-}$. Shared-prefix steps receive a smaller base weight because they contribute little to distinguishing successful from unsuccessful branches. The divergence step receives an emphasized weight modulated by the estimated criticality score, and post-divergence steps on favorable branches retain credit longer than those on unfavorable branches. The specific values of these hyperparameters are tuned on the validation set and omitted here to encourage adaptation to different browser environments.

In practice, the resulting step weights are normalized to maintain stable optimization across trajectories of different lengths. Overall, this mechanism acts as a robustness-oriented credit prior: it does not change which trajectories are preferred globally, but sharpens where the learning signal is concentrated within them.

A third challenge is a train–test mismatch in browser-policy optimization. During real interaction, the agent acts under a step-specific context that is dynamically reconstructed from the current page state, retained history, and optional visual evidence. If training instead optimizes all responses under a single concatenated trajectory transcript, the policy is updated under contexts that differ from those actually available at decision time.

Unlike standard GRPO, which assumes an append-only context and optimizes over full trajectory likelihoods, DAO-GRPO decouples context reconstruction from advantage estimation: the group-relative advantage $A_{i}$ retains GRPO’s variance-reduction benefit, while each $\log\pi_{\theta}(y_{i,t}\mid c_{i,t})$ is evaluated under the exact information set available at decision time. We perform a single on-policy update per rollout batch, so no importance correction is required.

The trajectory-level preference signal and the step-level credit signal are combined into a weighted training target for each step. In this way, optimization is concentrated on semantically decisive branch steps rather than diluted across long shared prefixes.

Each step-specific context $c_{i,t}$ is produced by the harness state manager (Section 4.1): DAO-GRPO replays the interaction log through the same reconstruction rules used at online inference, so that $\log\pi_{\theta}(y_{i,t}\mid c_{i,t})$ is evaluated under exactly the information set available at decision time.

In browser interaction, the information available to the policy at each step is determined by the current page state together with the relevant retained history, rather than by a naively concatenated transcript of everything that has happened before. Training should therefore follow the same context organization used at inference time. Otherwise, the policy may be optimized under contexts that are unnecessarily verbose, partially outdated, or inconsistent with the actual online decision process.

For this reason, our online RL stage optimizes each model response under the step-specific decision context actually available at that point in the interaction, while regularizing the policy against excessive drift from the supervised initialization. This can be interpreted as an advantage-weighted policy update under dynamically managed browser contexts, rather than imitation over a single fixed trajectory transcript.

In practice, each $c_{i,t}$ is reconstructed by replaying the interaction log up to step $t$ using the same state replacement and diff-appending rules as in online inference. When visual grounding is unnecessary, the screenshot block is simply omitted from the reconstructed context. As a consequence, non-critical steps receive only weak update strength, while semantically decisive branch steps dominate the optimization signal.

The updated policy is then used in the next round of grouped parallel rollout. This design aligns optimization more closely with the actual online inference condition, reducing context mismatch and improving robustness to changing browser states.

Table 4 studies the contribution of the two specialized data sources and the curriculum schedule in RUIC-SFT. The results show that $D_{u}$ and $D_{r}$ provide complementary benefits. Compared with Base SFT (#1), adding $D_{u}$ (#2) improves the overall success rate from 32.0% to 34.9% while keeping the recovery success rate nearly unchanged (8.5% vs. 9.0%), indicating that UI-specialized supervision primarily strengthens browser-operation competence rather than error correction. In contrast, adding $D_{r}$ (#3) nearly doubles the recovery success rate from 8.5% to 16.4% together with a comparable overall gain, showing that reflection data is particularly effective for off-trajectory correction and self-recovery.

Combining all three data sources with a fixed global mixture (#4) further improves overall success rate over the single-source variants, but also increases the average number of actions per completed task from 24.8 to 29.3. This suggests that naive uniform mixing may introduce more redundant or hesitant behavior during execution, despite improving task completion.

In contrast, the full RUIC-SFT curriculum (#5) achieves the best overall performance, improving success rate to 38.0% and recovery success rate to 18.5%, while reducing the average step count to 22.4. This indicates that phased curriculum scheduling better balances execution efficiency and recovery behavior than fixed-ratio multi-source training.

Finally, introducing reflection data aggressively from the beginning of training (#6) degrades overall performance relative to both the fixed mixture and the full curriculum, despite maintaining a recovery success rate comparable to the fixed-mixing variant. This supports our design choice of delaying reflection-heavy supervision until a stable browser-operation prior has first been established.

Table 5 evaluates the contribution of the main design choices in our online RL framework, including denser progress supervision, more targeted credit assignment, and training under step-wise decision contexts. All variants are initialized from the same RUIC-SFT checkpoint and trained for the same number of online iterations.

Vanilla online GRPO improves only modestly over the RUIC-SFT initialization (38.0% $\rightarrow$ 39.4%), confirming that sparse terminal rewards alone provide limited supervision for long-horizon browser tasks. Adding PBRS improves the overall success rate to 40.7%, and also yields gains on hard tasks and recovery-oriented tasks. This result suggests that dense progress signals are beneficial in browser environments where terminal rewards are sparse and meaningful intermediate progress is otherwise weakly supervised.

Adding divergence-aware credit assignment further improves performance, with the largest gains appearing exactly where the method targets: the hard subset (25.7% $\rightarrow$ 27.6%) and recovery-oriented tasks (26.2% $\rightarrow$ 29.7%). This supports our hypothesis that browser trajectories often share long prefixes and only diverge at a few decisive branch points, making localized credit assignment more effective than uniformly weighting all response segments.

Finally, enabling the full response-level objective under reconstructed contexts further improves all metrics, raising the overall success rate to 42.9%, the hard-task success rate to 30.5%, and the recovery-oriented success rate to 32.8%. This result confirms the importance of optimizing each response under its own decision context, rather than under a single concatenated trajectory transcript, since browser-agent interaction is naturally state-reconstructed rather than append-only.

Overall, the ablation validates that the three components are complementary: PBRS alleviates sparse supervision, divergence-aware credit sharpens optimization around decisive branch steps, and response-level optimization aligns the learning objective more faithfully with actual browser inference.

Since divergence-aware credit assignment relies on an LLM-based estimator, we validate its reliability directly. Two annotators independently labeled the ground-truth divergence step $t^{*}$ on 100 rollout groups sampled from training (inter-annotator agreement within one step: 91%). As shown in Figure 7(a), the estimator localizes $t^{*}$ exactly in 61% of groups, within one step in 78%, and within two steps in 89%, while the favorable/unfavorable branch partition agrees with human annotation in 94% of groups. The estimator is thus approximately correct rather than exact, which matches its intended role as a credit prior (Section 4.3.3).

We further test whether DAO-GRPO tolerates this level of localization noise. We retrain the policy under the identical schedule while perturbing every detected $t^{*}$ by a random offset of up to $\pm 2$ or $\pm 4$ steps, or replacing it with a uniformly random step. As shown in Figure 7(b), performance degrades gracefully: $\pm 2$ perturbation costs $0.9$ points (42.9% $\rightarrow$ 42.0%) and even $\pm 4$ perturbation retains 41.1%, still above the uniform-credit variant (+PBRS, 40.7%), indicating that the method only requires the divergence prior to be approximately correct. Random placement drops below uniform credit (40.3%), confirming that concentrating updates at wrong steps is worse than not concentrating at all; degradation is amplified on the hard subset (30.5% $\rightarrow$ 25.7%), where accurate branch localization matters most.

The remaining LLM-based component is the progress estimator $\Phi$, which converts trajectory prefixes into the potential values used by PBRS. We validate it against human judgment on 150 trajectory prefixes sampled from held-out dev rollouts, each rated by two annotators on a five-level progress rubric anchored by subgoal descriptions (inter-annotator agreement: 93% within one level, Spearman $\rho=0.89$). $\Phi$ tracks the human ratings closely (Spearman $\rho=0.81$, mean absolute error $0.11$ on the unit interval), agrees with the human progress ordering on 86% of 200 same-task prefix pairs, and is monotone non-decreasing along 87% of adjacent within-trajectory steps. Since PBRS only requires the potential to rank prefixes by progress rather than to be exact, this level of agreement suffices for the shaping signal to be informative, consistent with the net gain of the +PBRS variant in Table 5. Moreover, potential-based shaping preserves the optimal policy for any bounded $\Phi$, so residual estimator noise degrades credit quality gracefully rather than biasing the optimization objective.

Figure 8 presents the online RL behavior of DAO-GRPO on Wuying-Browser-Agent-9B. As shown in Figure 8(a), both the training reward and the held-out validation reward exhibit an initial decline – bottoming out around iteration 30 – followed by sustained improvement, with the training reward recovering to its initial level by roughly iteration 90. We attribute this early dip to the transition from the RUIC-SFT initialization to online policy optimization, where the agent must first adapt to exploration, grouped relative rewards, and dynamically reconstructed browser contexts. After this adaptation phase, the reward increases steadily, indicating that direct browser interaction provides useful supervision beyond offline demonstrations. Importantly, the validation reward computed on valid rollouts only follows the same upward trend and remains consistently above the raw validation reward (0.76 vs. 0.73 at iteration 150), showing that the gain is not merely due to changes in invalid-rollout frequency.

Figure 8(b) further shows that the invalid-action ratio consistently decreases on both the training and validation environments (15.1% $\rightarrow$ 6.1% and 16.4% $\rightarrow$ 3.1%, respectively, comparing the first and last ten iterations). This suggests that DAO-GRPO improves not only task-level return but also the executability and reliability of browser actions, which is critical for real-world browser-agent deployment.

Figure 8(c) reports the fraction of rollout groups with zero reward variance, i.e., groups in which all rollouts in a group obtain identical returns and therefore provide no relative-advantage signal. Such degenerate groups are frequent – 26.7% of all groups on average and up to 49.9% in individual iterations – and, notably, their frequency does not decrease as the policy improves (21.2% in the first ten iterations vs. 22.2% in the last ten), because they arise predominantly from tasks that remain unsolvable rather than from early-stage format failures. Dynamic sampling masks these groups from every update, so the effective batch is substantially smaller than the nominal one throughout training; this persistent filtering is what makes the rollout-throughput considerations below practically important rather than incidental.

Figure 8(d) studies the effect of sandbox concurrency under the asynchronous rollout scheduler. Under all training budgets, higher concurrency improves final performance, and the benefit widens as training proceeds. Note that the group size is fixed throughout training; concurrency instead affects optimization through data freshness and selectivity. With more sandboxes active, completed trajectory groups are returned faster relative to policy updates, so each update consumes rollouts generated by a more recent policy, and dynamic sampling can draw from a larger pool of candidate groups when discarding degenerate ones. Together, these results validate that online RL yields complementary gains over RUIC-SFT and that rollout throughput is itself an important factor for effective browser-agent optimization.

Figure 9 analyzes success rate by task difficulty (a) and trajectory length (b) for Qwen3.5-9B, RUIC-SFT, and DAO-GRPO.

In the left panel, performance decreases consistently from easy to medium to hard tasks for all three models, confirming that the empirical solvability-based difficulty labels align well with the actual challenge faced by the browser agent. RUIC-SFT improves over the base Qwen3.5-9B model across all difficulty levels. The largest gains appear on easy and medium tasks (38.1% $\rightarrow$ 51.4% and 17.9% $\rightarrow$ 42.9%), reflecting the browser-operation and UI-interaction competence instilled by curriculum supervision, while the gain on hard tasks is smaller (14.3% $\rightarrow$ 18.1%), as these tasks additionally demand long-horizon credit assignment that offline supervision alone cannot provide.

DAO-GRPO further improves performance at all difficulty levels, and its gains grow steeply with task difficulty: $+1.0$ point on easy tasks, where the supervised policy is already near saturation, $+2.1$ points on medium tasks, and $+12.4$ points on hard tasks (18.1% $\rightarrow$ 30.5%), the largest improvement across all slices. This suggests that online optimization is especially beneficial once the task requires long-horizon correction, branch-sensitive decision making, and recovery from off-trajectory states beyond what can be fully covered by offline supervision.

In the right panel, success rate drops as trajectory length increases, indicating that longer browser interactions are substantially more challenging. RUIC-SFT improves performance across all trajectory-length bins, with gains concentrated on shorter tasks (e.g., $+22.2$ points on 15–25 step tasks), showing that better browser-operation priors and recovery-oriented supervision are broadly useful but insufficient for the longest interactions. More importantly, the gain of DAO-GRPO becomes increasingly pronounced as trajectories grow longer: from $+1.1$ points on 15–25 step tasks to $+2.8$ points on 26–35 step tasks, $+5.2$ points on 36–50 step tasks, and $+13.4$ points on tasks exceeding 50 steps (13.3% $\rightarrow$ 26.7%). This pattern is well aligned with the motivation of DAO-GRPO: as browser trajectories become longer, sparse terminal reward, delayed credit assignment, and semantically decisive branch steps become increasingly important, making online RL particularly effective.

Overall, the figure shows that RUIC-SFT primarily strengthens difficult browser capabilities, while DAO-GRPO further improves long-horizon decision quality and recovery under challenging interaction settings.

We first examine three rollout groups sampled from online RL training (Figure 10), which illustrate why the three components of DAO-GRPO match the structure of browser-agent data. Figure 10(a) exemplifies the shared-prefix structure that motivates divergence-aware credit assignment: trajectories within a group are identical for the first three steps and diverge at a single decisive step, where the branch choice largely determines the outcome. Top-down browsing succeeds in 12 of 16 cases, whereas jumping directly to the product anchor stalls at partial credit in 3 of 4 cases. The shaped credit therefore concentrates on the post-divergence steps of the favorable branch. Figure 10(b) shows a group in which 16 of 17 rollouts fail with returns of at most $0.06$. Under terminal-only rewards, such a group has zero advantage variance and would be discarded by dynamic sampling (Figure 8(c)), whereas PBRS progress increments keep the rollouts correctly ordered and the group informative. Figure 10(c) shows that even when all displayed rollouts reach the same terminal reward, their efficiency differs by more than a factor of two, ranging from 21 to 49 steps. Signed PBRS increments penalize individual detour steps. For example, one eventually successful rollout receives fourteen consecutive penalties during a filter re-entry loop, providing step-level supervision toward efficient behavior that identical terminal rewards cannot express.

Figure 11 presents a representative long-horizon trajectory that illustrates the self-corrective browser behavior of Wuying-Browser-Agent-9B. The task requires the agent to enter the Chery automotive website, navigate to the privacy-policy page, locate the privacy-policy update announcement, open the personal information collection list, and inspect the categories and purposes of collected personal information.

The trajectory exhibits a clear perceive–act–verify–recover loop rather than a monotonic execution pattern. In the early stage (Steps 1–5), the agent correctly loads the website, handles the cookie banner, scrolls to the footer, and enters the privacy-related page. At Step 7, however, it clicks a misleading footer link and is silently redirected to an unrelated brand history page. Instead of assuming progress from the issued action, the agent verifies the resulting browser state against the intended goal and, at Step 8, detects the mismatch from the URL and page content. It then recovers by re-entering the privacy path. When repeated scrolling still fails to reveal the target section (Steps 9–10), the agent escalates from low-cost navigation to structured page probing before retrying the correct route. This eventually allows it to reach the detailed policy page, open the personal information collection list, and complete the final inspection steps (Steps 12–15).

This recovery behavior is consistent with the complementary roles of RUIC-SFT and DAO-GRPO. RUIC-SFT provides recovery-oriented browser priors through UI-specialized supervision and reflection-rich trajectories, teaching the agent to judge progress from observed environment outcomes rather than from action issuance alone. DAO-GRPO further strengthens this ability under long-horizon online interaction. In this example, the successful and unsuccessful paths share a long common prefix, while only a few branch decisions determine whether the agent reaches the target document or remains trapped in an irrelevant page. This aligns with our divergence-aware online RL design, which concentrates optimization on decisive branch steps such as erroneous redirects, recovery actions, and strategy switches from scrolling to structured extraction. In addition, the response-level objective optimizes each decision under its actual reconstructed browser context, helping the model rely on the current page state, URL, and visual evidence when deciding whether to continue, recover, or escalate.

Overall, the case study shows that the gain of our method is not merely higher task success, but stronger closed-loop interaction quality. The model can detect navigation failure, revise its hypothesis, avoid ineffective repetition, and recover toward the original goal under dynamically changing web contexts.