Change2Task: From Repository Changes to Executable Coding Agent Tasks and Environments

Agents operating at repository scale inspect code, edit files, invoke tools, and revise solutions from execution feedback. SWE-agent studies interfaces between agents and computers, OpenHands provides a sandboxed platform, AutoCodeRover combines program structure with search, and Agentless uses fixed localization, repair, and validation stages (Yang et al. 2024; Wang et al. 2025b; Zhang et al. 2024; Xia et al. 2025). SWE-bench established issue resolution on repository snapshots with hidden tests; SWE-Gym and R2E-Gym extend executable feedback to training (Jimenez et al. 2024; Pan et al. 2025; Jain et al. 2025). Together, these systems show that performance depends on the model, repository state, interaction interface, and exposed verifier.

Recent benchmarks expand freshness, industrial difficulty, and task breadth through Multi-SWE-bench, SWE-bench-Live, SWE-Bench Pro, and SWE-bench Multimodal (Zan et al. 2025; Zhang et al. 2025; Deng et al. 2025; Yang et al. 2025a). FeatureBench and SWE-Cycle cover feature development and fuller issue resolution cycles; SWE-EVO and TestEvo-Bench study repository evolution and joint code and test evolution (Zhou et al. 2026; Guan et al. 2026; Le et al. 2025; Wang, Wang, and Nie 2026). Live, multimodal, and evolutionary settings also require environments to remain executable as code, dependencies, and interfaces change. This breadth increases demand for maintained environments, outputs specific to each task, and reliable verifiers.

Executable evaluation requires dependencies, services, build commands, test discovery, and isolation. EnvBench and SetupBench document persistent setup obstacles (Eliseeva et al. 2025; Arora, Jang, and Moghaddam 2025). Repo2Run synthesizes Dockerfiles from feedback, RepoLaunch automates setup and test management, and MEnvAgent combines planning, verification, and cross-language reuse (Hu et al. 2025; Li et al. 2026b; Guo et al. 2026a). Their focus is recovering or maintaining a runnable revision, an essential prerequisite for agent evaluation. They do not determine how many distinct, historically grounded task states one prepared base can support.

At scale, SWE-Factory coordinates environment, test, and instance construction, while daVinci-Env and SWE-Universe combine specialized builders, self-verification, and distributed execution (Guo et al. 2026b; Fu et al. 2026; Chen et al. 2026). SWE-smith reuses configured images for synthetic instances (Yang et al. 2025b). These efforts reduce setup cost or increase corpus scale. Change2Task is complementary: it begins after a healthy base is executable and increases the verified, developer-grounded task supply obtained from that investment.

Historical benchmarks ground tasks in developer issues, patches, and tests. Defects4J, SWE-bench, and SWE-Gym preserve realistic outcomes at original revisions (Just, Jalali, and Ernst 2014; Jimenez et al. 2024; Pan et al. 2025); SWT-Bench emphasizes tests that distinguish faulty and repaired states (Mündler et al. 2024). R2E-Gym and SWE-Synth scale through mutation, model generation, and executable feedback (Jain et al. 2025; Pham et al. 2026), but generated tasks need not match a maintenance intent observed in the repository.

The closest method, SWE-smith’s PR Mirror, asks a model to undo a historical PR in a current repository and retains candidates that break target tests (Yang et al. 2025b). FeatureBench removes tested feature units to construct feature-development tasks (Zhou et al. 2026). Both demonstrate task generation inside prepared repositories, but emphasize one construction operator or task objective. A broader construction system must adapt to repository evolution, preserve the historical maintenance contract, and support different agent outputs and oracles.

Change2Taskreconstructs a real maintenance event on a maintained descendant of the same repository. Shared lineage and target, regression, restoration, scope, and source change profile checks connect the modern task to its source PR. One validation core then serves multiple objectives without rebuilding the base environment.

Change2Taskderives task evidence from a merged PR in the repository’s own history. Its description, implementation patch, and executable checks identify the changed condition, the checks that observe it, and nearby behavior that must remain intact. This stage produces target checks, regression checks, and a source change profile. Target checks expose the condition to be reconstructed; regression checks protect surrounding behavior; the source change profile records the affected components and edit extent.

Candidates are excluded when the changed condition is ambiguous, its checks cannot be executed, or the relevant change cannot be separated from unrelated modifications. The retained evidence provides provenance and the source change profile for later validation.

For each eligible PR, Change2Task selects a fixed descendant revision from the same repository lineage as the healthy modern base $H$. It first honors a declared healthy or target revision. If neither is available, it resolves the accessible upstream default branch in a fixed priority order. A fallback to the historical base is recorded separately and is not counted as a modern base. Selection is completed before task construction, and the resolved commit hash is frozen for reproducibility.

The selected revision must support a clean checkout, its runtime dependencies and services, and executable checks. It must also still host the relevant behavior: the mapped target and regression checks must pass on $H$, and there must be a defensible transformation surface in the modern code. Native repository configuration or an external preparation tool may establish this environment; Change2Task operates after the base is runnable. Appendix C gives the full resolution order and acceptance gates.

PR alignment locates the modern realization of the changed behavior, implementation, and checks despite refactoring, file movement, API changes, or test reorganization. A candidate advances only when this correspondence is sufficiently grounded and both target and regression checks pass on $H$.

This stage also instantiates a task specification: the goal presented to the agent, the expected output, the validation oracle, and the permitted edit scope. Our evaluation adapters create an unfixed behavior for Bug Fix, a missing capability for Feature Addition, an observable failure for Test Generation, old API usage for API Migration, or a sandboxed vulnerability for Security Repair. These adapters vary the objective and oracle while sharing one construction and validation core. Additional task families can use the same core whenever their task condition, output contract, and executable oracle can be derived from historical PR evidence on a healthy descendant revision.

Given the task evidence, healthy modern base, and task specification, Change2Task constructs $D$ so that $C=\mathrm{apply}(H,D)$ presents the intended maintenance condition while preserving unrelated behavior. Figure 3 expands the three construction levels. Construction escalates only when an earlier level cannot produce a valid candidate.

Level 1: Patch Reversal. Level 1 reverses the source patch when it remains compatible with the modern context. This route preserves the closest structural link to the change written by the developer.

Level 2: Code Mapping. When direct replay fails but a unique source-block correspondence remains, Level 2 locates the historical post-change block in the modern file. It allows indentation normalization and replaces the block with its reindented pre-change form. The method favors conservative local transformations, reparses the result, and requires the mapped patch to pass the same scope and executable gates. Level 2 makes one structure-guided attempt. Missing, ambiguous, or incompatible correspondence advances the case to Level 3.

Level 3: Agent Reconstruction. When the historical behavior remains present but structural correspondence is insufficient, a construction agent receives PR evidence, relevant modern context, the task specification, and feedback from prior attempts. It proposes scoped candidate patches and filters them through apply, syntax, and scope checks, then ranks survivors by source change profile fidelity and edit complexity before lifecycle validation. A failed candidate returns structured evidence identifying an unmet target condition, broken regression check, restoration failure, or fidelity deviation. The loop allows at most four attempts and terminates at acceptance or budget exhaustion. Cases without a defensible modern behavior host are rejected, and downstream agent outcomes never select or revise tasks. Appendix C gives Level 2 pseudocode and the complete Level 3 protocol.

Patch checks reject candidates that fail to apply, exceed the permitted scope, or break basic syntactic and build integrity. Complexity matching compares their files, hunks, changed lines, symbols, and check surface with the source change profile. For an accepted task patch $D$, Change2Task records a restoration patch $G$: it is derived as the inverse transformation when possible, otherwise constructed as the corresponding modern repair, and must restore the intended behavior from $C$ to $H^{\prime}$.

Change2Taskvalidates each candidate over three executable states. On healthy modern base $H$, target and regression checks pass. On task state $C$, target checks expose the specified task condition while regression checks continue to pass. After applying $G$, both check sets pass on restored state $H^{\prime}$. The target checks therefore pass, fail, and pass across the three states, while regression checks pass throughout. Repeated runs reject unstable cases.

The scope gate permits only artifacts allowed by the task specification and prevents protected checks or metadata from manufacturing validity. The fidelity gate compares the source PR with the modern realization using affected components, edit extent, and check surface. It rejects scope collapse, in which a substantial source change becomes trivial, and scope inflation, in which a narrow change requires broad unrelated edits.

An accepted variant contains $H$, $D$, $G$, the target and regression checks, the task specification, and source PR provenance. Only variants that pass lifecycle, scope, and fidelity validation are frozen for evaluation. The task families therefore demonstrate different agent goals and scoring rules over one reusable construction core.

We collect 1,130 construction-eligible changes from 12 public benchmark collections and releases (Jimenez et al. 2024; Zhang et al. 2025; Deng et al. 2025; Li et al. 2025; Badertdinov et al. 2025; Wang et al. 2025a; Islam et al. 2023; Wei et al. 2025; Bui, Scandariato, and Díaz Ferreyra 2022; Wu et al. 2023). We define verified task recovery as the fraction reaching the finalized corpus after lifecycle, scope, fidelity, and semantic qualification. Appendix E reports exact counts and Wilson 95% confidence intervals (CIs).

Figure 4(a) shows that Change2Task reconstructs 900 of 1,130 eligible changes, yielding 79.6% overall recovery. Family estimates range from 76.3% for API Migration to 82.0% for Feature Addition, with wider CIs for the four smaller families.

Figure 4(b) assigns each finalized task to its first successful construction level. Level 1 contributes 95 tasks directly; Level 2 contributes 190 after Level 1 fails; Level 3 contributes 615 after both earlier levels are exhausted. The shares are 10.6%, 21.1%, and 68.3%, quantifying the incremental coverage supplied by each level.

Figure 4(c) assigns each of the 230 rejected candidates one terminal reason. Lifecycle validation accounts for 50 cases, environment or preflight failures for 44, and exhausted candidate construction for 42. Attrition is therefore distributed across base readiness, task construction, executable validation, and final qualification.

Table 1 compares methods on the same 621 Bug Fix candidates under one construction and verification envelope. Direct reversal recovers 81 tasks and SWE-smith PR Mirror 387 (Yang et al. 2025b); Change2Task recovers 500. It therefore adds 113 verified tasks, a 29.2% relative and 18.2-point absolute gain. Appendix C specifies the matched model, evidence, context, attempts, tools, and budgets.

RQ2 examines how well the 900 finalized tasks retain the complexity, scope, and maintenance meaning of their source changes. We compare each historical source patch with its modern restoration patch along six dimensions: changed files, hunks, lines, symbols, target checks, and regression checks. Dimension similarity is the ratio between smaller and larger counts, with identical empty counts assigned 1. Their weighted combination emphasizes changed lines, followed by hunks, files, the two check surfaces, and symbols.¹¹1Weights allocate 76% to implementation footprint and 24% to verification surface. Appendix F reports component values and an equal-weight check.

Figure 5(a) reports family means from 0.825 to 0.925, with a macro average of 0.887 and task-weighted average of 0.894. Test Generation scores 0.925, Bug Fix 0.903, Feature Addition 0.893, API Migration 0.890, and Security Repair 0.825. The connected profiles show that the aggregate reflects multiple aspects of the source change rather than one size statistic.

Figure 5(b) reports mean fidelity of 0.992 for Patch Reversal, 0.888 for Code Mapping, and 0.881 for Agent Reconstruction. These values characterize finalized tasks as code correspondence becomes weaker; route assignment is descriptive rather than randomized.

Kimi K3 and DeepSeek V4 Pro independently review 912 anonymized historical and modern pairs without seeing the construction route, trace, or model. The review records behavioral triggers, expected behavior, affected APIs, source-to-modern mappings, contrastive checks, and restoration. Both judges directly accept 834 pairs. Manual adjudication retains 66 of the remaining 78 and excludes 12, producing 900 tasks. Figure 6 reports each judge’s rate, direct joint acceptance, and inter-judge agreement; 92.7% of the final corpus is directly accepted by both judges. Appendix F gives the labels and adjudication protocol.

The audit also defines the applicability boundary. An adapter needs a historical PR with identifiable intent, a healthy descendant hosting the behavior, an executable oracle, and a bounded edit policy. Outputs may include code, tests, configuration, or other artifacts; grounded evidence and executable validation matter more than a fixed task taxonomy.

The Original Branch is the historical behavioral anchor, and the Change2Task Branch realizes the same maintenance intent on modern code. Table 2 reports branch solve rates, directional differences, complete paired outcomes, raw agreement, and Cohen’s $\kappa$. Agreement is the fraction with the same solved or unsolved outcome; $\kappa$ adjusts for the branches’ marginal solve rates.

Across 3,600 matched agent-task pairs, each branch contains 1,478 solved outcomes, or 41.1%. Discordance is exactly balanced: 186 outcomes are solved only on the Original Branch and 186 only on the Change2Task Branch. Aggregate agreement is 89.7%, Cohen’s $\kappa$ is 0.787, positive agreement is 87.4%, and solved-set Jaccard overlap is 77.6%. The paired solve-rate difference is 0.0 points with a 95% interval of [-1.1, 1.1], and exact McNemar $p=1.0$. Appendix G defines these statistics.

Agent-level behavior is stable. Agreement ranges from 86.4% for Copilot with GPT-5.6 Terra to 95.3% for Gemini 3.1 Pro, with aggregate $\kappa$ from 0.729 to 0.852. Gemini 3.1 Pro combines the highest agreement with lower solve rates. Copilot with GPT-5.6 Terra and Claude Code Sonnet 5 have the highest overall solve rates, while Codex with GPT-5.5 performs best on Test Generation and API Migration.

Directional changes remain small and dispersed. Claude Code Sonnet 5 and Copilot with GPT-5.6 Terra gain 12 and 10 solved tasks on the Change2Task Branch, Codex with GPT-5.5 changes by four, and Gemini 3.1 Pro loses 18. No task-family shift exceeds five percentage points, and 11 of 20 comparisons differ by at most two points. Reconstruction therefore does not systematically favor one branch, configuration, or task family.

RQ4 measures the systems benefit of deriving multiple finalized tasks from a healthy modern base. The matched baseline prepares and retains one environment for each of 900 tasks. Change2Task instead prepares 388 healthy modern bases and reuses them across associated variants while accounting for task-specific materialization and verification. Both conditions use the same hardware, retention period, and accounting rules.

Table 3 contrasts rebuilding an environment for every task with reusing 388 healthy modern bases. Reuse lowers amortized environment time from 3.60 h to 1.50 h and storage from 6.20 GB to 1.79 GB per task. Figure 7 reports family-level reductions. Bug Fix achieves the greatest reuse and savings; API Migration currently maps each task to a distinct base and benefits less.

Benefits depend on reuse density: families placing several tasks on one maintained base save more than those using distinct bases. Matched hardware, prices, and retention isolate this reuse effect, while RepoLaunch shows how modern reconstruction and historical preparation combine when either route alone is insufficient.

Prior systems report costs over different outputs and accounting boundaries (Yang et al. 2025b; Fu et al. 2026; Chen et al. 2026), so our cost conclusion uses the matched internal comparison. Reuse lowers expenditure from $1,917 to $1,710. The $207 saving equals $0.23 per task and a 10.8% net reduction, including additional automation and Agent Reconstruction expense. Appendix H gives the accounting breakdown.

RepoLaunch supplies a complementary path when modern reconstruction is unavailable. It makes another 195 historical revisions executable, raising coverage from 900 to 1,095 of 1,130 cases, or 96.9%, and leaving 35 unresolved. It is an environment-preparation complement rather than a task-construction baseline.