EvoSCM: Scientific Belief Revision Through Causal Model Evolution and Experimentation

Figure 6 zooms into a single revision event to show how the four stages of EvoSCM’s loop jointly recover the finite core from Figure 4. (A) After identifying the screened Yukawa mechanism, the surviving hypotheses agree in the far field but diverge at short range: predicted accelerations differ by $2.39\times$ at $r^{*}=0.01$, making this the maximally discriminative intervention point. (B) EvoSCM selects a paired-CRN off-axis intervention over a multi-radius scan for its higher information score and lower cost ($1.080$ vs. $0.830$; 2 vs. 4 episodes). (C) The finite-core hypothesis produces a substantially smaller residual than the vanishing-core alternative ($4.12\sigma$ vs. $12.04\sigma$), enabling decisive model selection. (D) The resulting revision inserts a softened distance $\rho_{t}=\sqrt{r_{t}^{2}+\varepsilon^{2}}$, replacing the direct edge $r_{t}\to S_{t}$ with $r_{t}\to\rho_{t}\to S_{t}$ while preserving the learned screening kernel and coupling structure.

Figure 7 illustrates the evolution of EvoSCM’s task-specific epistemic model for solid-state electrolyte discovery on LLEMA (Abhyankar et al., 2026). At Stage A, after 16 oracle-evaluated observations, EvoSCM forms its first actionable hypothesis: higher halogen content and coordination adaptability are associated with lower energy above the convex hull, while higher mean electronegativity favors formation energy and lattice anisotropy introduces an energetic penalty. Four additional experiments, including informative chemical counterexamples, lead to Stage B, where unsupported relations are revised and composition-related mechanisms are promoted: metallic fraction becomes positively associated with hull energy, while electronegativity contrast complements mean electronegativity in explaining formation energy. After another four boundary-refinement experiments, Stage C integrates elemental diversity, coordination variation, halogen content, electronegativity statistics, and lattice anisotropy into a multi-factor stability hypothesis. The model continues to abstain from positing band-gap relations because their estimated reliability remains below the acceptance threshold, demonstrating calibrated epistemic restraint rather than forcing a complete explanation. These structures should be interpreted as local, intervention-guiding epistemic relations learned within the frozen ALIGNN evaluation environment, not as recovery of universal causal laws for solid-state electrolytes.

Figure 8 illustrates how EvoSCM revises a mechanistic hypothesis on ActiveSciBench-GRN (Kabra et al., 2026). With only four initial observations, the coherent feed-forward model is preferred over the incoherent alternative (fit score $0.061$ vs. $0.104$, lower is better), leading to the incorrect hypothesis that $B$ activates $C$. EvoSCM then selects continuous soft interventions in regions where the candidate SCMs disagree most strongly, jointly varying signal and perturbation multipliers rather than changing a single variable in isolation. The resulting observations increasingly contradict the coherent explanation: for example, in two selected high- and low-$B$ regions, the observed reporter values are $1.61$ and $115.25$, compared with coherent-model predictions of $6.69$ and $24.56$ and incoherent-model predictions of $0.60$ and $78.25$, respectively. After 16 observations, the working hypothesis revises the critical relation from $B\!\rightarrow\!C\,(+)$ to $B\!\dashv\!C\,(-)$, yielding an antagonistic feed-forward mechanism in which $A$ activates $C$ directly while also activating its suppressor $B$. Following four additional boundary-testing experiments, the conditional-Hill cross-validation procedure selects the complete signed graph $\{\mathrm{signal}\!\rightarrow\!A,\ A\!\rightarrow\!B,\ A\!\rightarrow\!C,\ B\!\dashv\!C\}$ with a MSE of $3.36\times 10^{-27}$, compared with $0.283$ for the coherent alternative. Because coherent and incoherent motifs both belong to the public candidate grammar, this case demonstrates active discrimination, sign revision, and validation among competing biological mechanisms rather than unconstrained de novo discovery.

The graph $G_{k}^{t}$ is stored as an edge list of directed causal relationships. Each variable is annotated with its role (state, parameter) and observability, enabling the Revision step to identify which variables can be added or removed:

"variables": [
  {"name": "r_t", "role": "state", "observed": true},
  {"name": "C", "role": "parameter", "observed": false}
],
"edges": [
  {"source": "r_t", "target": "a_t"},
  {"source": "p1", "target": "a_t"}
]

Each mechanism $f_{i}\in\mathbf{F}_{k}^{t}$ is stored as a symbolic expression string together with its parent variables:

"mechanisms": [{
  "target": "a_t",
  "parents": ["r_t", "p1", "p2"],
  "equation": "a_t = -C*(p1/p2)*r_vec/
             (r**2 + eps**2)"
}]

Storing the symbolic expression alongside its declared parents allows the LLM to reason about functional dependencies and supports direct numerical evaluation during the Prediction step.

Parameters $\bm{\Theta}_{k}^{t}$ are stored with point estimates and optional bounds:

"parameters": {
  "C": {"estimate": 0.1585,
        "lower": 0.155, "upper": 0.163}
}

Separating parameters from functional forms enables the Update Parameter operator to adjust quantitative values without modifying the structural equations.

Beyond the formal SCM tuple, each hypothesis carries metadata that supports the discovery loop: a natural-language description summarizing the causal explanation, a list of assumptions (e.g., "isotropic", "static"), falsification conditions that would trigger revision, the hypothesis status (active or pruned), and a revision counter tracking how many times the hypothesis has been edited. This metadata is not part of the formal SCM but provides context that helps the LLM generate more targeted revisions and more interpretable explanations. A complete example of the JSON representation is provided in Listing 1.

The population $\mathcal{P}_{t}=\{\mathcal{H}_{1}^{t},\ldots,\mathcal{H}_{K}^{t}\}$ is maintained as an indexed collection of JSON objects. At each round, the serialized population and the accumulated evidence $\mathcal{D}_{t}$ are provided to the LLM so that inter-hypothesis comparisons (e.g., prediction divergence for causal experiment design) can be performed within the same call. In practice, the population sizes used in our experiments ($K\leq 8$) remain well within the context limits of all evaluated LLM backbones.

The LLM receives each hypothesis $\mathcal{H}_{k}^{t}$ with the accumulated evidence and is instructed to infer latent variable values consistent with the observed data:

Given the following SCM hypothesis and experimental evidence, infer the values of the latent exogenous variables that best explain the observed outcomes. For each latent variable, provide a point estimate and a brief justification based on the structural equations and evidence.

The LLM receives the full population $\mathcal{P}_{t}$ and is instructed to select the intervention that maximally discriminates between competing hypotheses:

You have $K$ competing SCM hypotheses. Design an intervention $\mathrm{do}(\mathbf{X}=\mathbf{x})$ that maximizes the disagreement among their predicted outcomes. Identify the variable(s) to intervene on and the value(s) to set, prioritizing interventions where different hypotheses make the most divergent predictions.

For each hypothesis $\mathcal{H}_{k}^{t}$, the LLM derives the predicted outcome using that hypothesis’s structural equations and abduced latent states. The prompt enforces pre-commitment:

Using the structural equations and parameters of this hypothesis, and the latent variable values inferred during abduction, compute the predicted outcome of the intervention $\mathrm{do}(\mathbf{X}=\mathbf{x})$. Commit to a specific numerical prediction before observing the experimental result.

The LLM receives each hypothesis’s committed prediction alongside the observed outcome and is instructed to identify systematic patterns of failure:

Compare the predicted outcome with the experimental observation. Identify any systematic discrepancy and distill it into a concise correction rule describing the empirical pattern that the current hypothesis fails to capture (e.g., “the relationship between X and Y follows an inverse-square law rather than a linear law”).

The LLM receives the correction rules and is instructed to apply targeted structural edits. The prompt explicitly enumerates the available operators:

Based on the correction rule, revise the SCM hypothesis using one or more of the following operators: (1) Add/Remove Edge: insert or remove a directed edge; (2) Add/Remove Latent: introduce or remove a latent variable; (3) Update Mechanism: change the functional form of a structural equation; (4) Update Parameter: adjust the numerical value of an existing parameter. Apply the minimal set of operators sufficient to accommodate the correction rule. Prefer shallow revisions (parameter and mechanism updates) when possible, resorting to structural edits only when the evidence warrants them.

The LLM receives each revised hypothesis and performs two validation checks:

Validate the revised hypothesis: (1) Evidence Validation: derive predictions for all past interventions in the accumulated evidence and verify that the revised hypothesis accounts for previous observations without introducing new contradictions. (2) Consistency Check: verify that the causal graph is a valid DAG, all structural equations are well-defined given the revised graph, and the posited mechanisms are mutually compatible. Report whether the hypothesis passes both checks.

DiscoverPhysics (Wiemann et al., 2026) is an interactive benchmark for discovering hidden laws of motion in simulated worlds whose physics deliberately departs from familiar physical assumptions. The complete benchmark comprises 22 curated worlds, including 11 publicly released worlds and 11 private worlds. Table 12 summarizes the public worlds, which cover unusual force laws, hidden sources and particle types, preferred spatial directions, and time-dependent interactions.

A discovery session begins with initial trajectory observations and an experimental protocol, without revealing the underlying force law or hidden particle types. Each experiment specifies controllable particles through their initial positions, initial velocities, generalized charges, and requested measurement times. The simulator evolves these particles together with any existing particles and returns the requested positions and velocities. Observations are generated on demand rather than retrieved from a fixed dataset. The interface also supports fitting the free parameters of a proposed law against previously collected trajectories. Observation noise is configurable, whereas the held-out trajectories used for prediction evaluation are noise-free.

The final submission consists of a natural-language explanation and an executable Python implementation of the inferred dynamics. Predictive accuracy is evaluated on held-out particle trajectories. For a world $w$, let $\mathcal{I}_{w}$ denote the evaluated particle–time pairs, and let $\mathbf{r}_{i}(t)$ and $\widehat{\mathbf{r}}_{i}(t)$ denote the reference and predicted positions, respectively. The normalized trajectory error is

where $V_{w}$ is the benchmark’s reference trajectory variance for that world. Normalization accounts for differences in the natural motion scales of different worlds, and trajectory errors are aggregated geometrically across worlds.

Explanation quality is evaluated separately by an LLM judge using an expert-written, world-specific rubric, with scores reported on a normalized scale. Discovery success requires both sufficient predictive accuracy and sufficient explanation quality. The $\mathrm{pass}@k$ metric measures the expected fraction of worlds with at least one successful discovery among $k$ independent attempts.

LLEMA (Abhyankar et al., 2026) introduces a suite of 14 application-driven materials-discovery tasks, publicly released as LLEMABench. Here, LLEMA refers to this benchmark suite, rather than the evolutionary method introduced in the same work. Each task specifies an application goal together with numerical property constraints and, where applicable, compositional requirements. The objective is to identify candidate materials satisfying these requirements, rather than to recover a unique reference equation or causal graph. The released task specifications are summarized in Table 13.

Candidates are represented by chemical compositions and crystallographic information files (CIFs), which describe lattice geometry, symmetry, and atomic sites. The evaluation pipeline combines available Materials Project data with pretrained property predictors, including CGCNN and ALIGNN, to obtain task-relevant material properties. These include electronic, mechanical, dielectric, and thermodynamic quantities, with the required property set determined by the task. Candidate validity is established by checking the resulting properties and composition against the task specification.

The benchmark therefore evaluates computationally screened materials. Its property estimates and stability assessments should not be interpreted as direct experimental measurements or confirmation of successful synthesis.

Hit rate measures the percentage of generated candidates satisfying all task-specific constraints. The stability metric measures the percentage that are both task-valid and classified as thermodynamically stable. For $M$ generated candidates, let $v_{i}\in\{0,1\}$ indicate whether candidate $i$ satisfies the task constraints, and let $s_{i}\in\{0,1\}$ indicate whether it passes the stability screen. The two metrics, expressed as percentages, are

Thus, the stability metric uses all generated candidates as its denominator, rather than only the valid subset. The original benchmark reports an energy-above-hull threshold of $0.1$ eV/atom for thermodynamic stability screening. It additionally uses Pareto-front analysis to assess trade-offs among competing material properties.

ActiveSciBench-GRN, introduced with LLM-AutoSciLab (Kabra et al., 2026), evaluates the recovery of hidden gene-regulatory networks through active perturbation experiments. The reference suite comprises five regulatory motif families, three benchmark versions, and three difficulty levels, yielding $5\times 3\times 3=45$ task configurations per random seed. Table 14 summarizes the motif families. The discovery target is a signed directed graph specifying which regulatory interactions exist, their directions, and whether they are activating or repressing. The inclusion of feedback motifs means that the target graphs are not restricted to directed acyclic graphs.

An experiment sets an upstream signal and multiplicative perturbations of nodes $A$, $B$, $C$, and $R$. Perturbation factors below or above one represent knockdown or overexpression, respectively; one denotes the reference setting. The public simulator returns a reporter response and a marker panel for these nodes. Responses are generated by motif-specific nonlinear regulatory mechanisms rather than retrieved from a fixed observational dataset.

Difficulty is controlled through nonlinear response regimes. Easy instances exhibit approximately linear responses; medium instances introduce saturation that can obscure weak regulatory effects; and hard instances emphasize stronger nonlinearities and, for the relevant motifs, feedback-dependent switching behavior. The governing graph and dynamical parameters are withheld from the learner.

Let $\mathbf{A}^{\star},\widehat{\mathbf{A}}\in\{-1,0,+1\}^{d\times d}$ denote the reference and predicted signed adjacency matrices, where $+1$, $-1$, and $0$ represent activation, repression, and absence of an edge, respectively. Define the corresponding signed edge sets $\mathcal{E}^{\star}$ and $\widehat{\mathcal{E}}$, whose elements are $(\text{source},\text{target},\text{sign})$ triples. The public evaluator computes

with undefined ratios assigned zero. Thus, an edge is counted as correct only when its direction and sign both match.

Sign accuracy measures sign agreement over directed edges present in both graphs and is zero when this overlap is empty. Exact graph accuracy requires equality of the complete signed adjacency matrices. The evaluator additionally provides motif accuracy, measuring whether the predicted motif-family label is correct.

The SCM maintained by EvoSCM is an epistemic model of the environment, not a copy of the simulator’s ground-truth causal program. For each hypothesis $\mathcal{H}_{k}^{t}=(\mathbf{V},\mathbf{U}_{k}^{t},G_{k}^{t},\mathbf{F}_{k}^{t},\bm{\Theta}_{k}^{t})$, the variable set is partitioned as

In two-particle environments, $\mathbf{V}_{\mathrm{obs}}$ contains particle positions $\mathbf{x}_{t}$, velocities $\mathbf{v}_{t}$, and the observed trajectory; $\mathbf{V}_{\mathrm{act}}$ contains the public control scalars $p_{1}$ and $p_{2}$, the initial position and velocity of the mobile particle, and, when available, the absolute start time. The hypothesis may introduce derived quantities $\mathbf{V}_{\mathrm{der},k}^{t}$ such as the separation $r_{t}$, radial direction $\hat{\mathbf{r}}_{t}$, field gradient $\nabla\phi_{t}$, and acceleration $\mathbf{a}_{t}$. The semantic roles of $p_{1}$ and $p_{2}$ are not hard-coded: competing hypotheses may interpret them as source strength, response inertia, charge, or nuisance controls, and these interpretations are resolved through interventions.

The latent variables $\mathbf{V}_{\mathrm{lat},k}^{t}$ represent unobserved physical causes proposed by the LLM, such as hidden source locations, particle species, screening mechanisms, or time-dependent couplings. Hypothesis-specific parameters $\bm{\Theta}_{k}^{t}$ include quantities such as a coupling constant $G$, radial exponent $q$, screening length $\lambda$, and numerical regularization $\epsilon$. These variables and parameters are inferred from observations; they are not populated from the benchmark’s hidden ground truth.

For population environments, the observed variables instead contain positions and velocities of visible particles and probes. Action variables include probe positions, velocities, and masses, depending on the environment topology. Hidden dark sources and latent species assignments may appear in the SCM but are not directly manipulable by the agent.

An intervention $\mathrm{do}(\mathbf{X}=\mathbf{x})$ is implemented by assigning one or more fields of the public experiment schema before the trajectory is generated. EvoSCM cannot intervene on hidden physical laws, hidden simulator parameters, or unexposed latent state. Table 15 summarizes the available action spaces by environment topology.

EvoSCM supports both ordinary and paired-counterfactual experiments. For an ordinary intervention, the selected input is passed directly to the simulator. For a paired counterfactual, the agent specifies a factual input $\mathbf{x}$ and a set of sparse assignments $\{\delta_{j}\}$. The runtime evaluates

and returns both branch outputs and paired differences $\mathbf{y}^{(j)}-\mathbf{y}^{(0)}$. All branches share the same hidden environment and a restored random-number state, so paired differences use common random numbers and isolate the declared intervention’s effect. The factual rollout and each counterfactual branch consume one simulator episode.

At each round, the LLM proposes two to four candidate experiment designs together with pre-committed predictions from the competing hypotheses. EvoSCM executes only the feasible design with the largest posterior-weighted predictive disagreement, ensuring that interventions are selected using predictions made before observing the corresponding outcome.

The maximum population size is $K_{\max}=6$. At the first round, the LLM initializes between three and six structurally distinct hypotheses with uniform evidence weights, irrespective of any confidence values reported by the LLM, to avoid favoring a familiar textbook law before any experiment has been performed.

Each session runs for at most $T_{\max}=16$ LLM interaction rounds and may terminate earlier when the agent submits its final executable law. Since one counterfactual design can contain multiple simulator branches, the number of LLM rounds and simulator episodes are recorded separately. In paired comparisons, EvoSCM uses the same world, noise seed, LLM backbone, explanation judge, and maximum round count as the matched baseline session. Its simulator-episode budget is additionally capped by the number of episodes consumed by that baseline session.

We evaluate with three LLM backbones: GPT-5.4 (OpenAI, 2026b), GPT-5.5 (OpenAI, 2026c), and GPT-5.6-Sol (OpenAI, 2026a). The maximum output length is 8,192 tokens per call. Evaluation results and reference explanations are not exposed to the discovery agent.

The system prompt is assembled from three components: (i) the benchmark-wide interactive discovery template, (ii) a topology-specific instruction file, and (iii) the EvoSCM protocol. The topology-specific instructions are inserted into the master template, after which the EvoSCM protocol is appended. At each round, the runtime supplies a compact JSON state containing the current hypothesis population, posterior weights, recent prediction errors, the latest abduction, induction and deduction records, completed intervention coverage, and remaining budget. The LLM returns structured blocks: <scm_update> for abduction and hypothesis revision, <design_experiments> for candidate interventions and pre-committed predictions, and <final_law>, <explanation>, <scm_final> at termination. Complete prompt templates and schemas are included in the released code.

The EvoSCM loop is unchanged across environments, but the following adaptations tailor evidence collection to physical-law discovery.

Causal-invariance audits. For two-particle environments, the runtime tracks a set of systematic audits rather than trusting the most familiar initial law. These include a null-source intervention, controlled magnitude changes in $p_{2}$, sign reversals of $p_{1}$ and $p_{2}$, a multi-scale radius scan (at least four noise-cancelled responses spanning at least an eight-fold radius range), and, when absolute time is exposed, a phase sweep over start_time. The scale audit allows the agent to distinguish a constant power law from Yukawa screening or a short-distance dimensional crossover.

Paired counterfactuals for noise cancellation. Paired counterfactuals cancel common observation noise and estimate local causal responses. The runtime derives empirical quantities (source and response-control exponents, local radial slopes, sign relations, absolute-time responses) exclusively from experiments observed by the agent. These quantities are returned as evidence summaries; no hidden world label or ground-truth equation is used.

Topology-specific action schemas. For population environments, EvoSCM reasons about latent source populations through controlled probe placements, velocities, and masses rather than intervening on hidden particles. For the circle environment, experiments vary collective ring-level initial conditions instead of individual two-particle controls.

Parameter calibration. Continuous parameters can be calibrated using the benchmark’s trajectory-level MSE-fitting tool on trajectories collected during discovery. The final output is an executable Python law, and the runtime checks its implied direction, control scaling, radial dependence, and time dependence against observation-derived causal evidence before accepting the submission.

For each task $\tau$, EvoSCM maintains an epistemic SCM over

where $\mathbf{X}_{\mathrm{mat}}$ contains composition and structure descriptors and $\mathbf{Y}_{\tau}$ contains the target properties required by the task. The causal direction is fixed by domain knowledge:

EvoSCM therefore does not search over arbitrary causal DAGs in this domain. Instead, it evolves the active descriptor set, edge signs and magnitudes, uncertainty estimates, and the relative support for competing local mechanisms.

The descriptor variables include structural quantities (volume per atom, density, lattice anisotropy, minimum pair distance, nearest-neighbor statistics), compositional quantities (composition entropy, elemental fractions, electronegativity range), and atomic-level statistics (mean and standard deviation of atomic number, electronegativity, mass, and radius). All descriptors are deterministically computed from the generated crystal using Pymatgen. For every task, energy above hull ($E_{\mathrm{hull}}$) is included as an additional stability outcome even when it is not a primary objective. Table 16 shows the task-to-property mapping.

We maintain $K=7$ competing SCM hypotheses (not to be confused with the number of material proposals). Each hypothesis is a bootstrap sparse ridge mechanism fitted independently for each property outcome. Features are standardized and screened per property, with a ridge coefficient of 2.0. Bootstrap resampling produces seven competing signed mechanisms; their median coefficient, sign agreement, and coefficient variance define the ensemble-level edge estimate and uncertainty.

The property-wise mechanisms are complemented by three additional components: (i) a calibrated LLM chemistry prior, (ii) non-parametric local mechanisms for the binary hit and stability outcomes, and (iii) a surrogate-support counterexample frontier. The numerical SCM starts with a reliability prior of 0.15, while the LLM chemistry prior is initialized at 0.25. Reliability is updated using pre-committed prediction error and parent-child delta-sign accuracy. Outcome-specific edges are withheld from the planner when their calibrated reliability falls below 0.20, allowing the system to retain a partially specified SCM rather than forcing unsupported edges.

EvoSCM does not directly apply interventions such as $\mathrm{do}(\texttt{halogen\_fraction}=x)$, because modifying a material generally changes several descriptors simultaneously. Instead, the action variable is a chemically explicit transformation

where each proposed action records its type, changed factor, source and target values, and parent formula, making the intervention auditable. Supported transformations include:

• •

  elemental substitutions (same-group, oxidation-state-preserving, or chemically similar);

• •

  stoichiometric and redox-aware composition changes;

• •

  crystal-prototype-preserving or prototype-changing interventions;

• •

  canonical polymorph realization at fixed composition;

• •

  isotropic lattice strain, polar displacement, and coordination changes at fixed composition;

• •

  stability-boundary, Pareto-frontier, and controlled-disagreement experiments.

A deterministic candidate compiler converts each intervention intention into an evaluable crystal. The compiler can normalize the formula, ground a composition in a common prototype, generate bounded structural counterfactuals, and repair geometry errors. It never queries the property oracle. Each compiled candidate retains its source and all repair actions, so compiler transformations remain distinct from oracle-derived evidence.

We use $T=5$ adaptive rounds. A shared initialization requests eight candidates from the LLM and evaluates four diverse candidates. In each subsequent round, the LLM proposes $K_{\mathrm{proposal}}=8$ materials, the compiler constructs at most $K_{\mathrm{compiled}}=40$ pre-oracle candidates, and the policy evaluates $B=4$ candidates. Each task-seed session therefore uses $B_{0}+TB=4+5\times 4=24$ oracle episodes and five adaptive LLM calls. Compiler-generated candidates that are not selected do not consume oracle budget.

The active memory retains at most six success and six failure exemplars in the prompt, while the fitted SCM uses all valid observations. Five deterministic island labels provide provenance and diversity tracking. The formula exclusion memory contains at most 96 previously tested formulas.

The main comparison uses Qwen3.6-35B-A3B (Qwen Team, 2026) with reasoning mode disabled, a maximum output length of 12,288 tokens, and strict JSON schema enforcement. The transfer experiment uses GPT-5.6-Luna (OpenAI, 2026a) with the same configuration: reasoning mode disabled, the same token limit, and the same budget.

After structural validation, each candidate is serialized as a CIF and evaluated on the properties required by the current task plus $E_{\mathrm{hull}}$. All methods use a single frozen ALIGNN oracle version. Table 17 lists the property models.