Stage-Replay Divergence Follows the KV Cache
Fixed-Prefix Precision Controls and Bidirectional Cache Transplantation

LLM outputs depend on the execution contract even under nominally deterministic decoding. Batch size, GPU count, and hardware can alter responses and benchmark scores because early rounding differences cross later decision boundaries (Yuan et al., 2025). Recent systems address this dependence through different mechanisms: TBIK aligns reduction orders across tensor-parallel configurations, whereas MarginGate verifies low-margin BF16 steps and repairs confirmed mismatches in the current K/V column (Zhang et al., 2026; Chu et al., 2026).

Two studies are especially close. Chodavarapu and Xu (2026) compare cache-on and cache-off inference and report deterministic FP16 trajectory divergence that disappears behaviorally under an FP32 falsification. Their cache-off arm recomputes the full sequence at every step and therefore differs from our live-stage versus one-shot-prefill comparison, but it establishes the relevant precision sensitivity. They intervene on the residual stream and propose direct intervention on cached K/V tensors as future work; our experiment implements that intervention class at a whole-stage boundary. Matteson (2026) compare continuations resumed from verified decode-time KV blocks, a second exact-resume pass, and fresh re-feeding for counterfactual token-credit estimation. Re-feeding changes estimates above the replica floor; a batch-invariant vLLM confirmation makes all stored channels identical in the tested configuration.

We do not claim the first demonstration that re-feeding differs from state resumption. Our extension operates at a whole reasoning-stage boundary, tracks long greedy trajectories and paired task correctness, fixes identical prefixes across precision, and intervenes directly on every boundary K/V tensor. The matched live-cache matrix is closest to the three-pass logic of Matteson (2026); the fixed-prefix crossing and bidirectional transplants ask which state carries the resulting whole-stage trajectory. We do not provide their batch-invariant causal closure.

Table 1 states the overlap and separation directly. The novel unit is not re-feeding or precision sensitivity by itself; it is their combination in a fixed-token, whole-stage task design with live and replay replica floors, a prospective live/incremental bridge, direct bidirectional KV transplantation, and an outcome-blind checkpoint replication.

Self-consistency samples diverse reasoning paths and selects an answer by marginalizing over them (Wang et al., 2023). Branch–Solve–Merge decomposes a task into parallel subtasks and fuses their solutions (Saha et al., 2024). Recent systems move beyond answer-only voting by training an aggregator on concatenated sample sets (Qi et al., 2025), recursively refining populations of reasoning chains (Venkatraman et al., 2025), synthesizing solutions from complete reasoning traces (Fadnavis et al., 2026), or jointly optimizing parallel trace generation and aggregation (Hamid et al., 2026). PaCoRe coordinates multi-round parallel exploration through message passing and synthesis (Hu et al., 2026), whereas ARTS decouples generation from verification (Hong et al., 2026).

For our purposes, these staged decompositions expose intermediate boundaries at which one can perform ablation, distillation, or counterfactual measurement. Counterfactual-likelihood work likewise holds public tokens fixed while intervening on a private block to measure indirect influence (Lorup, 2026). Here the issue is mechanical: a fixed-context probe can be repeatable while evaluating a fresh-prefill state that the end-to-end system never occupied. Replay fidelity is therefore a prerequisite for interpreting a stage-level rate, not an efficiency detail.

For problem $x$, the model generates three branch traces $C=(c_{1},c_{2},c_{3})$, a merge trace $m$, and an answer $a$:

The flattened token sequence has exactly six ordered, non-nested blocks:

Opening and closing delimiters receive the role of their block. Raw roles $0,\ldots,5$ denote prompt, the three branches, merge, and answer; the adapted checkpoint maps them to appended role-embedding rows $8,\ldots,13$. At every layer, the attention mask is the conjunction of the causal mask, key/query padding validity, and the role-visibility relation in Table 2. In particular, answer tokens cannot attend directly to branch tokens.

The structural logits mask enforces the block order above. Inside an open block, ordinary vocabulary items and only that block’s closing delimiter are eligible structural outputs; between blocks, only the next opening delimiter is eligible. A closing delimiter is forced when a branch, merge, or answer reaches its respective 768, 512, or 64 content-token cap. Greedy selection uses temperature 0, top-$p=1$, and top-$k=0$.

We study the state immediately after the unique <|merge|> token has been processed and before the first merge-content token is selected. Let $z$ denote the exact integer-token prefix through that delimiter. In the primary batch-of-two duplicate design there is no padding; position IDs and cache positions enumerate non-padding tokens from zero. Incremental steps advance both by one, and the one-shot path receives the same role, position, visibility, and padding arrays reconstructed from the integer IDs. The live path reaches $z$ through cache updates during branch decoding. The replay path discards that cache and reconstructs $z$ by one-shot prefill before the common decoder generates merge and answer tokens.

We use three construction names throughout. A live cache is retained from ordinary autoregressive generation. An incremental cache is rebuilt from fixed tokens one token at a time under teacher forcing. A prefill cache is built from those tokens in one forward call. A replica is a second storage-isolated instance or independent repeat used only to measure the within-construction floor. The bridge experiment tests when live and incremental caches can be treated as equivalent; the main comparisons never conflate those labels.

Live and prefill paths are mathematically intended to represent the same prefix, but they are not the same computation. The matched design in Figure 1 holds the suffix surface fixed and changes only how the boundary state is constructed.

The primary checkpoint is a 14B multi-branch model derived from Qwen2.5-14B-Instruct (Yang et al., 2024) after a merge-targeted policy update. The matched and fixed-prefix experiments use all 200 rows of a frozen holdout drawn from GPQA Main (Rein et al., 2024), with no exclusions after decoding. This domain had been selected after an earlier breadth screen; that screen is not used as evidence in this paper. The transplantation replication uses a distinct later 14B checkpoint from the same training family. It removes single-checkpoint dependence but is not an architecture-family replication.

The experiment generates the boundary once per problem and creates four storage-isolated continuations:

• •

  the retained live DynamicCache and a bit-exact, storage-isolated replica;

• •

  a one-shot prefill from the same integer-token prefix and a second, independently constructed prefill replica.

All four arms use one shared suffix decoder and identical token and role state, visibility masks, positions, cache positions, limits, and physical batch membership. Each request contains two duplicate copies of one problem and fails closed if they desynchronize. Decoding is greedy through the Hugging Face Transformers library (Wolf et al., 2020) and uses PyTorch SDPA, with 768/512/64 branch, merge, and answer content caps.

The design is run separately in BF16 and FP32. FP32 disables TF32 matrix multiplication and serves as a high-precision falsification; neither surface uses a certified batch-invariant kernel. An eight-row preflight at each precision must reproduce both replica pairs exactly. The full integrity gate is conjunctive over all 200 problems: both replica pairs must agree exactly in boundary state, cache tensors, logits, suffixes, and recorded geometry; all arms must expose the same functional token, role, visibility, position, and padding state; and every compared cache or logit tensor must have disjoint storage.

Conditional on those gates, prefill versus live is the primary comparison. Endpoints are boundary-cache and next-token-logit exactness and magnitude, complete suffix, merge, and answer disagreement, first divergence position, paired correctness, and directional correctness flips. Wilson intervals are used for binary disagreement rates; accuracy differences use 20,000 problem-paired bootstrap replicates. The analysis also reports exact cross-precision boundary-prefix identity. If BF16 and FP32 reach different token prefixes, their endpoint difference is interpreted as a precision-surface comparison rather than a fixed-prefix precision effect.

The decisive precision test freezes one discrete boundary state per problem before crossing numerical precision. The bank contains all 200 BF16 token prefixes from Experiment 1 in source order, without outcome-based filtering; prefix lengths range from 1,025 to 5,080 tokens. Every BF16 and FP32 cell therefore consumes the same integer token, role, visibility, position, and padding sequence for a given problem.

Within each precision, the incremental cache is constructed by teacher forcing the fixed prefix one token at a time without allowing the forced discrete state to change. It receives a bit-exact, storage-isolated replica. The prefill cache is built in one call and independently repeated. The suffix decoder, physical duplicate batch of two, generation limits, scoring, and exact integrity gates are otherwise inherited from Experiment 1. A two-row preflight at each precision must pass before the 200-row arm runs.

The analysis plan was frozen before either full arm. The primary estimand is prefill minus incremental, crossed over precision. Primary endpoints are suffix, answer, and correctness-label disagreement within precision and the problem-paired FP32-minus-BF16 change in each indicator. Binary rates use Wilson intervals; paired differences use 20,000 problem bootstrap replicates, with exact two-sided McNemar tests as directional side reads. No rows are excluded.

For incremental and prefill cache tensors $K_{\mathrm{inc}}$ and $K_{\mathrm{pre}}$, relative drift is

The same quantity is aggregated over all key/value tensors. Shapes, strides, dtypes, devices, absolute differences, and raw norms are retained for every row.

Experiment 2 reconstructs its incremental cell by teacher forcing rather than retaining the original cache from Experiment 1. We therefore freeze a prospective bridge before new decode. Twelve GPQA Main rows are selected by evenly spaced order statistics after sorting the 200 saved BF16 prefixes by length; expected answers and prior outcomes are not read. The selection spans prefix lengths from 1,025 to 5,080 tokens.

For each row, ordinary greedy BF16 decoding first reaches and retains a new live merge-boundary cache. The exact integer-token prefix just reached is then consumed one token at a time to build an incremental cache in the same model process. Each cache receives a bit-exact, storage-isolated replica, and the live and incremental states are continued with the same physical duplicate batch and suffix decoder. Primary endpoints are cache, boundary-logit, and complete-suffix identity. Agreement with the saved prefix bank is descriptive rather than a gate: the experiment tests construction equivalence conditional on one shared newly reached prefix, because the earlier cache tensors were not retained.

The saved ledgers also permit a deterministic 200-row behavioral audit without new decoding. Joined by problem identifier in frozen order, it compares exact prefix IDs, complete suffix-token arrays, and every scalar cache/logit summary shared by the two experiments. This endpoint can establish trajectory and numerical-fingerprint reproduction, but not equality of cache tensors whose values or injective digests were not retained.

We next intervene on the candidate state variable directly. Before any transplant outcome is decoded, 32 rows are frozen from the completed BF16 fixed-prefix matrix: 24 of its 166 suffix-divergent rows and eight of its 34 suffix-exact rows. Within each stratum, rows are evenly spaced order statistics after sorting by prefix length and problem identifier. This outcome-conditioned design tests causal sufficiency on known sensitive states and includes a bounded procedural control; it does not estimate prevalence. The selected prefixes span 1,025–5,080 tokens. Expected answers and correctness scores are neither read nor recomputed.

Each fixed prefix reconstructs an incremental state and a prefill state in BF16. Each construction is instantiated twice with disjoint storage. The interventions are bidirectional. In one direction, all 48 key/value layer pairs in the prefill recipient are replaced with storage-isolated copies from the incremental donor; the other direction reverses donor and recipient. The recipient’s token, role, mask, position, padding, cache position, suffix decoder, and pre-intervention boundary logits remain unchanged. The raw next-token argmax must already agree, so both interventions begin from the same first suffix token. For every tensor, the runner verifies provenance, shape, stride, dtype, sequence length, and absence of storage aliasing.

The registered primary endpoint is exact donor-trajectory recovery in both directions among rows whose contemporaneous construction baselines diverge. The negative-control endpoint is whether either full transplant creates a third trajectory among contemporaneously exact rows. Binary endpoints use Wilson 95% intervals.

The first transplantation panel is deliberately conditioned on known divergence. We therefore freeze a second panel before observing any suffix or transplant outcome on a later 14B checkpoint from the same training family. From the complete 200-prefix BF16 bank, 48 rows are selected as evenly spaced order statistics after sorting by prefix length and problem identifier. Selection reads neither prior suffix identity nor expected answers or correctness; the selected prefixes span 1,025–5,080 tokens.

Each row reconstructs incremental and prefill states under the later checkpoint, with the same functional-state, raw-argmax, storage-isolation, tensor-provenance, and exact-replica gates as Experiment 4. Only six arms are permitted: two replicas of each construction and the two bidirectional whole-cache transplants. No partial-layer patch or adaptive refinement is run. The endpoints are the contemporaneous incremental/prefill suffix-divergence rate over all 48 rows, donor recovery in each direction and jointly among divergent rows, and production of a new trajectory among naturally exact rows. Wilson 95% intervals are computed on each eligible set.

The interpretation matrix is frozen with the design. “Strong replication” requires at least 12 divergent and six exact rows, bidirectional donor recovery on every divergent row, and no new trajectory among exact controls. If recovery is complete but fewer than six exact controls occur naturally, the result is a conditional replication with an underpowered control stratum. Any recovery failure is checkpoint-sensitive recovery; any novel exact-control trajectory is a procedural-control failure; and fewer than 12 divergent rows is divergence-underpowered.

Answers are scored with one deterministic matcher that normalizes answer blocks, multiple-choice forms, and common mathematical formats. Accuracy comparisons join arms by problem identifier. We report paired percentile bootstrap intervals and exact two-sided McNemar tests on discordant correctness labels. McNemar tests describe directional imbalance; a nonsignificant test does not establish equivalence.

The primary model is derived from Qwen/Qwen2.5-14B-Instruct and uses 48 decoder layers with role-conditioned embeddings and the six-role token/mask contract in Section 3.1. Raw multi-branch roles 0–5 index appended embedding rows 8–13; visibility is computed from the raw roles before that offset is applied. The exact inference identity is:

These digests distinguish the primary checkpoint from the public backbone. The outcome-blind transplantation replication uses a distinct later training checkpoint in the same model family. Its inference identity is:

Table 3 gives the numerical contract for the matched HF experiments. Each DynamicCache contains one key and one value tensor per layer, each shaped $[2,8,|z|,128]$; the evidence records shape, stride, dtype, and device for every compared tensor. The live replica is produced by a deep copy of the complete cache under inference mode. The runner rejects a replica if its tensor inventory, values, metadata, or inference-tensor status changes, or if any key/value storage aliases the source. Fresh-prefill replicas are independent model calls rather than copies.

The reproducibility release is scoped to the claims in this paper. It contains a redacted environment/checkpoint manifest, frozen problem-ID order and dataset digests, exact boundary-prefix and suffix token IDs, per-item scores, cache-difference summaries, and the primary generation and analysis harnesses. It excludes prompt text, private training corpora, training lineage, unrelated auxiliary experiments, and internal filesystem names. Thus the released rows permit independent recomputation of every reported primary statistic and inspection of tensor drift; an exact end-to-end rerun still requires access to the digest-identified adapted checkpoint.

The retained-live four-arm matrix passes every integrity gate at both precisions. Both replica pairs are exact on all 200 items, including boundary caches, logits, decoded suffixes, and correctness labels. Physical duplicates remain synchronized, tensor stores are disjoint where required, and the arms share the same suffix decoder and recorded geometry. The live/prefill contrast therefore has no observed replica floor.

In BF16, the two state constructions are non-bit-exact at the boundary on every item, although their immediate greedy argmax always agrees. The continuations subsequently diverge on 166/200 suffixes (83.0%; Wilson 95% CI [77.18,87.57]), 49/200 answers (24.5%; [19.06,30.90]), and 20/200 correctness labels (10.0%). The first suffix mismatch occurs a median of 88 tokens after the boundary (range 4–453), never on the first suffix token. Live and prefill score 93/200 and 95/200. The $+1.0$-point difference has a paired 95% CI of $[-3.5,+5.5]$, with nine live-only and eleven prefill-only successes (exact McNemar $p=.824$). The result is trajectory and item churn, not an accuracy gain.

The independently generated FP32 matrix retains small non-bit-exact cache and logit differences but produces no decoded disagreement. Because only 1/200 boundary prefixes is token-identical across those original precision runs, this first matrix establishes live-state failure in BF16 but supplies only a high-precision surface falsification.

The prospective $2\times 2$ removes that confound. The integer prefix is identical across precision on 200/200 problems. At each precision, both replica pairs are exact on all 200 boundary caches, logits, functional-state fields, suffixes, answers, and labels; storage-disjointness and recorded geometry also pass on every row.

On these fixed states, BF16 again changes 166 suffixes, 49 answers, and 20 correctness labels under one-shot prefill, whereas FP32 changes 0/200 of each. Each zero-event rate has Wilson 95% interval $[0,1.88]\%$. The paired FP32-minus-BF16 reductions are 83.0 percentage points for suffix disagreement (95% bootstrap CI [77.5,88.0]), 24.5 points for answer disagreement ([18.5,30.51]), and 10.0 points for correctness churn ([6.0,14.5]). Exact paired tests give $p=2.14\times 10^{-50}$, $3.55\times 10^{-15}$, and $1.91\times 10^{-6}$, respectively. BF16 incremental/prefill accuracy is 93/200 versus 95/200; FP32 is 92/200 versus 92/200. The precision difference in the accuracy effect is $-1.0$ point (95% CI $[-5.5,+3.5]$), so the result concerns fidelity rather than gain.

The cache remains non-bit-exact at both precisions, but its median relative $L_{2}$ difference falls from 0.01224 in BF16 to $4.27\times 10^{-6}$ in FP32. Conditional on exact replicas and fixed integer states, numerical precision therefore moderates whether the construction difference becomes behaviorally visible. The experiment does not isolate a unique low-level cause: reduction order, kernel selection, cache implementation, and their interactions remain candidates.

All 12 bridge rows pass the replica, state, storage, and physical-duplicate integrity gates. Conditional on the newly reached live token prefix, the incremental construction is bit-exact to the retained live cache on all 48 layers for 12/12 rows. Boundary logits and complete greedy suffixes are likewise exact on 12/12. The newly reached prefixes span 1,463–4,642 tokens; the outcome-blind selection had been spread over saved-prefix lengths of 1,025–5,080 tokens.

None of the 12 newly decoded prefixes is token-identical to its earlier saved prefix. The result therefore does not claim recovery of discarded cache tensors. It establishes the relevant prospective bridge: when ordinary decoding and incremental construction consume the same newly reached prefix under this HF contract, they produce the same cache, logits, and continuation. Experiment 2’s incremental cell is consequently validated as an implementation of live-style cache updates on these tested states.

The saved-ledger join closes the corresponding behavioral endpoint. Prefix IDs and problem order agree on 200/200 rows. The later incremental arm reproduces the original live suffix token for token on 200/200 rows; both replica and prefill arms also reproduce on 200/200. All three suffix-comparison objects agree on every row. For each cache comparison, all 1,158 scalar leaves common to the two schemas agree on 200/200 rows; the analogous 14 common logit leaves agree on 200/200 for each logit comparison. These per-tensor drift fingerprints are highly constraining, but they are not injective hashes of discarded tensors. The bridge therefore establishes 12-row prospective tensor equality and 200-row trajectory and fingerprint equality, not retrospective tensor equality.

The transplantation run completes all 32 frozen rows. Every row passes the prefix, functional-state, replica, tensor-provenance, storage, and raw-argmax gates, and all 24 prior-divergent and eight prior-exact strata are reproduced by the contemporaneous incremental/prefill baselines.

On every divergent row, each complete transplant produces its cache donor’s suffix (Table 5). Bidirectional recovery therefore holds on 24/24 selected rows (95% Wilson CI [86.2,100]%). Among the eight nondivergent controls, neither full-cache direction produces a trajectory outside the shared baseline (0/8; [0,32.4]%). The latter interval reflects the deliberately small procedural-control stratum and does not establish a low population-wide artifact rate.

The full transplant changes only the cached K/V tensors: recipient boundary logits and all non-cache functional state are retained, and the immediate argmax is shared. The downstream switch must therefore be mediated by the donated cache once suffix decoding resumes. On these selected sensitive states, the complete boundary K/V cache is causally sufficient to select between the two observed greedy trajectories. This is stronger than localization by residual-stream elimination and instantiates the class of KV-tensor intervention identified as future work by Chodavarapu and Xu (2026), here at a whole-stage boundary.

All 48 frozen rows complete and pass the prefix, functional-state, raw-argmax, storage-isolation, tensor-provenance, and exact-replica gates. The later checkpoint’s contemporaneous incremental and prefill suffixes diverge on 43/48 rows (89.6%; Wilson 95% CI [77.8,95.5]%). This is an outcome-blind, deterministic length-spread panel rather than a sample chosen for known sensitivity.

On all 43 divergent rows, each transplant exactly follows its cache donor; joint bidirectional recovery is therefore 43/43 (95% CI [91.8,100]%). The other five rows are naturally exact across constructions, and neither transplant direction creates a trajectory outside their shared baseline (0/5; [0,43.4]%). Table 6 reports the registered endpoints.

The frozen interpretation is conditional replication with an underpowered control stratum: donor recovery is complete, but only five naturally exact controls occur, one fewer than the registered minimum of six for the strongest label. The wide 0/5 interval therefore remains explicit. That limitation concerns the precision of the procedural-control bound, not the separately conditioned 43/43 recovery endpoint. Together with the primary-checkpoint intervention, the result removes known-divergence selection and single-checkpoint dependence from the donor-recovery finding. It does not turn the 48 fixed, primary-checkpoint-reachable prefixes into a model- or task-general prevalence sample.