JEV typed-choice policy: complete 92-case evaluation against a seeded-random baseline

JEV on RestoreBench: typed legal-action selection

Experiment date: 2026-09-26 (UTC). Upstream: https://github.com/Mansutti081/RestoreBench
Commit: accdda5be82047394b13486235501510bf593fd6. Requested model: typesafe/jev-1.13.
Observed model IDs: typesafe/jev-1.13-20260917.

Results

Corpus	Selector	Success	Rate	Median moves on success	Decision cost (USD)
ieee118	jev	12/46	26.1%	4.5	$0.269519
ieee118	random	13/46	28.3%	5	$0.000000
pegase89	jev	4/46	8.7%	3.0	$0.251813
pegase89	random	5/46	10.9%	6	$0.000000
all	jev	16/92	17.4%	4.0	$0.521332
all	random	18/92	19.6%	5.5	$0.000000

JEV minus seeded random: -2.2 percentage points across these 92 scenarios.
This is a separate action-selection experiment, not a directly comparable entry in the published chat-agent leaderboard.

Protocol

All 46 frozen evaluation scenarios in each of IEEE118 and PEGASE89 were run once per selector;
there are no missing result cells. The upstream scenario loader validated the frozen roster and
artifact hashes. Both selectors receive the same policy interface: current public scenario card,
current solver diagnostics, remaining move budget, and previous selected moves with solver feedback.
Private labels and witness solutions were not accessed or supplied to either policy.

At each step, enumerate public legal Q–V moves with q_saturation_context, then use upstream
select_candidates(..., budget=12) to select an evenly spread deterministic shortlist, with no
candidate-outcome probes. JEV's Decisions API picks exactly one typed choice. Apply that move,
run the locked pandapower solver, and stop at success or 10 moves. The random selector samples
uniformly from its current shortlist; its per-scenario Python PRNG seed is the integer SHA256
of <corpus>/<scenario>/seed-0. Action history and shortlists are included in each raw result.

Success means AC convergence and non-voltage feasibility as defined by the upstream runtime.
It does not require acceptable bus voltages. Voltage-clean successes: JEV 0/16;
random 0/18. Full terminal voltage quality and feasibility
are retained, including the broader feasible flag that can remain false due to voltage violations.
These simulation results are not evidence of safe real-grid restoration.

Native evaluator verification

Every saved sequence was independently replayed through unmodified upstream resolve, using
configuration 1 solely to inject an already-recorded sequence. This is a replay method, not a claim
that JEV was evaluated as the benchmark's configuration-1 chat agent. Status disagreements: 0/184.
Native rejected actions: 0. The replay preserves the native action guards and success checks.
The external policy uses directional Q saturation while the native sandbox maps saturated generator
IDs to upper-limit guards; the replay checks the practical consequences on these actual sequences.

Version stamps: solver locked-nr-q-limited-v1, action qv-atomic-v1,
ranking retreat=snapshot-anchored-retreat-v1, result resolution-response-v3.
Python 3.11.16; pandapower 3.4.0; frozen uv.lock. Upstream's legacy-retreat diagnostic fallback
remained enabled for both policies. No solver, feasibility, or native evaluator code was changed.

Paired outcomes and resource use

Paired scenario outcomes: {"all": {"both": 7, "jev_only": 9, "neither": 65, "random_only": 11}, "ieee118": {"both": 5, "jev_only": 7, "neither": 26, "random_only": 8}, "pegase89": {"both": 2, "jev_only": 2, "neither": 39, "random_only": 3}}.
JEV made 826 recorded decisions: {"GEN_V_SETPOINT": 189, "SHUNT_STEP": 216, "TAP_ADJUSTMENT": 421}.
Provider-reported usage: 12,412,666 input tokens, 96,971 output tokens,
$0.521332 for recorded decisions. This is API usage cost, excluding hosting/compute.

A private credential gateway kept the API key out of the runner and artifacts. It cached successful
responses by request identity. Network interruptions were resumed, reusing completed decisions;
transport failures were not scored as policy failures. Client concurrency was initially six, then four plus a separate two-worker PEGASE batch,
and finally ten after a checkpoint resume. The gateway capped active provider requests at six.
Runtime measurements include solver, network, concurrency, and cache effects; they are not a
model inference-speed comparison. No model seed was supplied; the random baseline has one seed
per case. One run per frozen scenario cannot establish multi-run variance or general superiority.
The shortlist may exclude a solving action and is an important part of the evaluated policy.

Per-scenario data

Complete paired CSV (92 scenarios)

corpus,scenario,jev_status,jev_moves,jev_cost_usd,jev_native_status,jev_invalid_actions,jev_voltage_clean,jev_runtime_seconds,random_status,random_moves,random_cost_usd,random_native_status,random_invalid_actions,random_voltage_clean,random_runtime_seconds
ieee118,S0008,SUCCESS,5,0.003407376,SUCCESS,0,False,51.878,SUCCESS,6,0.0,SUCCESS,0,False,14.082
ieee118,S0012,SUCCESS,4,0.0027208859999999996,SUCCESS,0,False,44.027,SUCCESS,8,0.0,SUCCESS,0,False,16.939
ieee118,S0014,BUDGET_EXHAUSTED,10,0.006902363999999999,BUDGET_EXHAUSTED,0,False,112.237,SUCCESS,2,0.0,SUCCESS,0,False,2.783
ieee118,S0016,BUDGET_EXHAUSTED,10,0.006904926,BUDGET_EXHAUSTED,0,False,114.035,SUCCESS,7,0.0,SUCCESS,0,False,20.73
ieee118,S0019,BUDGET_EXHAUSTED,10,0.006905808000000001,BUDGET_EXHAUSTED,0,False,112.18,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,38.372
ieee118,S0029,SUCCESS,1,0.000675822,SUCCESS,0,False,9.415,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,23.493
ieee118,S0039,SUCCESS,3,0.002034102,SUCCESS,0,False,32.174,SUCCESS,3,0.0,SUCCESS,0,False,6.966
ieee118,S0041,SUCCESS,1,0.000675612,SUCCESS,0,False,8.363,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,18.926
ieee118,S0046,BUDGET_EXHAUSTED,10,0.00691068,BUDGET_EXHAUSTED,0,False,112.535,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,28.944
ieee118,S0048,BUDGET_EXHAUSTED,10,0.00690816,BUDGET_EXHAUSTED,0,False,107.462,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,29.772
ieee118,S0049,SUCCESS,7,0.004801860000000001,SUCCESS,0,False,71.986,SUCCESS,5,0.0,SUCCESS,0,False,8.682
ieee118,S0052,SUCCESS,8,0.005497884,SUCCESS,0,False,82.193,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,31.11
ieee118,S0053,SUCCESS,1,0.00067515,SUCCESS,0,False,9.076,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,24.673
ieee118,S0056,BUDGET_EXHAUSTED,10,0.006895224000000001,BUDGET_EXHAUSTED,0,False,103.321,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,27.525
ieee118,S0057,BUDGET_EXHAUSTED,10,0.0069113939999999995,BUDGET_EXHAUSTED,0,False,94.942,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,24.904
ieee118,S0058,BUDGET_EXHAUSTED,10,0.006901734,BUDGET_EXHAUSTED,0,False,110.868,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,41.216
ieee118,S0059,BUDGET_EXHAUSTED,10,0.006904422,BUDGET_EXHAUSTED,0,False,109.971,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,30.219
ieee118,S0069,BUDGET_EXHAUSTED,10,0.006905682000000001,BUDGET_EXHAUSTED,0,False,110.539,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,24.198
ieee118,S0070,BUDGET_EXHAUSTED,10,0.006903665999999999,BUDGET_EXHAUSTED,0,False,109.74,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,26.506
ieee118,S0072,BUDGET_EXHAUSTED,10,0.00688905,BUDGET_EXHAUSTED,0,False,90.885,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,32.203
ieee118,S0073,BUDGET_EXHAUSTED,10,0.006905262,BUDGET_EXHAUSTED,0,False,102.383,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,28.985
ieee118,S0081,SUCCESS,2,0.001352946,SUCCESS,0,False,13.242,SUCCESS,3,0.0,SUCCESS,0,False,5.987
ieee118,S0086,BUDGET_EXHAUSTED,10,0.006894552,BUDGET_EXHAUSTED,0,False,102.486,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,22.724
ieee118,S0087,BUDGET_EXHAUSTED,10,0.006903330000000001,BUDGET_EXHAUSTED,0,False,108.154,SUCCESS,6,0.0,SUCCESS,0,False,16.649
ieee118,S0096,BUDGET_EXHAUSTED,10,0.006907866,BUDGET_EXHAUSTED,0,False,114.383,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,36.637
ieee118,S0097,BUDGET_EXHAUSTED,10,0.006907487999999999,BUDGET_EXHAUSTED,0,False,112.381,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,27.171
ieee118,S0102,BUDGET_EXHAUSTED,10,0.006908244000000001,BUDGET_EXHAUSTED,0,False,112.497,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,30.169
ieee118,S0105,BUDGET_EXHAUSTED,10,0.006906522,BUDGET_EXHAUSTED,0,False,115.912,SUCCESS,2,0.0,SUCCESS,0,False,2.78
ieee118,S0107,BUDGET_EXHAUSTED,10,0.0068950980000000005,BUDGET_EXHAUSTED,0,False,112.917,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,23.515
ieee118,S0111,SUCCESS,8,0.0054904080000000004,SUCCESS,0,False,86.905,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,38.278
ieee118,S0117,BUDGET_EXHAUSTED,10,0.006894846,BUDGET_EXHAUSTED,0,False,118.875,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,25.076
ieee118,S0136,BUDGET_EXHAUSTED,10,0.006906438000000001,BUDGET_EXHAUSTED,0,False,116.235,SUCCESS,10,0.0,SUCCESS,0,False,24.207
ieee118,S0137,BUDGET_EXHAUSTED,10,0.006900347999999999,BUDGET_EXHAUSTED,0,False,116.952,SUCCESS,2,0.0,SUCCESS,0,False,2.757
ieee118,S0138,BUDGET_EXHAUSTED,10,0.006903204,BUDGET_EXHAUSTED,0,False,116.829,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,33.069
ieee118,S0141,BUDGET_EXHAUSTED,10,0.006877878,BUDGET_EXHAUSTED,0,False,105.12,SUCCESS,1,0.0,SUCCESS,0,False,0.109
ieee118,S0166,BUDGET_EXHAUSTED,10,0.00690291,BUDGET_EXHAUSTED,0,False,109.89,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,37.405
ieee118,S0167,BUDGET_EXHAUSTED,10,0.0069069,BUDGET_EXHAUSTED,0,False,110.115,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,25.66
ieee118,S0168,BUDGET_EXHAUSTED,10,0.006907361999999999,BUDGET_EXHAUSTED,0,False,110.476,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,29.288
ieee118,S0170,SUCCESS,5,0.0034080480000000003,SUCCESS,0,False,52.87,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,34.846
ieee118,S0171,BUDGET_EXHAUSTED,10,0.00690648,BUDGET_EXHAUSTED,0,False,115.233,SUCCESS,7,0.0,SUCCESS,0,False,15.769
ieee118,S0175,BUDGET_EXHAUSTED,10,0.006905514000000001,BUDGET_EXHAUSTED,0,False,131.035,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,21.497
ieee118,S0177,BUDGET_EXHAUSTED,10,0.006901817999999999,BUDGET_EXHAUSTED,0,False,125.289,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,39.776
ieee118,S0181,SUCCESS,6,0.00408702,SUCCESS,0,False,64.994,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,20.142
ieee118,S0182,BUDGET_EXHAUSTED,10,0.006895392,BUDGET_EXHAUSTED,0,False,151.889,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,31.73
ieee118,S0198,BUDGET_EXHAUSTED,10,0.006908831999999999,BUDGET_EXHAUSTED,0,False,155.344,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,24.854
ieee118,S0200,BUDGET_EXHAUSTED,10,0.006902238,BUDGET_EXHAUSTED,0,False,159.272,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,34.738
pegase89,S0001,BUDGET_EXHAUSTED,10,0.005793816,BUDGET_EXHAUSTED,0,False,158.961,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,20.595
pegase89,S0002,BUDGET_EXHAUSTED,10,0.0057894059999999995,BUDGET_EXHAUSTED,0,False,152.099,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,24.528
pegase89,S0003,BUDGET_EXHAUSTED,10,0.005794026,BUDGET_EXHAUSTED,0,False,153.509,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,21.978
pegase89,S0004,BUDGET_EXHAUSTED,10,0.0057897,BUDGET_EXHAUSTED,0,False,106.781,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,24.671
pegase89,S0005,BUDGET_EXHAUSTED,10,0.005793312,BUDGET_EXHAUSTED,0,False,113.348,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,22.002
pegase89,S0006,BUDGET_EXHAUSTED,10,0.005788944000000001,BUDGET_EXHAUSTED,0,False,193.278,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,21.763
pegase89,S0007,BUDGET_EXHAUSTED,10,0.005792136,BUDGET_EXHAUSTED,0,False,180.447,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,20.016
pegase89,S0008,BUDGET_EXHAUSTED,10,0.00579348,BUDGET_EXHAUSTED,0,False,138.009,SUCCESS,6,0.0,SUCCESS,0,False,12.594
pegase89,S0009,SUCCESS,2,0.00113043,SUCCESS,0,False,15.914,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,28.033
pegase89,S0010,BUDGET_EXHAUSTED,10,0.005788524000000001,BUDGET_EXHAUSTED,0,False,165.868,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,28.025
pegase89,S0011,BUDGET_EXHAUSTED,10,0.005794488,BUDGET_EXHAUSTED,0,False,140.499,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,21.903
pegase89,S0012,BUDGET_EXHAUSTED,10,0.005793774,BUDGET_EXHAUSTED,0,False,155.736,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,22.672
pegase89,S0013,BUDGET_EXHAUSTED,10,0.005790750000000001,BUDGET_EXHAUSTED,0,False,141.053,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,20.796
pegase89,S0014,BUDGET_EXHAUSTED,10,0.005795622,BUDGET_EXHAUSTED,0,False,140.927,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,21.998
pegase89,S0015,BUDGET_EXHAUSTED,10,0.00579033,BUDGET_EXHAUSTED,0,False,138.386,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,25.281
pegase89,S0016,BUDGET_EXHAUSTED,10,0.005789364,BUDGET_EXHAUSTED,0,False,183.638,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,22.288
pegase89,S0017,SUCCESS,1,0.000563934,SUCCESS,0,False,6.345,SUCCESS,2,0.0,SUCCESS,0,False,2.767
pegase89,S0018,BUDGET_EXHAUSTED,10,0.005791758,BUDGET_EXHAUSTED,0,False,128.18,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,20.763
pegase89,S0019,SUCCESS,8,0.004605006000000001,SUCCESS,0,False,97.685,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,24.985
pegase89,S0020,BUDGET_EXHAUSTED,10,0.005789616000000001,BUDGET_EXHAUSTED,0,False,160.283,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,26.197
pegase89,S0021,BUDGET_EXHAUSTED,10,0.00579453,BUDGET_EXHAUSTED,0,False,117.102,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,24.331
pegase89,S0022,BUDGET_EXHAUSTED,10,0.005788188,BUDGET_EXHAUSTED,0,False,125.383,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,20.591
pegase89,S0023,BUDGET_EXHAUSTED,10,0.005791673999999999,BUDGET_EXHAUSTED,0,False,119.29,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,22.913
pegase89,S0024,BUDGET_EXHAUSTED,10,0.005794152,BUDGET_EXHAUSTED,0,False,126.865,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,22.007
pegase89,S0025,BUDGET_EXHAUSTED,10,0.0057858779999999995,BUDGET_EXHAUSTED,0,False,115.775,SUCCESS,6,0.0,SUCCESS,0,False,11.581
pegase89,S0026,BUDGET_EXHAUSTED,10,0.005794404000000001,BUDGET_EXHAUSTED,0,False,139.749,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,22.446
pegase89,S0027,BUDGET_EXHAUSTED,10,0.005791925999999999,BUDGET_EXHAUSTED,0,False,151.226,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,25.241
pegase89,S0028,BUDGET_EXHAUSTED,10,0.00579054,BUDGET_EXHAUSTED,0,False,141.187,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,23.903
pegase89,S0029,BUDGET_EXHAUSTED,10,0.0057928499999999996,BUDGET_EXHAUSTED,0,False,158.215,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,24.369
pegase89,S0030,BUDGET_EXHAUSTED,10,0.005792514000000001,BUDGET_EXHAUSTED,0,False,116.617,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,22.662
pegase89,S0031,BUDGET_EXHAUSTED,10,0.0057912120000000004,BUDGET_EXHAUSTED,0,False,115.705,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,23.475
pegase89,S0032,BUDGET_EXHAUSTED,10,0.005794026,BUDGET_EXHAUSTED,0,False,116.469,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,21.207
pegase89,S0033,BUDGET_EXHAUSTED,10,0.005793438000000001,BUDGET_EXHAUSTED,0,False,117.571,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,22.333
pegase89,S0034,BUDGET_EXHAUSTED,10,0.005789028,BUDGET_EXHAUSTED,0,False,117.307,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,25.917
pegase89,S0035,SUCCESS,4,0.0022746779999999996,SUCCESS,0,False,42.823,SUCCESS,9,0.0,SUCCESS,0,False,17.77
pegase89,S0036,BUDGET_EXHAUSTED,10,0.005788272,BUDGET_EXHAUSTED,0,False,111.274,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,22.52
pegase89,S0037,BUDGET_EXHAUSTED,10,0.0057895740000000005,BUDGET_EXHAUSTED,0,False,115.274,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,26.761
pegase89,S0038,BUDGET_EXHAUSTED,10,0.005790834,BUDGET_EXHAUSTED,0,False,112.259,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,24.27
pegase89,S0039,BUDGET_EXHAUSTED,10,0.005793983999999999,BUDGET_EXHAUSTED,0,False,137.109,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,20.425
pegase89,S0040,BUDGET_EXHAUSTED,10,0.005795747999999999,BUDGET_EXHAUSTED,0,False,151.723,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,20.484
pegase89,S0041,BUDGET_EXHAUSTED,10,0.00579285,BUDGET_EXHAUSTED,0,False,167.137,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,21.096
pegase89,S0042,BUDGET_EXHAUSTED,10,0.005793522000000001,BUDGET_EXHAUSTED,0,False,178.305,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,20.079
pegase89,S0043,BUDGET_EXHAUSTED,10,0.005789658,BUDGET_EXHAUSTED,0,False,191.873,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,21.032
pegase89,S0044,BUDGET_EXHAUSTED,10,0.0057894059999999995,BUDGET_EXHAUSTED,0,False,202.454,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,21.13
pegase89,S0045,BUDGET_EXHAUSTED,10,0.00578508,BUDGET_EXHAUSTED,0,False,172.533,BUDGET_EXHAUSTED,10,0.0,BUDGET_EXHAUSTED,0,False,25.154
pegase89,S0046,BUDGET_EXHAUSTED,10,0.005786843999999999,BUDGET_EXHAUSTED,0,False,128.419,SUCCESS,1,0.0,SUCCESS,0,False,0.096

Minimal JEV runner

Save the following as restorebench/scoring/jev_policy.py in the pinned checkout, then run uv run python -m restorebench.scoring.jev_policy with OPENROUTER_API_KEY set securely. See its CLI flags for both corpora. The random selector is also available via run_episode(..., selector="random").

Self-contained policy adapter

# ABOUTME: Evaluates a typed Jev decision policy on frozen RestoreBench scenarios.
# ABOUTME: Keeps model selection separate from solver-owned action and success checks.
from __future__ import annotations

import argparse
import hashlib
import json
import os
import random
import time
from pathlib import Path
from typing import Any
from urllib.error import HTTPError, URLError
from urllib.request import Request, urlopen

import pandapower as pp

from restorebench.environment.card_render import render_scenario_card
from restorebench.environment.scenarios import load_full_net, load_scenario
from restorebench.physics.actions import apply_qv_action, enumerate_legal_qv_actions
from restorebench.physics.feasibility import satisfies_non_voltage_constraints
from restorebench.physics.q_saturation import q_saturation_context
from restorebench.scoring.score_maneuvers import MANEUVER_BUDGET
from restorebench.tools.candidate_ranking import select_candidates
from restorebench.tools.power_flow import run_ac_pf

ROOT = Path(__file__).resolve().parents[2]
MODEL = "typesafe/jev-1.13"
ENDPOINT = "https://openrouter.ai/api/alpha/decisions"
POLICY = "jev-legal-shortlist-v1"


def _criterion(action: Any) -> str:
    # Match JSON.stringify for the finite, bounded numeric action schema.
    fields = action.model_dump(mode="json")
    fields = {k: int(v) if isinstance(v, float) and v.is_integer() else v for k, v in fields.items()}
    return "Select this exact atomic maneuver: " + json.dumps(fields, separators=(",", ":"))


def request_choice(state: dict[str, Any], actions: list[Any], api_key: str) -> tuple[int, dict[str, Any]]:
    """Select one action from a deterministic shortlist; fail closed on malformed responses."""
    criteria = {f"a{i}": _criterion(action) for i, action in enumerate(actions)}
    body = {
        "model": MODEL,
        "state": {**state, "legal_maneuvers": criteria},
        "questions": {
            "maneuver": {
                "type": "choice",
                "instructions": (
                    "Which listed legal atomic reactive-control maneuver is most likely to move the "
                    "current non-convergent network toward AC convergence and feasible operation? "
                    "Choose exactly one of the listed action IDs. Use the scenario and previous solver feedback."
                ),
                "criteria": criteria,
            }
        },
    }
    endpoint = ENDPOINT
    headers = {"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"}
    gateway = os.environ.get("RESTOREBENCH_JEV_GATEWAY")
    if gateway:
        if not gateway.startswith("https://") or "@" in gateway:
            raise ValueError("gateway must be an HTTPS origin without embedded credentials")
        endpoint = gateway.rstrip("/") + "/api/decision"
        body = {"state": state, "actions": [a.model_dump(mode="json") for a in actions]}
        body["request_id"] = (
            "rb-" + hashlib.sha256(json.dumps(body, sort_keys=True, allow_nan=False).encode()).hexdigest()
        )
        headers["OAI-Sites-Authorization"] = f"Bearer {api_key}"
    request_bytes = json.dumps(body, allow_nan=False).encode("utf-8")
    if len(request_bytes) > 120000:
        raise ValueError("Jev request exceeds the experiment's 120 kB limit")
    req = Request(
        endpoint,
        data=request_bytes,
        headers=headers,
        method="POST",
    )
    started = time.perf_counter()
    for attempt in range(3 if gateway else 1):
        try:
            with urlopen(req, timeout=110) as response:
                result = json.load(response)
            break
        except HTTPError as exc:
            # Gateway caches successful decisions by request ID, permitting safe replay.
            if gateway and exc.code in {429, 502, 503, 504} and attempt < 2:
                time.sleep(2 ** (attempt + 1))
                continue
            raise RuntimeError(f"Jev HTTP {exc.code}; episode incomplete") from exc
        except (URLError, TimeoutError) as exc:
            if gateway and attempt < 2:
                time.sleep(2 ** (attempt + 1))
                continue
            raise RuntimeError("Jev unavailable; episode incomplete") from exc
    answer = result.get("answers", {}).get("maneuver", {})
    chosen = answer.get("choice")
    if answer.get("type") != "choice" or chosen not in criteria:
        raise ValueError("Jev returned an invalid action choice; episode not scored")
    return int(chosen[1:]), {
        "choice": chosen,
        "confidence": answer.get("confidence"),
        "probabilities": answer.get("probabilities"),
        "resolved_model": result.get("model"),
        "usage": result.get("usage"),
        "latency_seconds": round(time.perf_counter() - started, 6),
        "request_sha256": hashlib.sha256(request_bytes).hexdigest(),
    }


def run_episode(
    scenario_id: str, dataset: Path, api_key: str, *, shortlist_size: int = 12, selector: str = "jev"
) -> dict[str, Any]:
    if selector not in {"jev", "random"}:
        raise ValueError("unknown selector")
    rng = random.Random(int(hashlib.sha256(f"{dataset.name}/{scenario_id}/seed-0".encode()).hexdigest(), 16))
    scenario = load_scenario(scenario_id, data_dir=dataset)
    net = load_full_net(scenario)
    baseline = run_ac_pf(net)
    if baseline.converged:
        raise ValueError(f"{scenario_id} converges at baseline")
    previous: list[dict[str, Any]] = []
    started = time.perf_counter()
    spent = 0.0
    for step in range(MANEUVER_BUDGET):
        context = q_saturation_context(net)
        actions = select_candidates(enumerate_legal_qv_actions(net, context), budget=shortlist_size)
        if not actions:
            break
        state = {
            "scenario_card": render_scenario_card(net),
            "solver_diagnostics": baseline.diagnostics.model_dump(mode="json") if baseline.diagnostics else None,
            "prior_steps": [
                {key: item[key] for key in ("action", "converged", "feasible", "overstress_after")} for item in previous
            ],
            "maneuvers_remaining": MANEUVER_BUDGET - step,
        }
        if selector == "jev":
            index, decision = request_choice(state, actions, api_key)
        else:
            index = rng.randrange(len(actions))
            decision = {"choice": f"a{index}", "resolved_model": "uniform-random", "usage": {"cost": 0}}
        spent += float((decision.get("usage") or {}).get("cost") or 0)
        action = actions[index]
        net = apply_qv_action(net, action, context)
        baseline = run_ac_pf(net)
        feasible = baseline.converged and satisfies_non_voltage_constraints(baseline.feasibility)
        previous.append(
            {
                "action": action.model_dump(mode="json"),
                "shortlist": [a.model_dump(mode="json") for a in actions],
                "choice": decision,
                "converged": baseline.converged,
                "feasible": feasible,
                "overstress_after": baseline.diagnostics.overstress if baseline.diagnostics else None,
            }
        )
        if feasible:
            break
    return {
        "scenario_id": scenario_id,
        "status": "SUCCESS" if previous and previous[-1]["feasible"] else "BUDGET_EXHAUSTED",
        "n_maneuvers": len(previous),
        "steps": previous,
        "model": MODEL if selector == "jev" else "uniform-random",
        "policy": POLICY if selector == "jev" else "random-legal-shortlist-v1",
        "shortlist_size": shortlist_size,
        "cost_usd": spent,
        "runtime_seconds": round(time.perf_counter() - started, 3),
        "terminal_quality": baseline.quality.model_dump(mode="json") if baseline.quality else None,
        "terminal_feasibility": baseline.feasibility.model_dump(mode="json") if baseline.feasibility else None,
        "scenario_sha256": hashlib.sha256((dataset / "full" / f"{scenario_id}.json").read_bytes()).hexdigest(),
        "manifest_sha256": hashlib.sha256((dataset / "manifest.json").read_bytes()).hexdigest(),
        "pandapower_version": pp.__version__,
    }


def main() -> None:
    parser = argparse.ArgumentParser(description="Run the Jev legal-action selection policy on RestoreBench")
    parser.add_argument("--dataset", type=Path, default=ROOT / "dataset/ieee118")
    parser.add_argument("--out", type=Path, default=ROOT / "results/jev-policy")
    parser.add_argument("--scenario", action="append", help="scenario ID; default: all frozen evaluation IDs")
    parser.add_argument("--shortlist-size", type=int, default=12)
    parser.add_argument("--dry-run", action="store_true")
    parser.add_argument("--summary", action="store_true", help="summarize saved result cells; makes no API calls")
    args = parser.parse_args()
    if args.shortlist_size < 1:
        parser.error("--shortlist-size must be positive")
    manifest = json.loads((args.dataset / "evaluation_manifest.json").read_text(encoding="utf-8"))
    roster = [entry["scenario_id"] for entry in manifest["scenarios"]]
    scenarios = args.scenario or roster
    if any(scenario not in roster for scenario in scenarios):
        parser.error("every scenario must appear in the frozen evaluation manifest")
    if args.summary:
        stored = [
            json.loads((args.out / f"{sid}.json").read_text(encoding="utf-8"))
            for sid in scenarios
            if (args.out / f"{sid}.json").exists()
        ]
        print(
            json.dumps(
                {
                    "dataset_version": manifest["dataset_version"],
                    "policy": POLICY,
                    "total": len(scenarios),
                    "completed": len(stored),
                    "missing": [sid for sid in scenarios if not (args.out / f"{sid}.json").exists()],
                    "success": sum(row["status"] == "SUCCESS" for row in stored),
                    "cost_usd": sum(row["cost_usd"] for row in stored),
                },
                indent=2,
            )
        )
        return
    if args.dry_run:
        print(json.dumps({"dataset": manifest["dataset_version"], "scenarios": scenarios, "policy": POLICY}))
        return
    key_name = "RESTOREBENCH_GATEWAY_TOKEN" if os.environ.get("RESTOREBENCH_JEV_GATEWAY") else "OPENROUTER_API_KEY"
    key = os.environ.get(key_name)
    if not key:
        parser.error(f"set {key_name} in the process environment")
    args.out.mkdir(parents=True, exist_ok=True)
    for scenario in scenarios:
        target = args.out / f"{scenario}.json"
        if target.exists():
            saved = json.loads(target.read_text(encoding="utf-8"))
            if (
                saved.get("dataset_version") != manifest["dataset_version"]
                or saved.get("policy") != POLICY
                or saved.get("shortlist_size") != args.shortlist_size
            ):
                raise ValueError(f"{target} belongs to a different experiment configuration")
            print(f"skip {scenario}: checkpoint exists")
            continue
        result = run_episode(scenario, args.dataset, key, shortlist_size=args.shortlist_size)
        result["dataset_version"] = manifest["dataset_version"]
        temporary = target.with_suffix(".tmp")
        temporary.write_text(json.dumps(result, indent=2, allow_nan=False) + "\n", encoding="utf-8")
        temporary.replace(target)
        print(f"{scenario}: {result['status']} ({result['n_maneuvers']} maneuvers, ${result['cost_usd']:.5f})")


if __name__ == "__main__":
    main()

Exact recorded action sequences

These sequences permit solver-only verification without another model call. Actions are in execution order, separated by semicolons: G<gen_id>=<new_vm_pu> is GEN_V_SETPOINT, H<shunt_id>=<new_step> is SHUNT_STEP, and T<trafo_id>=<new_tap_pos> is TAP_ADJUSTMENT. Replay each list through the pinned native resolve with configuration 1, the corresponding public scenario, and a ProposeSequenceResult(ManeuverSequence(...)). These are recorded sequential policy choices; the replay setting is only a verification mechanism.

All 184 action sequences

corpus,scenario,policy,actions
ieee118,S0008,jev,T3=0;T3=1;T3=2;T3=1;G36=0.975
ieee118,S0012,jev,G0=0.95;T3=0;T3=1;T3=2
ieee118,S0014,jev,G11=1.025;G11=1.035;G11=1.045;G11=1.05;T3=0;T3=1;T3=2;H0=0;H0=1;T3=1
ieee118,S0016,jev,T3=0;H10=0;H10=1;T3=1;G35=1.05;T3=2;H9=0;H9=1;H9=0;H9=1
ieee118,S0019,jev,H11=0;H11=1;T3=0;G48=0.963;G48=0.953;G48=0.963;T3=1;G48=0.973;T3=2;T4=-2
ieee118,S0029,jev,T3=-2
ieee118,S0039,jev,H11=0;H11=1;G47=0.962
ieee118,S0041,jev,T3=0
ieee118,S0046,jev,G47=0.962;G47=0.972;G47=0.982;G47=0.992;G47=1.002;G47=1.012;G47=1.022;G47=1.032;G47=1.042;G47=1.05
ieee118,S0048,jev,G28=1.04;G28=1.05;T3=0;T3=1;T3=2;H0=0;H0=1;H10=0;H10=1;T3=1
ieee118,S0049,jev,T3=0;T3=1;T3=2;H9=0;H9=1;G52=1.015;G52=1.025
ieee118,S0052,jev,G11=1.025;G11=1.035;G11=1.045;G11=1.05;T3=0;T3=1;T3=2;T4=-2
ieee118,S0053,jev,G28=1.04
ieee118,S0056,jev,T3=1;T3=2;T3=1;H0=0;H0=1;H10=0;H10=1;G41=1.0;G41=0.99;G41=1.0
ieee118,S0057,jev,G35=1.03;G35=1.02;T3=0;T3=1;T3=2;G11=1.025;G11=1.035;G11=1.045;G11=1.05;G42=1.0
ieee118,S0058,jev,G48=0.963;G11=1.025;G11=1.035;G11=1.045;H0=0;H0=1;H0=0;T3=-2;T2=0;T2=1
ieee118,S0059,jev,T3=0;T3=1;T3=2;G35=1.05;H10=0;H10=1;T3=1;G36=0.975;H11=0;H11=1
ieee118,S0069,jev,G11=1.025;G11=1.035;G11=1.045;G11=1.05;T3=0;T3=1;T3=2;H0=0;H0=1;T3=1
ieee118,S0070,jev,G0=0.95;T3=0;T3=1;T3=2;H0=0;H0=1;H0=0;G1=0.988;G1=0.978;G1=0.968
ieee118,S0072,jev,G11=1.025;G11=1.035;G11=1.045;G11=1.05;G5=1.0;T3=0;T3=1;T3=2;H0=0;H0=1
ieee118,S0073,jev,G11=1.025;G11=1.035;G11=1.045;G11=1.05;T3=0;T3=1;T3=2;T4=-2;H0=0;H0=1
ieee118,S0081,jev,T3=0;T3=1
ieee118,S0086,jev,T3=0;T3=1;H0=0;H0=1;H10=0;H10=1;T3=2;H9=0;H9=1;G41=1.0
ieee118,S0087,jev,T3=0;T3=1;T3=2;T3=1;H0=0;H0=1;G30=0.99;G30=1.0;G30=1.01;G30=1.02
ieee118,S0096,jev,T3=0;H1=0;H1=1;T3=1;T3=2;T3=1;H11=0;H11=1;H1=0;H1=1
ieee118,S0097,jev,G28=1.04;G28=1.05;T3=0;T3=1;T3=2;G11=1.025;G11=1.035;G11=1.045;G11=1.05;T4=-2
ieee118,S0102,jev,T3=0;T3=1;T3=2;T3=1;H1=0;H1=1;H11=0;H11=1;G42=1.0;G42=0.99
ieee118,S0105,jev,G28=1.04;G28=1.05;T3=0;T3=1;T3=2;H0=0;H0=1;H10=0;H10=1;G35=1.03
ieee118,S0107,jev,T3=0;T3=1;T3=2;T3=1;H0=0;H0=1;H11=0;H11=1;G41=1.0;G41=0.99
ieee118,S0111,jev,G0=0.95;H1=0;H1=1;T3=0;H0=0;H0=1;H0=0;T3=1
ieee118,S0117,jev,T3=0;T3=1;T3=2;T3=1;H0=0;H0=1;H11=0;H11=1;G41=1.0;G41=0.99
ieee118,S0136,jev,G11=1.025;G11=1.035;G11=1.045;G11=1.05;T3=0;T3=1;T3=2;H0=0;H0=1;T3=1
ieee118,S0137,jev,T3=0;T3=1;T3=2;H0=0;H0=1;H0=0;H10=0;H10=1;H10=0;H10=1
ieee118,S0138,jev,T3=0;T3=1;T3=2;T3=1;H0=0;H0=1;G30=0.99;G30=1.0;G30=1.01;G30=1.02
ieee118,S0141,jev,T3=-2;T2=0;T2=1;T2=2;H8=0;H8=1;G35=1.05;G35=1.04;G35=1.05;T2=1
ieee118,S0166,jev,T3=0;T3=1;T3=2;T3=1;G35=1.05;G35=1.04;H1=0;H1=1;H11=0;H11=1
ieee118,S0167,jev,G28=1.04;G28=1.05;T3=0;T3=1;T3=2;H0=0;H0=1;H10=0;H10=1;T3=1
ieee118,S0168,jev,G11=1.025;G11=1.035;G11=1.045;G11=1.05;T3=-2;T3=-1;H0=0;H0=1;H10=0;H10=1
ieee118,S0170,jev,G41=1.0;G48=0.963;G41=0.99;G47=0.962;G47=0.972
ieee118,S0171,jev,G41=1.0;G48=0.963;G41=0.99;G47=0.962;G47=0.972;G47=0.982;G47=0.992;G47=1.002;G47=1.012;G47=1.022
ieee118,S0175,jev,G28=1.04;G28=1.05;T3=0;T3=1;T3=2;T3=1;H1=0;H1=1;H11=0;H11=1
ieee118,S0177,jev,T3=0;T3=1;T3=2;T3=1;H0=0;H0=1;H11=0;H11=1;G30=0.97;H1=0
ieee118,S0181,jev,T3=0;T3=1;T3=2;T3=1;T3=2;H0=0
ieee118,S0182,jev,G28=1.04;G28=1.05;T3=0;T3=1;T3=2;T3=1;H1=0;H1=1;H11=0;H11=1
ieee118,S0198,jev,G35=1.03;G35=1.02;T3=0;T3=1;T3=2;H9=0;H9=1;T3=1;H10=0;H10=1
ieee118,S0200,jev,G0=0.95;T3=0;H1=0;H1=1;T3=1;T3=2;H0=0;H0=1;H0=0;G1=0.988
pegase89,S0001,jev,T2=-2;T2=-1;T42=0;H28=0;H28=1;G0=1.02888;T42=-1;T42=-2;H27=0;H28=0
pegase89,S0002,jev,T42=0;T42=-1;T42=-2;T41=0;T41=1;T2=-2;T33=0;T33=1;T33=2;T33=1
pegase89,S0003,jev,T2=-2;T2=-1;T42=0;T42=-1;H28=0;H28=1;G0=1.02888;T42=-2;H27=0;H27=1
pegase89,S0004,jev,T42=0;T42=-1;T42=-2;T41=0;T41=1;T2=-2;T33=0;T33=1;T33=2;T33=1
pegase89,S0005,jev,T2=-2;T2=-1;T42=0;T42=-1;H28=0;H28=1;G0=1.02888;T42=-2;H27=0;H27=1
pegase89,S0006,jev,H28=0;H28=1;H28=0;H28=1;H28=0;H28=1;T2=-2;T2=-1;T2=-2;G0=1.02888
pegase89,S0007,jev,T22=-2;T2=-2;T42=0;T42=-1;T42=-2;T41=0;T41=1;H27=0;H27=1;T41=2
pegase89,S0008,jev,T22=-2;T2=-2;T1=0;T1=1;T1=2;T2=-1;H28=0;H28=1;G0=1.02888;H27=0
pegase89,S0009,jev,T2=-2;G0=1.02888
pegase89,S0010,jev,T42=0;T42=-1;T42=-2;T41=0;T41=1;T2=-2;T33=0;T33=1;T33=2;T33=1
pegase89,S0011,jev,T2=-2;T2=-1;T42=0;T42=-1;H28=0;H28=1;G0=1.02888;G9=1.013822;H27=0;H27=1
pegase89,S0012,jev,T22=-2;T2=-2;T1=0;T1=1;T1=2;T2=-1;G0=1.02888;G8=1.04;G8=1.05;H27=0
pegase89,S0013,jev,T22=-2;T2=-2;T1=0;T1=1;H28=0;H28=1;G0=1.02888;H27=0;H27=1;G8=1.04
pegase89,S0014,jev,T2=-2;T2=-1;T42=0;T42=-1;T42=-2;T41=0;T41=1;T41=2;T41=1;G0=1.02888
pegase89,S0015,jev,T42=0;T42=-1;T42=-2;T2=-2;T41=0;T33=0;T33=1;T33=2;T33=1;H28=0
pegase89,S0016,jev,T42=0;T42=-1;T42=-2;T41=0;T41=1;T2=-2;T33=0;T33=1;T33=2;T33=1
pegase89,S0017,jev,T34=-2
pegase89,S0018,jev,T2=-2;T2=-1;H28=0;H28=1;T2=-2;G0=1.02888;H27=0;H27=1;G8=1.04;G8=1.05
pegase89,S0019,jev,T42=0;T42=-1;T42=-2;T1=0;T1=1;T1=2;T7=0;T7=1
pegase89,S0020,jev,T22=-2;T21=0;T21=1;T21=2;T21=1;G0=1.02888;T34=-2;T33=0;T21=0;T21=-1
pegase89,S0021,jev,T42=0;T2=-2;T2=-1;T2=-2;H28=0;H28=1;G0=1.02888;H27=0;H27=1;G8=1.04
pegase89,S0022,jev,H28=1;H28=0;H28=1;T22=-2;G0=1.02888;H27=0;H27=1;G8=1.04;G8=1.05;T1=0
pegase89,S0023,jev,T2=-2;T2=-1;T42=0;H28=0;H28=1;G0=1.02888;T2=-2;H27=0;H27=1;G8=1.04
pegase89,S0024,jev,T2=-2;T2=-1;T42=0;H28=0;H28=1;T2=-2;T42=-1;T42=-2;H28=0;H28=1
pegase89,S0025,jev,H28=1;H28=0;H28=1;T22=-2;H27=0;H27=1;T14=0;T14=1;G0=1.02888;G8=1.04
pegase89,S0026,jev,T22=-2;T2=-2;T1=0;T1=1;T1=2;T2=-1;H28=0;H28=1;G0=1.02888;H27=0
pegase89,S0027,jev,T22=-2;T21=0;T21=1;T21=2;T21=1;T7=0;T7=1;T7=2;G0=1.02888;G0=1.01888
pegase89,S0028,jev,T34=-2;T33=0;T33=1;T33=2;T33=1;T21=0;T21=1;T21=2;T21=1;G0=1.02888
pegase89,S0029,jev,T22=-2;T7=0;T7=1;T7=2;T21=0;T21=1;T21=2;T22=-1;G0=1.02888;G8=1.04
pegase89,S0030,jev,T22=-2;T2=-2;T1=0;T1=1;H28=0;H28=1;T1=2;H27=0;H27=1;T2=-1
pegase89,S0031,jev,T22=-2;G0=1.02888;G0=1.01888;T14=0;T14=1;G8=1.04;G8=1.05;T21=0;T21=1;H27=0
pegase89,S0032,jev,T2=-2;T2=-1;T42=0;T42=-1;H28=0;H28=1;G0=1.02888;T42=-2;H27=0;H27=1
pegase89,S0033,jev,T2=-2;T2=-1;T42=0;H28=0;H28=1;T2=-2;T42=-1;T42=-2;H28=0;H28=1
pegase89,S0034,jev,T42=0;T42=-1;T42=-2;T41=0;T41=1;T41=2;T41=1;T2=-2;T33=0;T33=1
pegase89,S0035,jev,T42=0;T42=-1;T42=-2;T2=-2
pegase89,S0036,jev,T22=-2;G0=1.02888;G8=1.04;G8=1.05;H27=0;H27=1;T14=0;T14=1;T14=2;T14=1
pegase89,S0037,jev,T42=0;T42=-1;T42=-2;T41=0;T41=1;T2=-2;T33=0;T33=1;T41=2;T41=1
pegase89,S0038,jev,T1=0;T1=1;T1=2;G0=1.02888;G0=1.01888;G0=1.00888;G0=1.01888;G0=1.00888;G0=1.01888;G0=1.00888
pegase89,S0039,jev,T2=-2;T2=-1;T42=0;H28=0;H28=1;G0=1.02888;H28=0;H28=1;T42=-1;T42=-2
pegase89,S0040,jev,T2=-2;T2=-1;T42=0;H28=0;H28=1;G0=1.02888;G9=1.013822;H27=0;H27=1;G0=1.01888
pegase89,S0041,jev,T2=-2;T2=-1;H28=0;H28=1;T42=0;T42=-1;T42=-2;T41=0;T41=1;T41=2
pegase89,S0042,jev,T42=0;T2=-2;T2=-1;T2=-2;H28=0;H28=1;G0=1.02888;H27=0;H27=1;G8=1.04
pegase89,S0043,jev,T42=0;T42=-1;T42=-2;T41=0;T41=1;T2=-2;T33=0;T33=1;T33=2;T33=1
pegase89,S0044,jev,T22=-2;T14=0;T14=1;T7=0;T7=1;H28=0;H28=1;T14=2;T1=0;T7=2
pegase89,S0045,jev,G0=1.02888;T14=0;G8=1.04;G8=1.05;T14=1;T14=2;T14=1;T21=0;T21=1;T21=2
pegase89,S0046,jev,T1=0;T1=1;T1=2;H17=0;H17=1;G0=1.02888;G0=1.01888;G0=1.00888;G0=1.01888;T1=1
ieee118,S0008,random,G47=0.962;G29=0.974;H11=0;G29=0.984;G5=1.0;G41=1.0
ieee118,S0012,random,G7=0.963;T3=0;T3=1;G36=0.995;G48=0.963;H11=0;G47=0.962;T3=2
ieee118,S0014,random,G5=1.0;H11=0
ieee118,S0016,random,T3=0;G0=0.95;G35=1.05;G12=0.958;G47=0.962;G35=1.04;G0=0.96
ieee118,S0019,random,G0=0.95;G28=1.04;T3=0;G40=0.97;T3=1;G6=0.96;G12=0.958;G23=0.964;H1=0;G48=0.963
ieee118,S0029,random,G41=0.98;G23=0.964;H0=0;G48=0.963;G36=0.975;G36=0.985;H10=0;G18=0.975;G18=0.965;G18=0.955
ieee118,S0039,random,G28=1.04;G39=0.995;G47=0.962
ieee118,S0041,random,G18=0.995;G36=0.995;G5=1.0;G37=1.005;G13=0.957;G31=0.981;G24=0.995;G13=0.95;H11=0;G31=0.991
ieee118,S0046,random,G34=1.016;T3=-2;G0=0.95;G22=0.962;G23=0.964;G40=0.97;G11=1.025;H0=0;G47=0.962;G35=1.05
ieee118,S0048,random,G17=0.98;G17=0.99;G11=1.025;G17=1.0;G5=1.0;G42=1.02;G23=0.964;H12=0;G49=0.97;G29=0.994
ieee118,S0049,random,G47=0.962;T3=0;H10=0;G19=0.995;H0=0
ieee118,S0052,random,G17=0.98;G24=0.995;G24=1.005;G37=1.025;H11=0;G25=0.985;G24=0.995;G17=0.99;H11=1;G49=0.97
ieee118,S0053,random,G23=0.95;G37=1.005;G0=0.95;T3=0;G0=0.96;T3=1;G0=0.95;G37=1.015;H12=0;G0=0.96
ieee118,S0056,random,H0=0;H11=0;G12=0.978;G42=1.0;T3=1;G0=0.95;G19=1.015;G19=1.025;H0=1;G25=0.985
ieee118,S0057,random,G18=0.975;G48=0.963;G47=0.962;T3=0;G0=0.95;G48=0.973;H1=0;G23=0.964;H1=1;T3=1
ieee118,S0058,random,T3=-2;G0=0.95;G36=0.995;G12=0.958;H9=0;H9=1;H9=0;G17=0.98;G6=0.96;T2=0
ieee118,S0059,random,G48=0.963;H11=0;G18=0.975;G35=1.05;H11=1;H11=0;G18=0.965;G5=1.0;G12=0.978;T3=0
ieee118,S0069,random,G11=1.025;G30=0.99;G30=1.0;G24=0.995;G48=0.963;G11=1.035;H11=0;G18=0.975;T3=0;G18=0.965
ieee118,S0070,random,H11=0;G48=0.963;T3=0;G37=1.005;G48=0.973;G48=0.963;G42=1.0;H1=0;G19=1.015;G25=0.985
ieee118,S0072,random,H10=0;G17=0.98;G5=1.0;G25=0.985;H11=0;G48=0.963;G6=0.98;G42=1.02;G42=1.03;G18=0.995
ieee118,S0073,random,G25=0.985;G31=0.981;G17=0.98;G24=0.995;G5=1.0;G25=0.995;H2=0;G49=0.97;G0=0.965;G25=1.005
ieee118,S0081,random,H11=0;G12=0.958;G17=0.98
ieee118,S0086,random,G36=0.995;G48=0.963;T3=0;T3=1;G41=1.0;H0=0;G36=0.985;G41=0.99;G41=1.0;T3=2
ieee118,S0087,random,H0=0;G24=0.995;G30=0.99;G18=0.995;G37=1.005;G12=0.978
ieee118,S0096,random,H11=0;H11=1;G12=0.958;G18=0.995;G5=1.0;G24=0.995;G43=1.007;T4=-2;G24=1.005;G18=1.005
ieee118,S0097,random,G5=1.0;G36=0.975;G42=1.0;G6=0.98;G48=0.983;G49=0.97;G36=0.965;G49=0.96;G24=0.975;G19=0.995
ieee118,S0102,random,H1=0;G0=0.95;G25=0.985;G8=0.972;H1=1;G32=0.95;G43=1.007;G1=0.988;T4=-2;G7=0.983
ieee118,S0105,random,G28=1.04;G0=0.95
ieee118,S0107,random,G36=0.995;G5=1.0;G12=0.978;G42=1.02;H11=0;G31=0.981;H1=0;G48=0.983;G19=1.015;G37=1.025
ieee118,S0111,random,G7=0.963;G36=0.975;G0=0.95;T3=0;G42=1.0;G24=0.975;G36=0.985;G7=0.973;G36=0.975;G30=0.97
ieee118,S0117,random,G5=1.0;G42=1.02;G12=0.978;H1=0;G25=0.985;G6=0.98;G19=1.015;H12=0;G19=1.025;G25=0.975
ieee118,S0136,random,G11=1.025;H1=0;G37=1.005;G5=1.0;H1=1;H11=0;G0=0.965;G0=0.955;G18=0.995;T3=0
ieee118,S0137,random,T3=0;H0=0
ieee118,S0138,random,G24=0.995;G5=1.0;G42=1.02;H11=0;G7=0.963;G7=0.953;G38=0.995;G19=1.015;H11=1;G37=1.025
ieee118,S0141,random,G0=0.95
ieee118,S0166,random,G18=0.975;G18=0.965;G29=0.974;G17=0.98;T3=0;T3=1;G35=1.05;H1=0;G48=0.983;G22=0.962
ieee118,S0167,random,G0=0.95;G29=0.974;G42=1.0;G28=1.04;G12=0.958;G22=0.962;G29=0.984;T3=0;G12=0.95;G18=0.975
ieee118,S0168,random,G17=0.98;G41=0.98;G5=1.0;G30=0.97;H10=0;G48=0.963;G18=0.995;G36=0.975;G36=0.965;G42=1.0
ieee118,S0170,random,G48=0.963;H1=0;H1=1;H1=0;G0=0.95;G17=0.98;G48=0.973;G48=0.963;G41=1.0;T3=0
ieee118,S0171,random,G17=0.98;G24=0.995;G48=0.963;G24=1.005;G5=1.0;G37=1.025;T4=-2
ieee118,S0175,random,G23=0.95;G29=0.974;G5=1.0;G0=0.965;G0=0.955;G0=0.965;G0=0.955;G24=0.975;G6=0.96;G18=0.975
ieee118,S0177,random,G37=1.005;G5=1.0;G0=0.965;G18=0.995;G48=0.983;G37=0.995;G42=1.0;H11=0;G19=0.995;G6=0.96
ieee118,S0181,random,T3=0;G37=1.005;G13=0.977;G13=0.987;G31=1.001;G13=0.977;G0=0.95;G31=1.011;G48=0.983;G31=1.021
ieee118,S0182,random,G35=1.03;G0=0.95;G41=0.98;G18=0.995;G48=0.963;G0=0.96;G35=1.02;T3=0;G28=1.04;G40=0.97
ieee118,S0198,random,G35=1.03;G29=0.974;H10=0;T3=0;G48=0.963;G18=0.975;T4=-2;G11=1.025;G22=0.962;T3=1
ieee118,S0200,random,H11=0;G7=0.963;G42=1.0;G42=0.99;G37=1.005;T3=0;G0=0.95;G25=0.985;G48=0.983;G42=1.0
pegase89,S0001,random,H8=0;G0=1.02888;T9=-2;G9=1.013822;H38=0;H28=0;H8=1;H18=0;T17=-2;H28=1
pegase89,S0002,random,T9=-2;G8=1.04;G0=1.02888;T34=-2;T7=-2;H7=0;T21=0;H36=0;T42=0;H7=1
pegase89,S0003,random,H18=0;T9=-2;H17=0;H7=0;T1=0;H17=1;T22=-2;T34=-2;H27=0;G0=1.02888
pegase89,S0004,random,G0=1.02888;T34=-2;H8=0;H28=0;H38=0;T33=0;H28=1;H18=0;G0=1.01888;H27=0
pegase89,S0005,random,H28=0;T17=-2;T2=-2;T9=-2;H7=0;H27=0;T22=-2;H17=0;H7=1;T22=-1
pegase89,S0006,random,G0=1.02888;T34=-2;H27=0;T42=0;G8=1.04;T33=0;H7=0;H7=1;G8=1.05;H7=0
pegase89,S0007,random,T2=-2;T2=-1;T2=-2;T22=-2;H37=0;G0=1.02888;T34=-2;H7=0;G0=1.01888;H7=1
pegase89,S0008,random,T22=-2;T34=-2;H18=0;G9=1.013822;H37=0;T21=-2
pegase89,S0009,random,G0=1.02888;T1=0;H36=0;H36=1;T14=0;G0=1.01888;T14=1;G0=1.00888;T22=-2;T1=1
pegase89,S0010,random,T2=-2;G9=1.013822;T2=-1;G9=1.003822;H18=0;G0=1.02888;H8=0;T22=-2;T7=0;H37=0
pegase89,S0011,random,T42=0;H8=0;T2=-2;T42=-1;H28=0;H38=0;T34=-2;H18=0;T9=-2;H36=0
pegase89,S0012,random,T22=-2;H18=0;H8=1;T21=0;T7=0;T7=1;T42=0;T7=2;H8=0;H28=0
pegase89,S0013,random,T17=-2;H38=0;H8=0;H38=1;T2=-2;T22=-2;H27=0;H8=1;T22=-1;G9=1.013822
pegase89,S0014,random,G9=1.013822;T9=-2;T7=0;T7=1;T7=2;H8=0;T42=0;T17=0;T1=0;H37=0
pegase89,S0015,random,H8=0;H28=0;H28=1;H38=0;H18=0;T22=-2;T21=0;H38=1;H38=0;H28=0
pegase89,S0016,random,H38=0;H8=0;T42=0;H18=0;T22=-2;T14=0;T34=-2;T21=-2;T42=-1;H37=0
pegase89,S0017,random,G0=1.02888;T41=0
pegase89,S0018,random,H18=0;T34=-2;T7=0;H28=0;T33=0;T7=1;H8=0;H28=1;H18=1;T33=1
pegase89,S0019,random,T42=0;T34=-2;G9=1.013822;T21=0;H37=0;H27=0;T21=1;T21=2;G0=1.02888;G0=1.01888
pegase89,S0020,random,T9=-2;H28=0;G0=1.02888;H37=0;T17=-2;H16=0;T14=0;T14=1;T22=-2;H6=0
pegase89,S0021,random,T42=0;H8=0;G0=1.02888;G9=1.013822;H18=0;T9=-2;T22=-2;H37=0;G8=1.04;H8=1
pegase89,S0022,random,T2=-2;G9=1.013822;G0=1.02888;T9=-2;H27=0;T22=-2;G0=1.01888;T1=-2;H17=0;H17=1
pegase89,S0023,random,T42=0;T22=-2;G0=1.02888;G8=1.04;G0=1.01888;T42=-1;T42=-2;H37=0;H17=0;T33=0
pegase89,S0024,random,G9=1.013822;T17=-2;G9=1.003822;H17=0;G0=1.02888;T1=0;G0=1.01888;G0=1.00888;T1=1;G9=0.993822
pegase89,S0025,random,G0=1.02888;T2=-2;G8=1.04;G8=1.05;T1=0;T34=-2
pegase89,S0026,random,G9=1.013822;H28=0;T9=-2;H28=1;G0=1.02888;T22=-2;G0=1.01888;T14=0;T7=-2;T21=0
pegase89,S0027,random,H38=0;T2=-2;T17=-2;T17=-1;T42=0;H8=0;T9=-2;H7=0;G8=1.04;G8=1.05
pegase89,S0028,random,H18=0;G9=1.013822;T2=-2;T42=0;H17=0;H27=1;H7=0;T1=0;T9=-2;H17=1
pegase89,S0029,random,H18=0;T34=-2;H28=0;T21=0;T21=1;T14=0;G0=1.02888;T21=2;G8=1.04;H7=0
pegase89,S0030,random,H38=0;G0=1.02888;H18=0;G0=1.01888;H8=0;G0=1.00888;H28=0;T34=-2;T21=0;T7=0
pegase89,S0031,random,T42=0;T22=-2;H18=0;T2=-2;H37=0;G9=1.013822;H17=0;T7=0;H27=0;T14=0
pegase89,S0032,random,T34=-2;T33=0;H28=0;T33=1;H38=0;H8=0;H18=0;T2=-2;H37=0;T21=0
pegase89,S0033,random,T34=-2;T7=0;T2=-2;H18=0;H8=0;H18=1;T33=0;T21=0;H28=0;H18=0
pegase89,S0034,random,T42=0;T34=-2;T14=0;H28=0;H18=0;T2=-2;T33=0;T33=1;H28=1;T42=-1
pegase89,S0035,random,H8=0;G0=1.02888;H18=0;G9=1.013822;H38=0;T2=-2;T22=-2;H8=1;T34=-2
pegase89,S0036,random,T22=-2;H17=0;H37=0;H7=0;H27=0;H7=1;G0=1.02888;H7=0;T42=0;T21=0
pegase89,S0037,random,H38=0;H18=0;T42=0;H18=1;T9=-2;H17=0;T34=-2;H37=0;H37=1;H27=0
pegase89,S0038,random,H37=0;H7=0;T21=0;H37=1;H17=0;H17=1;H7=1;H27=0;H7=0;H17=0
pegase89,S0039,random,T17=-2;T42=0;H38=0;H18=0;T14=0;G9=1.013822;H28=0;T7=0;T14=1;G9=1.003822
pegase89,S0040,random,H28=0;G9=1.013822;H8=0;H18=0;G0=1.02888;T14=0;T1=0;G9=1.003822;T17=-2;T7=0
pegase89,S0041,random,T9=-2;G0=1.02888;T17=-2;H37=0;H7=0;T22=-2;T34=-2;T21=-2;G8=1.04;T6=0
pegase89,S0042,random,T42=0;H38=0;T42=-1;H28=0;T2=-2;G9=1.013822;T9=-2;G0=1.02888;T17=0;T17=1
pegase89,S0043,random,H28=0;G0=1.02888;T17=-2;T7=0;T7=1;T34=-2;H8=0;H8=1;G9=1.013822;G9=1.003822
pegase89,S0044,random,G9=1.013822;H28=0;T17=-2;G9=1.003822;T1=0;T14=0;G9=0.993822;H38=0;T42=0;T14=1
pegase89,S0045,random,T2=-2;H17=0;H17=1;H37=0;H17=0;G0=1.02888;H26=0;T1=-2;T22=-2;T14=0
pegase89,S0046,random,T41=0

This contribution reports a distinct typed-choice scaffold and a matched baseline; it is not a request to merge these rates into the existing chat-agent leaderboard.

Navigation Menu

JEV typed-choice policy: complete 92-case evaluation against a seeded-random baseline #1

JEV on RestoreBench: typed legal-action selection

Results

Protocol

Native evaluator verification

Paired outcomes and resource use

Per-scenario data

Minimal JEV runner

Exact recorded action sequences

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

JEV typed-choice policy: complete 92-case evaluation against a seeded-random baseline #1

Description

JEV on RestoreBench: typed legal-action selection

Results

Protocol

Native evaluator verification

Paired outcomes and resource use

Per-scenario data

Minimal JEV runner

Exact recorded action sequences

Activity

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions