What
skills/issue-reassess/ is the only one of the 71 shipped skills with no behavioural eval suite — there is no tools/skill-evals/evals/issue-reassess/ directory. (The 71st entry under evals/, non-asf-profile-smoke, is a profile smoke, not a skill suite.)
comm -23 <(ls skills | grep -v pyproject | sort) \
<(ls tools/skill-evals/evals | sort)
# issue-reassessThe validator already knows:
skills/issue-reassess/SKILL.md (1 warning)
eval-coverage: no eval suite at tools/skill-evals/evals/issue-reassess/ — add one before shipping
Why it matters
AGENTS.md is explicit: "ship a behavioural eval suite under tools/skill-evals/evals/<skill-name>/ […] A skill PR without a matching eval suite is incomplete." The skill shipped anyway and is in active use — it has a row in the docs/mode-economics.md Triage table — so regressions in its classification and aggregation behaviour are currently invisible. The eval-coverage check (#9) is deliberately SOFT so in-flight eval PRs don't fail the gate, which means this gap stays silent until the skill misbehaves in front of a user.
Suggested fix
Add the suite. The skill's surfaces look directly fixturable: the per-issue verdict flow (skills/issue-reassess/per-issue-flow.md), pool selection (pool-selection.md), and verdict aggregation (verdict-aggregation.md). The sibling issue-reassess-stats suite (step-1-fetch-verdicts / step-2-classify / step-3-aggregate) is the natural model, and it already contains reassess-shaped fixture reports to crib from.
Related
What
skills/issue-reassess/is the only one of the 71 shipped skills with no behavioural eval suite — there is notools/skill-evals/evals/issue-reassess/directory. (The 71st entry underevals/,non-asf-profile-smoke, is a profile smoke, not a skill suite.)The validator already knows:
Why it matters
AGENTS.mdis explicit: "ship a behavioural eval suite undertools/skill-evals/evals/<skill-name>/[…] A skill PR without a matching eval suite is incomplete." The skill shipped anyway and is in active use — it has a row in thedocs/mode-economics.mdTriage table — so regressions in its classification and aggregation behaviour are currently invisible. The eval-coverage check (#9) is deliberately SOFT so in-flight eval PRs don't fail the gate, which means this gap stays silent until the skill misbehaves in front of a user.Suggested fix
Add the suite. The skill's surfaces look directly fixturable: the per-issue verdict flow (
skills/issue-reassess/per-issue-flow.md), pool selection (pool-selection.md), and verdict aggregation (verdict-aggregation.md). The siblingissue-reassess-statssuite (step-1-fetch-verdicts / step-2-classify / step-3-aggregate) is the natural model, and it already contains reassess-shaped fixture reports to crib from.Related
tools/skill-and-tool-validatorcheck .asf.yaml: route all notification schemes (suppress dev@ default) #9 (validate_eval_coverage) — the SOFT advisory that flags this today.tools/skill-evals/evals/issue-reassess-stats/— the sibling suite to mirror.