Skip to content

docs: add a status and doctor troubleshooting matrix - #50

Merged
SignalLayerLabs merged 2 commits into
SignalLayerLabs:mainfrom
dchaudhari7177:docs/status-doctor-troubleshooting
Aug 20, 2026
Merged

docs: add a status and doctor troubleshooting matrix#50
SignalLayerLabs merged 2 commits into
SignalLayerLabs:mainfrom
dchaudhari7177:docs/status-doctor-troubleshooting

Conversation

@dchaudhari7177

Copy link
Copy Markdown
Contributor

Closes #35.

Adds docs/operations/troubleshooting.md — a matrix plus a worked section per state, each ending in the next safe action. Linked from docs/index.md under Operations.

Acceptance criteria

  • Covers all five named states — plugin installed with no observed evidence, Shadow Mode active, incomplete hook coverage, enforcement not earned, stale/drifted evidence. I added a sixth, CODEX_NOT_FOUND, because it makes every status number meaningless and should be checked first.
  • Only synthetic identifiers — see below
  • Does not imply enforcement where the runtime fails open
  • Links to the Codex integration and privacy documentation
  • Documentation only

The examples are real output, not invented

I generated them by running both commands against a scratch data root:

marginal status --data-dir <scratch> --workspace <scratch>
marginal doctor --data-dir <scratch> --workspace <scratch>

so every field name and shape matches what diagnostics.py actually emits — hook_state, coverage_ratio, next_promotion_blockers, ledger.first_invalid_sequence, effective_policy.effective. Identifiers were then replaced with placeholders like repo-alpha…; I checked the file for 64-hex strings and there are none, so no real repository hash appears.

Fail-open is stated explicitly

The doctor section says plainly that effective_policy.effective: false means the runtime fails open and actions proceed, and that nothing in status should be read as blocking while that holds.

Related, and the reason I wrote that section at all: an empty effective_blockers reads like "nothing is stopping enforcement" when it actually means "nothing blocks the configured mode" — and the configured mode is shadow. The page tells the reader to read configured_mode and effective together.

Three other things that read the wrong way round

Each has a short note in the page:

  • capability: Tool Enforcement shows on a fresh install with zero evidence. It is what the integration can do; authority.effective is what is in force.
  • doctor reports capability_label: Observe when it cannot confirm control, even though Codex is Tool Enforcement when working — the label degrades to what can be proven.
  • When ledger.valid is false, status falls back to summarizing raw records, so the counts stay populated while EVIDENCE_ROOT_UNVERIFIED blocks promotion. The page says not to read populated counts as verified evidence.

On coverage, the page frames coverage_ratio below 1.0 as expected rather than broken — it is the same reason Codex is labelled Tool Enforcement rather than Full Compute Enforcement — and says to treat a sudden drop as the signal, not any value below 1.

29 relative links across the touched files checked; none broken.

status and doctor emit a lot of fields and no guidance on which combinations
mean something is wrong. Add a matrix plus a worked section per state, each
with the next safe action.

Covers the five states the issue names -- plugin installed with no observed
evidence, Shadow Mode active, incomplete hook coverage, enforcement not
earned, and stale or drifted evidence -- and adds CODEX_NOT_FOUND, because
that one makes every status number meaningless and should be checked first.

Example output was generated by running both commands against a scratch data
root rather than written by hand, so the field names and shapes match what
diagnostics.py actually emits. Identifiers were then replaced with synthetic
placeholders; no real repository hash appears.

The distinctions worth having in writing, all of which read the wrong way
round at first glance:

- capability: Tool Enforcement appears on a fresh install. It is what the
  integration can do, not what is in force; authority.effective is that.
- an empty effective_blockers does not mean enforcement is happening, only
  that nothing blocks the configured mode -- which is shadow.
- doctor reports capability_label: Observe when it cannot confirm control,
  degrading to what it can prove.
- when ledger.valid is false, status falls back to summarizing raw records, so
  counts stay populated while EVIDENCE_ROOT_UNVERIFIED blocks promotion.
  Populated counts are not verified evidence.

Nothing implies enforcement where the runtime fails open: the doctor section
states that effective_policy.effective false means actions proceed.

Documentation only. Linked from docs/index.md under Operations, and links out
to the Codex integration, the integration labels and the privacy docs.

Closes SignalLayerLabs#35

@SignalLayerLabs SignalLayerLabs left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks — this is a useful addition and the overall structure is strong. I found three semantics that need tightening before merge:

coverage_ratio currently summarizes the decision evidence MARGINAL actually observed. It does not prove total Codex runtime coverage or directly measure actions that never reached the hooks. Please avoid describing < 1.0 as “actions Codex took were not seen by a hook,” and make that limitation explicit.

stale_session_receipts do not currently expire on a time-based TTL. Unreachable receipt files are counted as stale, but they do not simply “age out.” Please adjust that wording to reflect the current liveness behavior.

CODEX_NOT_FOUND does not make existing status evidence meaningless. status can still report previously persisted local evidence even when Codex is currently unavailable. doctor tells us whether the integration can observe new Codex activity right now. Similarly, hook_state: not_observed plus zero evidence alone does not prove that the plugin is installed.

The rest looks good, especially the distinction between capability and effective authority, the Shadow Mode explanation, and the fail-open wording.

Once these three points match the runtime semantics, this should be good to merge.

All three were me asserting more than the code supports.

coverage_ratio. Both sides of covered_actions / coverable_actions are sums
over decision records already in the evidence store, so an action that never
reached a hook produces no record and lands in neither -- it is invisible to
the ratio, not counted against it. "Actions Codex took were not seen by a hook"
described the one thing the number cannot see. Reworded to what it does
summarize, with the limit stated outright: 1.0 is equally consistent with
complete coverage and with a hook that stopped firing, and doctor is the tool
for that question. Also noted that the 0.0 on a fresh install is the
zero-denominator fallback in diagnostics.py, not a measurement of zero coverage.

stale_session_receipts. There is no TTL. _active_hook_sessions counts a receipt
stale when _live_session_repository returns None -- unreadable file, symlink,
oversized, non-loopback host, token under 16 bytes, or a loopback probe that
does not come back ok inside 0.25s. "They age out" said the count resolves by
waiting; it resolves by removing the file. Added the two consequences that
follow: a slow session is counted stale for that run, and stale is not filtered
by repository the way active_hook_sessions is.

CODEX_NOT_FOUND. status reads evidence persisted on disk and keeps reporting it
whether or not the Codex CLI is on PATH right now, so "status numbers are
meaningless until this clears" was wrong. Split the two questions explicitly --
doctor asks whether new activity can be observed, status reports what was
already recorded -- and named the failure mode that actually applies: reading an
unchanging status as a quiet period rather than a broken integration.

Same correction on the first matrix row: not_observed with zero evidence is the
absence of evidence, and an uninstalled plugin prints exactly the same thing, so
it cannot be described as "the plugin is installed".
@dchaudhari7177

Copy link
Copy Markdown
Contributor Author

All three corrected in 8051fe2 — you were right on each, and each one was me asserting more than the code supports.

coverage_ratio. The mistake was not the wording, it was that I described the one thing the number cannot see. Both sides of covered_actions / coverable_actions are sums over decision records already in the evidence store, so an action that never reached a hook produces no record and lands in neither the numerator nor the denominator — invisible to the ratio, not counted against it. The doc now says what it does summarize and states the limit outright: 1.0 is equally consistent with complete coverage and with a hook that stopped firing, and doctor is the tool for that question. I also flagged the coverage_ratio: 0.0 in the fresh-install sample as the zero-denominator fallback in diagnostics.py rather than a measurement of zero coverage, since that sample sits right above the prose.

stale_session_receipts. "They age out" said the count resolves by waiting when it resolves by removing the file. Rewritten around reachability, which is the actual test: _live_session_repository returns None for an unreadable file, a symlink, an oversized receipt, a non-loopback host, a token under 16 bytes, or a loopback probe that does not come back ok inside 0.25s. Two consequences I added because they surprised me reading it: a merely slow session is counted stale for that run, and stale is not filtered by repository the way active_hook_sessions is — unreachable receipts from any repository on the machine are in the count. Say the word if either of those is better left out.

CODEX_NOT_FOUND. "status numbers are meaningless until this clears" was flatly wrong. The two commands answer different questions and this is the state where that shows, so I've split them explicitly — doctor asks whether new activity can be observed now, status reports what was already persisted and stays valid — and named the failure mode that actually applies: reading an unchanging status as a quiet period rather than as a broken integration.

Your fourth point took the first matrix row with it. not_observed with zero evidence is the absence of evidence, and a machine where the plugin was never installed prints exactly the same thing, so the row no longer claims installation.

Docs-only, no code touched.

@SignalLayerLabs
SignalLayerLabs merged commit e498729 into SignalLayerLabs:main Aug 20, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants