feat(skills): evaluate global intent for SAMPLE_OR_TEST viability

Change-Id: Idcec29bebf7b4ecacd3e280dbad55271c2b77e7e

Author: nickg-ca

SCHEMA.md

@@ -39,7 +39,7 @@ evolves sequentially as different skills process it.
 
 -   **`production_viability`** (Enum): Whether the bug is triggerable in a
     release build.
-    -   Values: `"VIABLE"`, `"NON_VIABLE"`
+    -   Values: `"VIABLE"`, `"NON_VIABLE"`, `"SAMPLE_OR_TEST"`
 -   **`critic_reasoning`** (String): Rationale for viability (e.g., "Not
     protected by allocator padding").
 
@@ -127,8 +127,8 @@ trajectory.
     remove", "target_entity": "[e.g., auth_module.py]", "insight":
     "[description]", "source_stage": "[e.g., mantis_researcher]"}`
 -   Alternatively for findings: `{"title": "[finding_title]", "code_paths":
-    ["[path1:line1]"], "status": "[VIABLE / NON_VIABLE / FALSE_POSITIVE /
-    VERIFIED_SECURE / VERIFICATION_FAILED / ERROR]"}`
+    ["[path1:line1]"], "status": "[VIABLE / NON_VIABLE / SAMPLE_OR_TEST /
+    FALSE_POSITIVE / VERIFIED_SECURE / VERIFICATION_FAILED / ERROR]"}`
 
 This file serves as an ephemeral inbox/queue for new learnings. It is
 periodically synthesized into the permanent Markdown Knowledge Base

mantis_calibrate/SKILL.md

@@ -97,6 +97,14 @@ Execute the calibration as follows:
                     paths), drastically reduce the multiplier to 0.2.
             -   If `workspace/kb/THREAT_MODEL.md` does not exist or does not
                 mention the components, default the Multiplier to 1.0.
+        -   If `production_viability` is **SAMPLE_OR_TEST**:
+            -   Set the Context Multiplier to a reduced value (e.g. `0.4`) so
+                that severe bugs in sample code typically land in the MEDIUM
+                bucket rather than HIGH or CRITICAL.
+            -   In the `executive_summary`, explicitly state that this is not a
+                production bug. The recommendation MUST focus on fixing the
+                example/test so that developers do not copy insecure patterns
+                into production code.
 
     **Final Score (Hazard) = (Impact + Likelihood) * Multiplier** (Capped at
     10.0).

mantis_chain/SKILL.md

@@ -32,7 +32,7 @@ Execute the chaining stage as follows:
 
     -   Read the JSON files in the `workspace/findings/` directory. Filter for
         findings that have passed validation (e.g., status is `"VALID"` or
-        viability is `"VIABLE"`).
+        viability is `"VIABLE"` or `"SAMPLE_OR_TEST"`).
     -   Read the Markdown Knowledge Base (`workspace/kb/entities/` and
         `workspace/kb/vulnerabilities/`) to identify architectural primitives
         that might not be bugs on their own, but could serve as stepping stones

mantis_critic/SKILL.md

@@ -34,14 +34,21 @@ Execute the critic evaluation as follows:
     that false positives can be properly logged to long-term memory. If none
     exist, notify the user.
 
-2.  **Acquire Targeted Code Snippets:** For each finding where `status` is
+2.  **Evaluate Global Repository Intent:** Read `workspace/kb/THREAT_MODEL.md`
+    (if it exists). Check the **Deployment Intent** section. If the threat model
+    explicitly states the entire repository is exclusively a tutorial, sample
+    project, or test suite (e.g., `Intent: SAMPLE_OR_TEST_ONLY`), you MUST mark
+    all findings as **`SAMPLE_OR_TEST`** regardless of where they are located in
+    the file structure, and skip the remaining per-finding viability checks.
+
+3.  **Acquire Targeted Code Snippets:** For each finding where `status` is
     `"VALID"`, read the target file. Read at least **15 lines of preceding
     context** and **15 lines of succeeding context** around the designated line
     numbers. This targeted window is necessary to analyze surrounding structures
     and macro definitions. (Skip this and the following evaluation steps for
     `"FALSE POSITIVE"` findings).
 
-3.  **Evaluate Domain-Specific Viability Constraints:**
+4.  **Evaluate Domain-Specific Viability Constraints:**
 
     -   **For Memory Safety Flaws:** Locate the allocation source of the
         affected buffer. Determine if it is allocated with safety margins or
@@ -52,7 +59,7 @@ Execute the critic evaluation as follows:
         deployments. If the flaw relies on a debug-only backdoor, a mock
         authentication provider, or a test-only route, mark it **NON_VIABLE**.
 
-4.  **Check Critical Non-Viable Criteria:** Mark a finding as **NON_VIABLE** if
+5.  **Check Critical Non-Viable Criteria:** Mark a finding as **NON_VIABLE** if
     it meets any of the following criteria to ensure we do not waste patching
     resources on unreachable or compiled-out code:
 
@@ -65,37 +72,42 @@ Execute the critic evaluation as follows:
     -   **Debug-Only Features:** Security flaws that exist inside files or
         sections conditionally compiled with debug flags (e.g. `#ifdef DEBUG`)
         are NON_VIABLE.
-    -   **Harnesses & Mocks:** Issues residing in the helper configurations of
-        testing libraries, fuzzing suites, or validation frameworks rather than
-        the production library core are NON_VIABLE.
-
-5.  **Token-Optimized File Updates:** To minimize LLM output tokens, **do not
+    -   **Harnesses, Mocks, & Examples:** Issues residing in example code, test
+        suites, fuzzing harnesses, or validation frameworks are technically not
+        deployed to production. However, because developers often copy sample
+        code, these should NOT be marked NON_VIABLE. Instead, mark them as
+        **SAMPLE_OR_TEST** so the pipeline can properly adjust their risk
+        severity.
+
+6.  **Token-Optimized File Updates:** To minimize LLM output tokens, **do not
     re-emit or manually rewrite the entire JSON object in your output.**
     Instead, use in-place editing tools (like a short script in your preferred
     language, or `jq`) to programmatically append the new fields to the existing
     `workspace/findings/<id>.json` file.
 
     You must append the following to the existing object:
 
-    -   A `"production_viability"` field (either `"VIABLE"` or `"NON_VIABLE"`).
+    -   A `"production_viability"` field (`"VIABLE"`, `"NON_VIABLE"`, or
+        `"SAMPLE_OR_TEST"`).
     -   A `"critic_reasoning"` field explaining your evaluation.
     -   An entry to the `"history"` array:
 
     ```json
     {
       "stage": "critic",
       "action": "evaluated",
-      "details": "Determined production viability as [VIABLE/NON_VIABLE] because [reason]"
+      "details": "Determined production viability as [VIABLE/NON_VIABLE/SAMPLE_OR_TEST] because [reason]"
     }
     ```
 
-6.  **Append to Long-Term Memory:** For each finding you loaded (including both
-    `NON_VIABLE` ones and `FALSE POSITIVE`s), append a single structured JSON
-    line to a workspace database file named `learnings.jsonl` (using append
-    mode). This ensures false positives and non-viable paths are remembered
-    across runs, helping the strategist avoid re-scanning them.
+7.  **Append to Long-Term Memory:** For each finding you loaded (including
+    `NON_VIABLE`, `SAMPLE_OR_TEST`, and `FALSE POSITIVE`s), append a single
+    structured JSON line to a workspace database file named `learnings.jsonl`
+    (using append mode). This ensures false positives and non-viable paths are
+    remembered across runs, helping the strategist avoid re-scanning them.
 
     -   **Memory Entry Format:** `{"title": "[finding_title]", "code_paths":
-        ["[path1:line1]"], "status": "[NON_VIABLE / FALSE_POSITIVE / VIABLE]"}`
+        ["[path1:line1]"], "status": "[NON_VIABLE / SAMPLE_OR_TEST /
+        FALSE_POSITIVE / VIABLE]"}`
 
 When complete, notify the user.

mantis_dedupe/SKILL.md

@@ -45,9 +45,10 @@ Execute your task as follows:
     `learnings.jsonl` entries (using `code_paths` and `title` similarities). If
     a current finding exactly matches a flaw that was already processed in this
     loop (regardless of whether its status was `FALSE_POSITIVE`, `NON_VIABLE`,
-    or `VERIFIED_SECURE`), you must **delete the new finding file entirely** and
-    drop it from your active list. This prevents the pipeline from getting stuck
-    re-evaluating the same issues in the current pass.
+    `SAMPLE_OR_TEST`, or `VERIFIED_SECURE`), you must **delete the new finding
+    file entirely** and drop it from your active list. This prevents the
+    pipeline from getting stuck re-evaluating the same issues in the current
+    pass.
 
 3.  **Filter Duplicate Findings in Current Batch:** Check the current findings
     against each other to find duplicates (using `code_paths` and `title`

mantis_meta_agent/SKILL.md

@@ -93,8 +93,8 @@ Execute your orchestration duties in a continuous loop:
         `workspace/findings/` directory (e.g., move it to
         `workspace/archive/findings_pass_N/`) to clear the state for the next
         loop. *Crucially*, you must ensure that before archiving, any finalized
-        findings (especially `FALSE_POSITIVE`, `NON_VIABLE`, or
-        `VERIFIED_SECURE`) were successfully captured by the
+        findings (especially `FALSE_POSITIVE`, `NON_VIABLE`, `SAMPLE_OR_TEST`,
+        or `VERIFIED_SECURE`) were successfully captured by the
         `@mantis_architecture` subagent and written into the permanent Markdown
         Knowledge Base (`workspace/kb/`). If they are only archived but not in
         the KB, the Researcher will just re-find them in the next loop.

mantis_reproduce/SKILL.md

@@ -27,9 +27,9 @@ that reproduces a confirmed security flaw.
 Execute the reproduction stage under these constraints:
 
 1.  **Load Viable Findings:** Read the JSON files in the `workspace/findings/`
-    directory. Filter for findings where `production_viability` is `"VIABLE"`
-    (or skip the filter if you're not checking viability). If no applicable
-    findings exist, notify the user.
+    directory. Filter for findings where `production_viability` is `"VIABLE"` or
+    `"SAMPLE_OR_TEST"` (or skip the filter if you're not checking viability). If
+    no applicable findings exist, notify the user.
 
 2.  **Strict Host Isolation Constraint:**
 

mantis_threat_model/SKILL.md

@@ -55,6 +55,10 @@ Execute the threat modeling process as follows:
 
     -   **System Overview Summary:** A concise summary derived from
         `architecture.md`.
+    -   **Deployment Intent:** Determine if the entire repository is intended
+        for production use, or if it is exclusively a tutorial, sample project,
+        or test suite. State this clearly (e.g., `Intent: SAMPLE_OR_TEST_ONLY`
+        or `Intent: PRODUCTION`).
     -   **Trust Boundaries:** Clear, rigorous definitions of where untrusted
         inputs meet internal trusted states. Reference the specific entities
         (e.g., `[Auth Module](entities/auth_module.md)`).