Mantis: Improve reproduction decisiveness, quality filtering, and reporting

- Update mantis_reproduce/SKILL.md to instruct the reproducer to dynamically adjust finding details on failure or abandon if unsuccessful.
- Update mantis_meta_agent/SKILL.md to exclude un-reproduced, low-priority, or non-viable findings from confirmed reports.
- Update mantis_review/SKILL.md to add nuance for race conditions, ensuring they are not dismissed if they are automatable/brute-forceable.
- Update mantis_critic/SKILL.md to mark findings blocked by non-configurable environmental controls as non-viable.
- Update mantis_calibrate/SKILL.md to cap the impact of internal admin privilege escalation/lateral movement at 2, and add a "Critical Sanity Triage" step to force-downgrade weak, speculative, or minor hygiene findings to LOW priority.

TAG=agy
CONV=c12a2092-bd2e-4b83-8237-51876a76909a

Change-Id: I9ee9bce1a1b3aa499a9c93e9b0e42c8f9b796336

Author: nickg-ca

mantis_calibrate/SKILL.md

@@ -64,6 +64,13 @@ Execute the calibration as follows:
             the type "the code is fragile", "lack of defense-in-depth", or
             purely theoretical hygiene issues MUST have an Impact score of 1,
             ensuring they are rated LOW at most.
+        -   *Note on Internal Admin & Lateral Movement:* If a finding requires
+            pre-existing administrative privileges (internal privilege
+            escalation, e.g., admin to super-admin) or only allows lateral
+            movement/pivoting between internal components from an already
+            compromised state, cap its individual Impact score at 2. Its
+            escalated risk will be captured separately if it is successfully
+            chained into a "Super Finding" by the chainer.
     -   **Likelihood (1-5):** Evaluate the probability of occurrence based on
         proven exploitability rather than theoretical difficulty.
         -   5: Actively exploited in the wild, OR the agent successfully
@@ -114,7 +121,31 @@ Execute the calibration as follows:
     damage, user sentiment fallout) is taken into account. Do *not* include the
     outrage factor in the final numerical score.
 
-3.  **Determine Priority:**
+3.  **Critical Sanity Triage (Downgrading Weak Findings):** Before determining
+    the final priority, perform a second-level sanity check on the quality of
+    the finding and its accumulated evidence. You **MUST** force-downgrade the
+    finding's priority to **LOW** (and cap its final score at **2.0**) if it
+    meets any of the following "weak finding" criteria:
+
+    -   **Speculative Viability on Repro Failure:** The reproduction failed
+        (`repro_status: "failed_to_reproduce"`), and the reasoning for why it is
+        still viable in production is highly speculative, theoretical, or relies
+        on unverified assumptions about downstream systems.
+    -   **Minor Configuration Hygiene:** The issue represents a minor deviation
+        from best-practice configuration (e.g., slightly loose permissions on an
+        internal directory, lack of modern encryption on low-value internal
+        transport) but does not lead to a clear exploit path, privilege
+        escalation, or data exposure.
+    -   **Vague Code Paths / Fragile Assumptions:** The finding's description or
+        reasoning relies on unverified assumptions about caller behavior or
+        adjacent system components that are not documented in the active
+        Knowledge Base.
+    -   **Unreliable/Noisy Triggers:** The finding represents an issue that can
+        technically be triggered but is highly likely to be ignored in practice
+        due to high noise, or is indistinguishable from normal system operations
+        without causing real harm.
+
+4.  **Determine Priority:**
 
     -   **CRITICAL (8.0 - 10.0):** Immediate action required. Very high hazard
         (e.g. high impact and likelihood). **Must NOT be used unless it
@@ -130,7 +161,7 @@ Execute the calibration as follows:
         exclusively affects a single user's own data MUST be capped at LOW
         priority regardless of the calculated score.**
 
-4.  **Token-Optimized File Updates:** To minimize LLM output tokens, **do not
+5.  **Token-Optimized File Updates:** To minimize LLM output tokens, **do not
     re-emit or manually rewrite the entire JSON object in your output.**
     Instead, write a reusable helper script (e.g.,
     `workspace/helpers/append_calibrate.py`) during your first finding update.

mantis_critic/SKILL.md

@@ -72,6 +72,11 @@ Execute the critic evaluation as follows:
     -   **Debug-Only Features:** Security flaws that exist inside files or
         sections conditionally compiled with debug flags (e.g. `#ifdef DEBUG`)
         are NON_VIABLE.
+    -   **Blocked by Environmental Controls:** If the exploit path is blocked by
+        standard, non-configurable production environmental controls (e.g.,
+        OS-level permissions, kernel-level sandboxing, read-only filesystems, or
+        hardware-enforced write protections) that cannot be bypassed, mark it
+        NON_VIABLE.
     -   **Harnesses, Mocks, & Examples:** Issues residing in example code, test
         suites, fuzzing harnesses, or validation frameworks are technically not
         deployed to production. However, because developers often copy sample

mantis_meta_agent/SKILL.md

@@ -116,6 +116,14 @@ Execute your orchestration duties in a continuous loop:
     subagent), or when `@mantis_calibrate` finishes its scoring, you may output
     a brief text summary to the user.
 
+    -   **Do NOT report findings that failed to reproduce (`repro_status:
+        "failed_to_reproduce"`) as confirmed vulnerabilities.**
+    -   **Do NOT report findings that are marked `LOW` priority or are
+        `NON_VIABLE` as confirmed vulnerabilities.** These are considered
+        low-quality, fragile, or non-actionable. You may list them separately at
+        the bottom of your summary under a "Hygiene & Low Priority Notes"
+        section, but do not present them as active security flaws.
+
 4.  **Human-in-the-Loop Steering & Collaboration:** While you are designed for
     autonomy, remain responsive to user input. The user may interrupt the loop
     to ask for progress updates, collaboratively debug environmental issues, or

mantis_reproduce/SKILL.md

@@ -45,14 +45,20 @@ Execute the reproduction stage under these constraints:
 
 3.  **Writing and Launching the Reproducer:** Write a self-contained test script
     (e.g., `poc.py` or a C reproducer file) or write a raw crash input data
-    payload (e.g., `crash.payload`) that triggers the target bug. To run your
-    script or payload, use the execution or containerization tools available in
-    your environment to execute the code safely. Select the most appropriate
-    runtime image and flags for the target. **Execute your reproduction using
-    the appropriate environment:** If the target is firmware, you may write a
-    script to boot it via `qemu`, `unicorn`, or Firmadyne. If it's a binary, you
-    may use dynamic instrumentation or standard execution. Use your best
-    judgment to construct a working harness for the artifact.
+    payload (e.g., `crash.payload`) that triggers the target bug. Analyze the
+    code path and constraints carefully. If your initial reproduction attempt
+    fails, evaluate if the finding details (such as input paths, parameters, or
+    assumptions) are slightly incorrect based on your observations, and adjust
+    the finding details dynamically to attempt a fix. If you cannot find a
+    triggerable path after trying multiple approaches and adjustments, abandon
+    the attempt and mark it as `failed_to_reproduce`. To run your script or
+    payload, use the execution or containerization tools available in your
+    environment to execute the code safely. Select the most appropriate runtime
+    image and flags for the target. **Execute your reproduction using the
+    appropriate environment:** If the target is firmware, you may write a script
+    to boot it via `qemu`, `unicorn`, or Firmadyne. If it's a binary, you may
+    use dynamic instrumentation or standard execution. Use your best judgment to
+    construct a working harness for the artifact.
 
     -   *Optional Parallel Trajectory Search:* If your environment or agent
         framework supports spawning subagents, you can deploy multiple

mantis_review/SKILL.md

@@ -52,7 +52,14 @@ Execute your validation as follows:
         as security flaws.
     3.  **Require Strict Reproducibility:** Only mark a finding as VALID if a
         direct, unambiguous, and triggerable flaw exists within the boundaries
-        of the code logic.
+        of the code logic. If the finding is extremely fragile (e.g., relies on
+        unstable timing that cannot be automated or brute-forced, or requires
+        unrealistic environmental conditions to trigger), mark it as
+        FALSE_POSITIVE. *Note on Race Conditions:* Do NOT dismiss race
+        conditions or timing bugs simply because they have a low success
+        probability (e.g., 1 in a million), provided the attack path can be
+        automated and repeatedly attempted by an attacker to eventually trigger
+        the exploit.
     4.  **Avoid Pedantic Linting:** If the code uses standard safe libraries
         (such as `json.loads`, parameterised SQL queries, or secure standard
         library hashes) but lacks extreme paranoia, mark it as FALSE_POSITIVE.