Skip to content

fix(agent): never leak raw reasoning as a reply; transport-aware prompt - #286

Merged
plombeer31 merged 2 commits into
mainfrom
fix/issue-285-bug-reasoning-channel-leaks
Aug 31, 2026
Merged

fix(agent): never leak raw reasoning as a reply; transport-aware prompt#286
plombeer31 merged 2 commits into
mainfrom
fix/issue-285-bug-reasoning-channel-leaks

Conversation

@plombeer31

@plombeer31 plombeer31 commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

What

Two-part fix for the reasoning-channel leak and the native_tools prompt contradiction reported in #285.

1. Guardrail — stop salvaging raw CoT as a reply (src/agent/step-executor.ts)

tryParseToolCalls's native_tools branch handled a reasoning-only completion (empty content, no tool_calls, non-empty reasoning_content) by wrapping the raw reasoning text into a reply { text } call — delivering the model's internal chain-of-thought verbatim as deliberate agent speech, exactly as the reporter's transcript shows.

Now:

  • a GBNF-shaped [{tool, args}, ...] batch embedded in the think channel is still recovered as real tool calls (that path recovers genuine intent and is kept);
  • anything else returns { ok: false }, routing the completion through the existing one-shot repair path (parse_retry event + buildToolCallRepairPrompt + REPAIR_MAX_TOKENS cap) that already handles every other unparseable body;
  • a repair that fails too ends the step as a GrammarError — a normal parse error, never a CoT leak.

2. Root cause — transport-aware stable prefix (src/prompt/stable-prefix.ts + threading)

Under llm.toolTransport: "native_tools" the request carries an OpenAI tools array with tool_choice: "auto", while the prompt simultaneously ordered "Emit a JSON ARRAY of tool calls now" and the persona mandated "exactly one JSON array". The reporter measured 0/6 completions using native tool_calls under this dual mandate.

buildStablePrefix now takes toolTransport (threaded StepDependenciesBuildPromptInputbuildStablePrefix): under native_tools the persona's emission mandate and the ### instructions block switch to native function-calling guidance ("call tools through the function-calling interface; reply / plain text for the final answer").

The ### tools text catalog is deliberately kept in both modes: the provider fallback chain can hand a native-shaped request to a grammar-only llama-server link (see the buildLlmStreamParams comment about keeping grammar populated), and the catalog carries the tier / tool.view semantics. The JSON-array emission mandate was the contradiction, not the catalog.

Byte-stability

The grammar-path prefix is byte-identical to origin/main's output — verified by sha256-comparing buildStablePrefix from both revisions across variants (plain, reasoningSystemToken, Gemma-4 turnSystemOpen, maxParallelToolCalls, win32, custom systemPersona): all identical (fde2b2b9…). KV-cache reuse is unaffected for local models; within a session the transport is fixed, so the native prefix is byte-stable across turns too (pinned by a test).

Known caveat

On a native-to-grammar mid-session fallover the prompt lacks the JSON-array mandate while the grammar link parses GBNF. Acceptable: llama-server's GBNF grammar constrains decoding to the array shape regardless of the prompt mandate.

Test evidence

  • Rewrote the pinned salvage test: native_tools: unparseable reasoning-only completion routes through parse_retry, never leaks CoT as a reply — asserts the repair round-trip and that no reply anywhere in the transcript carries the reasoning body.
  • New: native_tools: recovers a GBNF-shaped batch embedded in reasoning_content without a retry (salvage of real tool calls is preserved).
  • New: native_tools: reasoning-only on both attempts surfaces a parse error, not the CoT (rejects with GrammarError).
  • New buildPrompt tool transport (issue #285) block: native prefix omits both text-JSON mandates but keeps ### tools; grammar prefix byte-identical whether the field is omitted or explicit; native prefix byte-stable across turns; explicit systemPersona override wins on both transports.
  • Without the fix, the two leak tests and the native-prefix test fail (verified by stashing the src changes); with it, all pass.
  • npm run lint clean. Full suite: 610/611 files, 6436 passed / 1 failed — the one failure (src/sidecar/send-message-concurrency.test.ts) fails identically on pristine origin/main @ 4caff55 and is unrelated to this diff.

Fixes #285

Валерий Брижатюк and others added 2 commits August 31, 2026 15:35
Two-part fix for issue #285 (verbatim chain-of-thought delivered to the
user as the agent's reply, and native_tools having no effect on model
behaviour).

Guardrail (step-executor): the native_tools reasoning-only salvage no
longer wraps an unparseable `reasoning_content` body into a
`reply { text }` call. A GBNF-shaped batch embedded in the think channel
is still recovered as real tool calls, but anything else now returns a
parse failure and routes through the existing one-shot repair
(parse_retry + buildToolCallRepairPrompt, REPAIR_MAX_TOKENS-capped). A
repair that fails too ends the step as a GrammarError — a parse error,
not a CoT leak.

Root cause (prompt): the stable prefix was transport-blind — under
native_tools the request carried an OpenAI `tools` array with
`tool_choice: "auto"` while the prompt simultaneously ordered "Emit a
JSON ARRAY of tool calls now" and the persona mandated "exactly one JSON
array". `buildStablePrefix` now takes `toolTransport` (threaded from
`StepDependencies` through `BuildPromptInput`): under native_tools the
persona's emission mandate and the `### instructions` block switch to
native function-calling guidance. The `### tools` text catalog stays in
both modes — the provider fallback chain can hand a native-shaped
request to a grammar-only llama-server link, and the catalog carries the
tier / `tool.view` semantics. The grammar-path prefix is byte-identical
to before (KV-cache safe; verified by sha256 against the previous
implementation across profile/turn-framing/win32/persona variants).

Known caveat: on a native-to-grammar mid-session fallover the prompt
lacks the JSON-array mandate while the grammar link parses GBNF. This is
acceptable because llama-server's GBNF grammar constrains decoding to
the array shape regardless of the prompt mandate.

Fixes #285

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…eplay drift

Adversarial review of the issue #285 fix found three residual problems;
this commit closes all three.

1. `### rules` still opened with the text-array mandate ("One tool-call
   array per step ... a solo action is a length-1 array") in BOTH
   transports, contradicting the native `### instructions` block in the
   same prefix. The rules line is now transport-variant like the persona
   and instructions: the grammar line is byte-identical to before
   (sha256-verified against v0.4.2 across omitted/explicit, win32,
   persona-override, reasoning-token and turn-framing variants), the
   native line mandates the function-calling interface. The same sweep
   caught one more survivor in the shared persona — "if another tool is
   next, emit that tool JSON" — which likewise becomes "call that tool"
   under native_tools only.

2. `buildToolCallRepairPrompt` was transport-blind: on the exact
   parse_retry path the #285 fix routes reasoning-only completions
   through, it appended "Emit a corrected JSON array only" / "Use a
   length-1 array" onto a native prefix that forbids text-JSON — the
   dual mandate recreated at the one retry a failing model gets. The
   repair mandate now follows `deps.toolTransport`: native repairs order
   a corrected native tool call, grammar repairs keep the legacy lines
   byte-for-byte. A new probe test captures the second llmComplete
   prompt under native transport and asserts no text-array mandate
   survives anywhere in it.

3. `atomic-agent trace replay` regressed into 100% false drift for
   native-transport sessions: the prefix now differs by construction per
   transport, but `replaySession` always rebuilt the grammar variant and
   traces do not record which transport served the session. The replay
   is now transport-aware: `ReplayContext.toolTransport` pins the
   comparison when the caller knows the transport; when omitted (the
   trace-command case) both variants are built and a recorded hash is
   clean when it matches either — old (pre-#285) traces keep matching
   through the byte-identical grammar variant. Each step reports which
   variant matched (`matchedTransport`).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@plombeer31
plombeer31 merged commit 39415c1 into main Aug 31, 2026
2 checks passed
plombeer31 added a commit that referenced this pull request Aug 31, 2026
…026-08-31

integrate: Discord-sourced fixes (#286#298)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant