[v2] One-shot provider-overflow recovery does not engage when the provider returns an opaque 400 (e.g. bare "invalid request") — session dead-ends with no recovery
Environment
- opencode2
v0.0.0-beta-18743 (Windows)
- Custom openai-compatible provider (Bitdeer
api-inference.bitdeer.ai, model context limit 262,144 / max output 32,768)
compaction: { "auto": false } (required when using a third-party context-management plugin)
Background
Per the V2 compaction docs: V2 "recognizes provider errors classified as context overflow" and performs one-shot compaction + retry "even when auto is false". Our provider rejects oversized requests with a bare 400 {"message":"invalid request"} — no "context length" / "too long" wording — so the error is not classified, and the only remaining safety net never engages.
With auto: false, preflight compaction is off by definition, so the overflow-recovery path is the only protection left — and opaque 400s bypass it entirely.
What happened
- A long session's transcript grew to ~256K tokens (independently measured by a loaded plugin:
tokensBefore: 256320, 408 messages, ~98% of the 262,144 window; plus system prompt + tool schemas → over the limit).
- The provider returned
400 "invalid request" → assistant message recorded with "error": {"type":"provider.invalid-request","message":"invalid request","status":400}, finish: "error".
- No compaction recovery was attempted (no
session.compaction.* events in the server log for that run); the session repeatedly failed with ERROR "Failed to drain Session" cause="AI.Error: invalid request" in a loop.
- The user reverted to several earlier messages to recover; each post-compaction revert point still left a > 250K-token payload (checkpoint summary + tail), so the same 400 repeated. Only reverting to a point before the last compaction checkpoint (dropping it, ~170K-token prefix) recovered the session.
- Session state:
outcome: "failed".
Why this is a V2 gap
- Preflight check: disabled by configuration (legitimate use case — third-party context plugins).
- Overflow recovery: requires the provider error to be classified as context overflow. A bare 400 with an uninformative message cannot be classified, so recovery is skipped.
- Result: a session can cross the window and become unrecoverable by any user action short of reverting far enough to fall under the limit blind (the TUI gives no context-size feedback at the point of failure).
Suggestions (any of these would help)
- Heuristic one-shot recovery: when a 4xx arrives and the estimated request size (V2 already computes this — JSON-serialized request, 4 chars/token) exceeds the model's context/input limit, treat the step as an overflow candidate: compact + retry once (same guard as the classified path).
- Per-provider overflow-error patterns: a config option (e.g.
providers.<id>.overflowErrorPatterns: ["invalid request"], matched against the provider error message) so users can classify opaque errors explicitly.
- Minimum — surface the size: when a model step fails with a 4xx on a request whose estimate exceeds the window, include the numbers in the TUI error, e.g.
request ≈ 256,320 tokens > 262,144 window — compact the session or revert to an earlier point. This alone turns an opaque dead-end into an actionable one.
Repro recipe
Any custom openai-compatible endpoint that answers oversized prompts with a bare 400 + a session that grows past the model window while compaction.auto: false (or before the preflight threshold on a slow-growing session). Observed with Bitdeer inference + Qwen3.8-27B (262K window).
Related: opencode-acp tracks the same failure class on the V1 side (ranxianglei/opencode-acp#346, #347 — silent overflow death on custom providers).
[v2] One-shot provider-overflow recovery does not engage when the provider returns an opaque 400 (e.g. bare "invalid request") — session dead-ends with no recovery
Environment
v0.0.0-beta-18743(Windows)api-inference.bitdeer.ai, model context limit 262,144 / max output 32,768)compaction: { "auto": false }(required when using a third-party context-management plugin)Background
Per the V2 compaction docs: V2 "recognizes provider errors classified as context overflow" and performs one-shot compaction + retry "even when
autoisfalse". Our provider rejects oversized requests with a bare400 {"message":"invalid request"}— no "context length" / "too long" wording — so the error is not classified, and the only remaining safety net never engages.With
auto: false, preflight compaction is off by definition, so the overflow-recovery path is the only protection left — and opaque 400s bypass it entirely.What happened
tokensBefore: 256320, 408 messages, ~98% of the 262,144 window; plus system prompt + tool schemas → over the limit).400 "invalid request"→ assistant message recorded with"error": {"type":"provider.invalid-request","message":"invalid request","status":400},finish: "error".session.compaction.*events in the server log for that run); the session repeatedly failed withERROR "Failed to drain Session" cause="AI.Error: invalid request"in a loop.outcome: "failed".Why this is a V2 gap
Suggestions (any of these would help)
providers.<id>.overflowErrorPatterns: ["invalid request"], matched against the provider error message) so users can classify opaque errors explicitly.request ≈ 256,320 tokens > 262,144 window — compact the session or revert to an earlier point. This alone turns an opaque dead-end into an actionable one.Repro recipe
Any custom openai-compatible endpoint that answers oversized prompts with a bare 400 + a session that grows past the model window while
compaction.auto: false(or before the preflight threshold on a slow-growing session). Observed with Bitdeer inference + Qwen3.8-27B (262K window).Related: opencode-acp tracks the same failure class on the V1 side (ranxianglei/opencode-acp#346, #347 — silent overflow death on custom providers).