Skip to content

fix(agents): keep downstream-cancelled stream channels from leaking unhandled rejections - #2362

Open
anzemur wants to merge 1 commit into
livekit:mainfrom
anzemur:fix/stream-channel-downstream-cancel
Open

fix(agents): keep downstream-cancelled stream channels from leaking unhandled rejections#2362
anzemur wants to merge 1 commit into
livekit:mainfrom
anzemur:fix/stream-channel-downstream-cancel

Conversation

@anzemur

@anzemur anzemur commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

What

Since 1.7.0, forwardAudio cancels its reader on abort (reader.cancel() with no reason). Cancelling the readable side of a stream channel errors the writable side with undefined, so every later fire-and-forget write() / close() from a realtime producer rejects with undefined and surfaces as an unhandled promise rejection (Error: undefined, no application frames).

In production (google realtime plugin, livekit-agents worker on node 22) this fires on every barge-in: the plugin keeps writing the frames that arrive after the interrupt into the cancelled audio channel (realtime_api.ts gen.audioChannel.write(audioFrame) / markCurrentGenerationDoneaudioChannel.close()). We saw ~640 events across 54 users in 24h after upgrading.

Fix

At the createStreamChannel seam, so every producer is covered:

  • Watch writer.closed so the channel learns about downstream cancellation / errors, flips isClosed, and debug-logs the transition (also fixes the stale closed getter).
  • write() attaches a handler to each write promise (no more unhandled rejections from fire-and-forget producers) while returning the original promise, so awaiting callers keep exact backpressure and rejection semantics.
  • close() swallows "already closed or errored" failures instead of only TypeError instances — an undefined cancel reason slipped past the e instanceof Error check.
  • Logging is best-effort via a new tryLog() export next to log() - it returns the logger or undefined before initializeLogger() has run (log() throws there, and a throw inside these handlers would recreate the unhandled rejection).

Complementary one-liner: forwardAudio now cancels with new Error('audio forwarding aborted') so anything that still observes the reason sees a real error instead of undefined. Note this line alone does NOT fix the bug - it only relabels the rejections; the createStreamChannel hardening is what removes them (verified below).

Tests

  • New: a cancelled-mid-stream consumer no longer produces unhandled rejections and flips closed; an awaited write() after cancel still rejects with the cancel reason.
  • Full agents suite passes (2283 tests).

Live verification

Ran our production voice agent (google realtime plugin, real LiveKit + Gemini session) with this fix applied to the installed 1.7.0 dist:

  • Full smoke suite passes patched (opening, barge-in, text filter, seeded history, typed text, tool injection, closing TTS) with warning sets identical to the unpatched control - no functional regression.
  • The live race is hard to script: interrupt() closes the audio channel via markCurrentGenerationDone() whenever the generation already has output, so a scripted interrupt lands after that window. A deterministic replay of the plugin's exact post-cancel calls (gen.audioChannel.write(audioFrame) and audioChannel.close() after forwardAudio's reader.cancel()) shows 8 unhandled rejections unpatched (including the undefined-reason ones we see in Sentry) vs 0 patched, on both the ESM and CJS builds.
  • The new debug line (stream channel writable errored or was cancelled downstream) doubles as a detector for the bug condition in the wild.
@changeset-bot

changeset-bot Bot commented Aug 28, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 32f2784

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 39 packages
Name Type
@livekit/agents Patch
@livekit/agents-plugin-anam Patch
@livekit/agents-plugin-anthropic Patch
@livekit/agents-plugin-assemblyai Patch
@livekit/agents-plugin-azure Patch
@livekit/agents-plugin-baseten Patch
@livekit/agents-plugin-bey Patch
@livekit/agents-plugin-cartesia Patch
@livekit/agents-plugin-cerebras Patch
@livekit/agents-plugin-deepgram Patch
@livekit/agents-plugin-did Patch
@livekit/agents-plugin-elevenlabs Patch
@livekit/agents-plugin-fishaudio Patch
@livekit/agents-plugin-google Patch
@livekit/agents-plugin-hedra Patch
@livekit/agents-plugin-hume Patch
@livekit/agents-plugin-inworld Patch
@livekit/agents-plugin-krisp Patch
@livekit/agents-plugin-lemonslice Patch
@livekit/agents-plugin-liveavatar Patch
@livekit/agents-plugin-livekit Patch
@livekit/agents-plugin-minimax Patch
@livekit/agents-plugin-mistral Patch
@livekit/agents-plugin-mistralai Patch
@livekit/agents-plugin-neuphonic Patch
@livekit/agents-plugin-openai Patch
@livekit/agents-plugin-perplexity Patch
@livekit/agents-plugin-phonic Patch
@livekit/agents-plugin-protoface Patch
@livekit/agents-plugin-resemble Patch
@livekit/agents-plugin-rime Patch
@livekit/agents-plugin-runway Patch
@livekit/agents-plugin-sarvam Patch
@livekit/agents-plugin-silero Patch
@livekit/agents-plugin-soniox Patch
@livekit/agents-plugin-tavus Patch
@livekit/agents-plugins-test Patch
@livekit/agents-plugin-trugen Patch
@livekit/agents-plugin-xai Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@anzemur
anzemur force-pushed the fix/stream-channel-downstream-cancel branch from ca6e5e7 to 32f2784 Compare August 28, 2026 11:56
@anzemur
anzemur marked this pull request as ready for review August 28, 2026 11:57
@anzemur
anzemur requested a review from a team as a code owner August 28, 2026 11:57

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant