Skip to content

feat(metrics): expose reasoning tokens in LLM metrics and traces - #2393

Open
rosetta-livekit-bot[bot] wants to merge 1 commit into
mainfrom
loaf-tree-capital
Open

feat(metrics): expose reasoning tokens in LLM metrics and traces#2393
rosetta-livekit-bot[bot] wants to merge 1 commit into
mainfrom
loaf-tree-capital

Conversation

@rosetta-livekit-bot

@rosetta-livekit-bot rosetta-livekit-bot Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Summary

Port livekit/agents#7068 so OpenAI-compatible inference usage carries reasoning tokens through completion usage, LLM metrics, model usage aggregation, metric logs, and the gen_ai.usage.reasoning_tokens trace attribute. Reasoning tokens remain a subset of completion/output tokens and are never added to totals.

Validation

  • pnpm build:agents
  • pnpm test agents --silent (2324 passed, 5 skipped)
  • pnpm --filter @livekit/agents lint (passes with existing warnings)
  • pnpm --filter @livekit/agents typecheck
  • pnpm --filter @livekit/agents throws:check
  • pnpm format:check
  • Ported test subset: 48 passed, 1 skipped

api:check still reports unrelated pre-existing API report drift on the base branch. The API report changes in this PR are intentionally scoped to the new reasoning-token fields and trace constant.

Source diff coverage
Source file Classification Target coverage
livekit-agents/livekit/agents/inference/llm.py Adapted agents/src/inference/llm.ts: map completion_tokens_details.reasoning_tokens to reasoningTokens, defaulting to 0.
livekit-agents/livekit/agents/llm/llm.py Adapted agents/src/llm/llm.ts: expose completion usage, propagate it to LLM metrics, and conditionally set the reasoning-token span attribute.
livekit-agents/livekit/agents/metrics/base.py Adapted agents/src/metrics/base.ts: expose documented reasoningTokens; optional TypeScript input preserves compatibility while emitted metrics normalize it to 0.
livekit-agents/livekit/agents/metrics/usage.py Adapted agents/src/metrics/model_usage.ts: expose and aggregate outputReasoningTokens separately from outputTokens.
livekit-agents/livekit/agents/metrics/utils.py Adapted agents/src/metrics/utils.ts: include camel-case reasoningTokens in LLM metric logs.
livekit-agents/livekit/agents/telemetry/trace_types.py Ported agents/src/telemetry/trace_types.ts: add ATTR_GEN_AI_USAGE_REASONING_TOKENS.
tests/test_inference_llm_usage.py Adapted agents/src/inference/llm.test.ts: port both detailed-reasoning and omitted-details cases using the existing stubbed stream helper.
tests/test_metrics_usage.py Adapted agents/src/llm/llm.test.ts and agents/src/metrics/model_usage.test.ts: port default, carry-through, aggregation, and subset-of-output assertions.
tests/test_trace_types_pii.py Ported agents/src/telemetry/trace_types.test.ts: add the trace attribute to the exhaustive safe-key list.

No source files are not applicable. Target-only generated/supporting changes are agents/etc/agents.api.md and .changeset/expose-reasoning-tokens.md. The shared agent-session protobuf has no reasoning-token usage field in either SDK, so remote-session serialization remains outside the source PR scope.

Source PR: livekit/agents#7068


Ported from livekit/agents#7068

Original PR description

Carries reasoning_tokens from OpenAI-compatible usage payloads through to LLMMetrics, LLMModelUsage, the log_metrics line, and a new gen_ai.usage.reasoning_tokens span attribute. Reasoning is a subset of completion_tokens, never additive.

The inference gateway already returns completion_tokens_details.reasoning_tokens, but inference/llm.py never read it, so the value was dropped before reaching metrics or traces.

@rosetta-livekit-bot
rosetta-livekit-bot Bot requested a review from a team as a code owner August 31, 2026 23:19
@changeset-bot

changeset-bot Bot commented Aug 31, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 426ad82

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 39 packages
Name Type
@livekit/agents Major
@livekit/agents-plugin-anam Major
@livekit/agents-plugin-anthropic Major
@livekit/agents-plugin-assemblyai Major
@livekit/agents-plugin-azure Major
@livekit/agents-plugin-baseten Major
@livekit/agents-plugin-bey Major
@livekit/agents-plugin-cartesia Major
@livekit/agents-plugin-cerebras Major
@livekit/agents-plugin-deepgram Major
@livekit/agents-plugin-did Major
@livekit/agents-plugin-elevenlabs Major
@livekit/agents-plugin-fishaudio Major
@livekit/agents-plugin-google Major
@livekit/agents-plugin-hedra Major
@livekit/agents-plugin-hume Major
@livekit/agents-plugin-inworld Major
@livekit/agents-plugin-krisp Major
@livekit/agents-plugin-lemonslice Major
@livekit/agents-plugin-liveavatar Major
@livekit/agents-plugin-livekit Major
@livekit/agents-plugin-minimax Major
@livekit/agents-plugin-mistral Major
@livekit/agents-plugin-mistralai Major
@livekit/agents-plugin-neuphonic Major
@livekit/agents-plugin-openai Major
@livekit/agents-plugin-perplexity Major
@livekit/agents-plugin-phonic Major
@livekit/agents-plugin-protoface Major
@livekit/agents-plugin-resemble Major
@livekit/agents-plugin-rime Major
@livekit/agents-plugin-runway Major
@livekit/agents-plugin-sarvam Major
@livekit/agents-plugin-silero Major
@livekit/agents-plugin-soniox Major
@livekit/agents-plugin-tavus Major
@livekit/agents-plugins-test Major
@livekit/agents-plugin-trugen Major
@livekit/agents-plugin-xai Major

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

0 participants