refactor(hotel_receptionist): split scenarios into scoped and open-ended files

[rosetta-livekit-bot]

Summary

Ports livekit/agents#7065 by splitting the hotel receptionist scenario suite into:

• scenarios_tool_accuracy.yaml: 57 deterministic tool-flow scenarios with userdata.expected_state
• scenarios_guardrails_and_faq.yaml: 43 open-ended, adversarial, policy, and FAQ scenarios graded on agent_expectations only

Scenario bodies retain the existing TypeScript-port adaptations. The parsed union is content-identical to the original 100-scenario target suite, and scenario membership/order exactly matches the authoritative source PR.

Source diff coverage

• examples/hotel_receptionist/scenarios_guardrails_and_faq.yaml: adapted to examples/src/hotel_receptionist/scenarios_guardrails_and_faq.yaml. Ported the new 43-scenario open-ended/adversarial suite and source header/name while retaining existing target-specific JS scenario wording and tags.
• examples/hotel_receptionist/scenarios_tool_accuracy.yaml: adapted to examples/src/hotel_receptionist/scenarios_tool_accuracy.yaml. Renamed and split the target scenarios.yaml counterpart into the 57 deterministic expected_state scenarios, preserving existing target-specific JS scenario wording and tags.
• Not applicable: none. Both source files have target counterparts and are included. The source PR adds or modifies no tests or infrastructure.

Validation

• YAML semantic validation: 57/43 counts, exact source membership/order, deterministic-state partition, and content-identical union
• pnpm build --filter=livekit-agents-examples...
• pnpm --filter livekit-agents-examples build
• pnpm --filter livekit-agents-examples lint (passes with two pre-existing TSDoc warnings)
• pnpm --filter livekit-agents-examples test (96 passed, 2 skipped; exits nonzero because survey_agent.test.ts emits two asynchronous pre-existing FakeLLM "No input text found" unhandled rejections after its tests complete)

No changeset is included because only the private, changeset-ignored examples package is touched and no package code changed.

---

Ported from livekit/agents#7065

Original PR description

What

Splits examples/hotel_receptionist/scenarios.yaml (100 scenarios) into two files, as agreed in the 2026-08-27 eng sync:

• scenarios_tool_flows.yaml (57) — scoped tool-based scenarios. Every scenario drives a concrete tool flow to a deterministic end state and is graded on the final DB diff (userdata.expected_state) in addition to agent_expectations. These are the ones representative of real customer flows and usable to evaluate simulation quality.
• scenarios_guardrails_and_faq.yaml (43) — open-ended/adversarial scenarios: disputes, complaints, policy/info Q&A, guardrail probes (prompt extraction, privacy pressure, harmful requests), and judgment calls. Graded on agent_expectations only; useful for guardrail testing, not sim-quality evaluation.

Split criterion

Presence of a deterministic userdata.expected_state — the objective encoding of "scoped tool-based": the scenario's outcome is asserted as DB state written through the agent's tools. Everything NL-judge-only goes to the guardrails file. If any individual scenario should live on the other side, moving it is a pure cut/paste.

Guarantees

The split was done textually (comments and formatting preserved) and validated by script:

• both files parse; the union == the original 100 scenarios, content-identical (compared as parsed YAML)
• every tool-flow scenario has expected_state; no guardrails/faq scenario does

Only the file headers and the two suite name fields are new. Nothing in the repo references scenarios.yaml by path, so no code changes are needed.

[changeset-bot]

⚠️ No Changeset found

Latest commit: a164cae

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[devin-ai-integration]

Devin Review found 1 potential issue.

[devin-ai-integration]

🟡 Scenario split breaks license validation

After the split, both scenario files fall outside REUSE.toml metadata coverage. The mandatory REUSE check rejects them and blocks every PR build.

Prompt for agents

Update the license metadata for both examples/src/hotel_receptionist/scenarios_guardrails_and_faq.yaml and examples/src/hotel_receptionist/scenarios_tool_accuracy.yaml. Either add valid SPDX headers to both YAML files or replace the obsolete scenarios.yaml entry in REUSE.toml with paths covering both split files, then verify the REUSE-3.2 check passes.

---

Was this helpful? React with 👍 or 👎 to provide feedback.