Phrase-Localized Language-Contrastive Guidance:
Training-Free Localized Accent Control for Code-Switching Text-to-Speech

Che Hyun Lee¹, Sangkwon Park¹, Donghun Kang¹, Dongwook Lee²,
Youngho Cho², Heeseung Kim³, Sungroh Yoon^1,2,4

¹Department of Electrical and Computer Engineering, Seoul National University
²Interdisciplinary Program in Artificial Intelligence, Seoul National University
³Department of Artificial Intelligence, University of Seoul
⁴AIIS, ASRI, INMC, and ISRC, Seoul National University

EMNLP 2026 · Main Conference · Oral Presentation

📄 arXiv ♪ Listen to Demos ⌨ Code ▤ Benchmark

Code, the 1,200-utterance synthetic code-switching benchmark, and reproduction instructions are available at github.com/saga1214/PhraseLocalizedLCG.

Hear the difference — language-learning content (EN→KO)

Today's key expression is 시간 내 주셔서 정말 감사합니다, which you can use at the end of a meeting.

Vanilla		Listen to the Korean phrase with the English carrier's language conditioning.
Ours — LCG		Compare the same phrase with localized Korean-language guidance.

What is LCG?

The problem. When a sentence embeds a foreign phrase ("This station is 홍대입구"), zero-shot TTS speaks the phrase with the carrier language's accent — because classifier-free guidance applies one language condition to the whole utterance.
The idea. Add a third guidance branch conditioned on the phrase's own language, and apply it only inside the phrase region.
The trick. The phrase region is located by the model itself: we read its self-attention to find which audio frames align to the embedded text — no fine-tuning, no external aligner.

Overview

The challenge

Global guidance flattens the embedded phrase into the carrier accent (left); LCG steers only the phrase region toward its own language (right).

Cross-lingual accent leakage and localized control

The method in three steps

Probe — read decoder self-attention (layers 8 & 12) to find the frames aligned to the embedded phrase.
Refine — union the masks from both language tags, then dilate for recall; the diffusion process absorbs the slack.
Guide — add a phrase-language guidance term, gated by the mask, with its own strength λ.

Method overview: attention probing, mask refinement, phrase-localized LCG

Real-World Scenarios

Everyday situations where a carrier sentence embeds a foreign phrase. Vanilla vs. our phrase-localized LCG (λ=7), paper-final configuration.

The last tab embeds French and German phrases in English carriers — the training-free control covers all five languages of the paper.

Learning, travel & culture (3)
Dining & service (5)
Transit announcements (2)
Workplace & phone (3)
Beyond KO/JA 🌍 (6)

Cafe pickup call EN→KO

In a Korean cafe, you will hear 주문하신 음료 나왔습니다 when your drink is ready.

» 주문하신 음료 나왔습니다 — your drink is ready

Vanilla
Ours λ7

Restaurant service phrase EN→JA

Your server will say ごゆっくりお召し上がりください, so please take your time.

» ごゆっくりお召し上がりください — please take your time and enjoy your meal

Vanilla
Ours λ7

Restaurant menu EN→JA

Tonight's special is えびと野菜の天ぷら盛り合わせ, served with matcha salt.

» えびと野菜の天ぷら盛り合わせ — assorted shrimp and vegetable tempura

Vanilla
Ours λ7

After a meal EN→JA

At the end of a meal, you can say ごちそうさまでした.

» ごちそうさまでした — thank you for the meal

Vanilla
Ours λ7

Shop greeting EN→KO

As you walk in, the staff will greet you with 어서 오세요, 찾으시는 거 있으세요?

» 어서 오세요, 찾으시는 거 있으세요? — welcome — are you looking for anything?

Vanilla
Ours λ7

Language-learning content EN→KO

Today's key expression is 시간 내 주셔서 정말 감사합니다, which you can use at the end of a meeting.

» 시간 내 주셔서 정말 감사합니다 — thank you so much for making the time

Vanilla
Ours λ7

Tour guide EN→KO

Our next stop is 조선 왕조의 정궁인 경복궁, followed by a walk through 북촌 한옥마을의 좁은 골목길.

» 조선 왕조의 정궁인 경복궁 — Gyeongbokgung, the main royal palace of the Joseon dynasty

» 북촌 한옥마을의 좁은 골목길 — the narrow alleys of Bukchon Hanok Village

Vanilla
Ours λ7

Museum exhibition EN→JA

The exhibition features 江戸時代の浮世絵の名作 from the museum's own collection.

» 江戸時代の浮世絵の名作 — masterpieces of Edo-period ukiyo-e woodblock prints

Vanilla
Ours λ7

Dessert course EN→FR

For dessert, we shared un fondant au chocolat coulant, still warm from the oven.

» un fondant au chocolat coulant — a molten chocolate fondant

Vanilla
Ours λ7

Christmas market EN→DE

The Christmas market smelled of frisch geröstete Maronen und Glühwein everywhere.

» frisch geröstete Maronen und Glühwein — freshly roasted chestnuts and mulled wine

Vanilla
Ours λ7

Dessert menu EN→FR

The dessert menu's highlight is une tarte aux pommes maison, served warm.

» une tarte aux pommes maison — a homemade apple tart

Vanilla
Ours λ7

Lunch classic EN→DE

The lunch classic here is Bratkartoffeln mit Speck und Zwiebeln, crispy at the edges.

» Bratkartoffeln mit Speck und Zwiebeln — pan-fried potatoes with bacon and onions

Vanilla
Ours λ7

Cheese course EN→FR

The cheese course featured un plateau de fromages affinés with fig jam.

» un plateau de fromages affinés — a platter of aged cheeses

Vanilla
Ours λ7

Family recipe EN→DE

Grandma's recipe calls for hausgemachter Apfelstrudel mit Vanillesoße on Sundays.

» hausgemachter Apfelstrudel mit Vanillesoße — homemade apple strudel with vanilla sauce

Vanilla
Ours λ7

Benchmark Samples

Samples drawn directly from our released 1,200-utterance synthetic code-switching benchmark, comparing all three systems from the paper (unguided baseline, coupled Swap, and phrase-localized LCG).

Isolated phrase segments

Korean carrier sentences with embedded English phrases, drawn from the LA=1 subset: all systems synthesize the correct words, so the difference you hear is residual accent quality on the isolated segment.

Example 1 — “practical workshops aligned with our buying committee” KO→EN

지난달 우리 팀이 도입한 기업 간 영업 컨설팅사는 기대보다 훨씬 실무적이었고, 리뷰를 남길 만큼 인상이 좋았다. 첫 진단 미팅에서 a clear account-based sales playbook 으로 논의를 정리해 주니, 영업대표와 마케팅 담당자가 같은 언어로 문제를 보게 됐다. 워크숍은 practical workshops aligned with our buying committee 와 실제 고객 미팅 사례를 엮어 진행돼, 강의보다 공동 설계에 가까웠다. 특히 actionable pipeline diagnostics grounded in real CRM data 가 가장 도움이 됐는데, 막연했던 리드 품질 문제와 딜 지연 원인이 숫자로 보였다. 마지막 제안서에 담긴 a lightweight operating model for cross-functional execution 과 transparent ROI assumptions for the first six months 까지 설득력 있어, 비용은 높지만 재계약은 긍정적으로 검토 중이다.

Vanilla
Swap λ3
Ours λ7

Example 2 — “lean household operations with zero emotional overhead” KO→EN

퇴근하고 집에 오자마자 운동복을 세탁기에 넣었는데, 오늘은 minimum viable weekday morning routine 으로 내일 입을 양말이랑 머그컵까지 미리 챙겼다. 싱크대에 컵 두 개랑 접시 하나만 남겨두기 싫어서 lean household operations with zero emotional overhead 처럼 바로 치웠고, 냉장고 앞 메모에는 data driven grocery budget tracking system 의 잔액을 작게 적어놨다. 베란다 화분에 물 주는 사이 고양이는 새 택배 박스에 들어가서 weekly user research session with a very picky stakeholder 에 참여하는 표정으로 나를 봤다. 별거 아닌 집안일도 이렇게 이름을 붙이면 묘하게 게임 같고, 잠들기 전 마음이 조금 정리된다.

Vanilla
Swap λ3
Ours λ7

Full sentences

Complete utterances with three embedded phrases each, comparing global naturalness and phrase nativeness.

English carrier · Japanese phrases EN→JA

After the rain, I ducked into a tiny old-school cafe by the station, and the 深煎り豆をネルで落とす香り made my whole jacket smell like comfort. The owner gave me a smile that said ごゆっくりお召し上がりください, not fussy, just warm. I took the corner seat and watched steam blur the window, feeling a sort of 雨上がりの縁側で飲むほうじ茶 settle into the evening. Next time, I’m bringing a book and pretending I don’t have notifications. Honestly, it beat any fancy latte art.

» 深煎り豆をネルで落とす香り — the aroma of dark-roasted beans brewed through a flannel filter

» ごゆっくりお召し上がりください — please take your time and enjoy it

» 雨上がりの縁側で飲むほうじ茶 — roasted green tea sipped on a veranda after the rain

Vanilla
Swap λ3
Ours λ7

English carrier · Korean phrases EN→KO

Han’s slim novel reads like a review of everything we keep misplacing: names, rooms, the voices of the dead. The 첫눈 녹듯 사라지는 어머니의 마지막 손길 gives the opening its ache, while the 제사상에 올린 식어 가는 흰밥 한 그릇 turns grief into something almost domestic. By the final chapter, even 봄비에 젖은 벚꽃잎이 흩어진 골목 feels less symbolic than inevitable. I admired its restraint; it left me quiet, not devastated, and somehow that was worse.

» 첫눈 녹듯 사라지는 어머니의 마지막 손길 — last trace of a mother’s touch fading like first snow

» 제사상에 올린 식어 가는 흰밥 한 그릇 — cooling bowl of white rice set out for the ancestors

» 봄비에 젖은 벚꽃잎이 흩어진 골목 — an alley strewn with cherry petals wet from spring rain

Vanilla
Swap λ3
Ours λ7

Evaluation Corpus

All audio above is synthesized from our 1,200-utterance balanced synthetic code-switching benchmark, generated with a large language model and spanning five languages across twelve directional configurations. The corpus is released with the code and documented in benchmark/.

1,200

utterances

directions

languages (EN/FR/DE/JA/KO)

~3.6

avg phrases/utt

topical domains

stylistic registers

Each utterance contains 3–5 dense, technical or literary embedded phrases (~6 words, ~35 characters per phrase). The corpus is balanced by direction (100 utterances per direction) to enable per-direction evaluation.

Citation

@misc{lee2026phraselocalizedlanguagecontrastiveguidancetrainingfree,
  title         = {Phrase-Localized Language-Contrastive Guidance: Training-Free Localized Accent Control for Code-Switching Text-to-Speech},
  author        = {Che Hyun Lee and Sangkwon Park and Donghun Kang and Dongwook Lee and Youngho Cho and Heeseung Kim and Sungroh Yoon},
  year          = {2026},
  eprint        = {2609.01016},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2609.01016},
}