feat: simultaneous-MT variant for mlx-llm-mt (AlignAtt commit policy) (depends on #422)

[clkao]

feat: simultaneous-MT variant for mlx-llm-mt (AlignAtt commit policy)

Summary

Add a simultaneous-MT variant that overlaps MT with ASR so a provisional
translation appears during speech, before the ASR finalizes the utterance.
The variant (MlxLlmTranslationSimul) subclasses the generic
MlxLlmTranslation base from the mlx-llm-mt PR; the base is unchanged.

The mechanism: the variant drafts a translation over (committed source +
unstable ASR tail) and commits only the target tokens whose attention
(argmax over the source span) lands on a source token the ASR has
committed. Held target tokens release from cached attention when the ASR
commits more source, without a new MT call.

A hysteresis (a 15-char minimum source delta) prevents a new MT call on
every tail token; the release path re-applies the commit policy on the
cached draft instead.

The attention capture hooks hunyuan_v1_dense.Attention with a manual
softmax(QK^T) so the alignment-head attention is capturable for 8
calibrated zh->en head indices (top head L9/H5). The capture is
bit-identical to the original forward (only attention storage is added).

The --simultaneous flag selects the variant. wants_hypothesis_tail
is set so the audio processor forwards the unstable ASR tail.

User impact

New --simultaneous flag for --translation-backend mlx-llm-mt. When
set, a provisional EN translation appears during speech, before the ASR
finalizes the utterance. The final translation still lands at utterance
close (the quality pass).

The audio_processor change (6 lines) forwards the provisional
translation buffer when no finalized translation is produced. This is a
general fix: a backend that returns a provisional buffer with no
finalized translation now reaches the display (the buffer was dropped
before). The base MlxLlmTranslation is unchanged.

No compatibility risk: the flag defaults to off; the base path is
identical to the mlx-llm-mt PR.

Validation

Hardware: Apple M4. ASR backend: faster-whisper (base) with the
localagreement policy. MT: mlx-community/Hy-MT2-1.8B-8bit (Hunyuan-MT
1.8B 8bit). Audio: 31.6s Mandarin, multi-sentence. Speed: 1.0
(real-time).

The metric that matters: when does the first EN translation reach the
user?

Variant	first EN reaches user	how	MT calls	MT-RTF
base (no --simultaneous)	16.52s	at utterance close (final)	12	0.19x
simul (--simultaneous)	2.44s	during speech (provisional)	24	0.42x

The simul variant's first EN translation reaches the user 14.08s earlier
(2.44s vs 16.52s) because it drafts over the unstable ASR tail and
commits the aligned prefix during speech.

Trade-off: the simul variant makes more MT calls (24 vs 12). It makes
the same 12 final calls as the base, plus 12 provisional calls (one
per sentence, the first draft after each reset). The release path
(re-applying the commit policy on cached attention when the ASR commits
more source) does NOT increment the call counter — verified by unit
tests. The hysteresis prevents calls on every tail token; the count is a
superset of the base, not a reduction.

Heads log confirms the 8 calibrated heads load and (9,5) is the top
head: MlxLlmTranslationSimul: alignment heads=[(9, 5), (13, 1), (9, 6), (12, 11), (14, 2), (14, 0), (4, 12), (1, 10)] top=(9, 5).

Tests: pytest tests/test_mlx_llm_mt.py tests/test_mlx_llm_mt_simul.py -q
-> 55 passed (11 mlx-llm-mt + 32 simul + 12 alignatt).

Checklist

• I searched for an existing issue or discussion and linked it when relevant.
• I added or updated tests for behavior changes.
• I updated documentation for user-facing changes.
• I ran ruff check . (on changed files).
• I ran the relevant pytest suite.
• I did not commit credentials, model weights, generated caches, or private data.

---

ks

[QuentinFuxa]

I approved the CI/CodeQL workflows after checking that this stack does not modify workflow files. This PR is correctly stacked on #422; I will review the incremental simultaneous-MT code once #422's packaging and per-session translation blockers are fixed and this branch is rebased on the corrected base. Keeping the review ordered this way avoids validating the same broken session/dependency contract twice.

[QuentinFuxa]

The approved CI exposed an incremental blocker independent of #422: whisperlivekit/simul_mt_capture.py imports mlx.core at module import time, so the default Linux test environment fails during collection with ModuleNotFoundError: No module named 'mlx'. MLX is an optional, macOS-arm64-only extra; importing the package and collecting tests must remain safe without it. Please make the dependency lazy/guarded and keep MLX-specific execution behind a clear runtime error, with a non-MLX import/collection regression test.

The incremental branch also has two additional Ruff failures: an unused released assignment in tests/test_mlx_llm_mt_simul.py and an unsorted import block in translation_mlx_llm_mt_simul.py (on top of the inherited #422 lint failures).

Please fix these together with #422's base blockers, rebase this branch on the corrected #422 head, and rerun the full default suite on Linux without MLX installed as well as the focused MLX suite on Apple Silicon.

[clkao]

Both blockers are addressed, and the branch is rebased on the corrected #422 head (9209316):

1. Lazy MLX import — simul_mt_capture.py no longer imports mlx.core/mlx.nn at module level. The imports moved inside CapturedAttention.__call__ (the only method that uses them), and CapturedAttention no longer inherits nn.Module. The module imports and pytest --collect-only succeed without MLX installed — added tests/test_simul_mt_capture_no_mlx.py as a collection regression test (35 tests collected with sys.modules['mlx'] = None).

2. Ruff — Removed the unused released assignment (F841) in tests/test_mlx_llm_mt_simul.py; sorted the import block (I001) in translation_mlx_llm_mt_simul.py. ruff check clean on all changed files.

Post-rebase fix: MlxLlmTranslationSimul now overrides new_session() (inherited from #422's per-session isolation) to return a MlxLlmTranslationSimul instance rather than a plain MlxLlmTranslation, so per-session simul clients keep the simultaneous-MT state. Updated the factory test from identity (is) to type (isinstance).

Validation: 56/56 tests pass (35 simul + 21 base), uv lock --check passes, ruff check clean, pytest --collect-only works without MLX.

[clkao]

btw @QuentinFuxa consider enabling stacked PR for the repo :)