feat: support MiniMax-H3 Lightning LoRA

[qinxuye]

Summary

• add video-model Lightning metadata, caching, and launch options for the official MiniMax-H3 Turbo LoRAs
• support 4step_v0.1, 8step_v1.0_bf16, and 4step_v1.0_768p_bf16 with their required LoRA alpha, scheduler shifts, and default evaluation counts
• download the Lightning checkpoint from the same selected hub as the base model, including both Hugging Face and ModelScope
• document the shared video Lightning options and expose the versions through the existing launch UI

Implementation notes

MiniMax-H3 Turbo checkpoints use PEFT-format LoRA tensors. The adapter is injected before group offload, while the existing INT4 and group-offload defaults remain unchanged. Since TorchAO INT4 weights cannot be dequantized for LoRA fusion, the adapter forward accumulates into the base output in place to avoid the extra full-size tensor that otherwise exceeds 24 GiB VRAM. Mixed-dtype LoRA operands are cast to the base output dtype before this in-place accumulation.

Per-request num_inference_steps represents actual transformer evaluations; Xinference adds the scheduler terminal sigma grid point internally. This correction applies to both Lightning and non-Lightning MiniMax-H3: a request for N evaluations now passes N + 1 scheduler grid points, so non-Lightning output may differ from earlier Xinference versions for the same num_inference_steps value. If a request omits the value, the selected Lightning version supplies its recommended 4 or 8 evaluations. The Web UI currently pre-fills 25, so users should change Inference Steps to match the selected Lightning version.

Performance

Single-run wall-clock comparison on an NVIDIA GeForce RTX 3090 (24 GiB), using the same MiniMax-H3 defaults (INT4 plus group offload), prompt, and output settings: 1344x768, 124 frames, 24 FPS, one video.

Configuration	Transformer evaluations	Denoising	End to end
Baseline, no Lightning	25	2h 12m 49s	2h 14m 24s (8064s)
4step_v1.0_768p_bf16	4	15m 23s	16m 48s (1008s)
Measured speedup	—	8.6x	8.0x

Both requests completed successfully with HTTP 200. These are single-run results on one machine; Lightning reduces denoising work but does not reduce the model's memory requirements.

Validation

• python -m pytest xinference/model/video/tests/test_minimax_h3_video.py xinference/model/video/tests/test_diffusers_video.py -q (15 passed, 2 skipped)
• pre-commit run --files ...
• Sphinx dummy build succeeded; only existing unrelated reference warnings were reported
• GPU runtime validation passed for both the 25-step baseline and 4step_v1.0_768p_bf16 on the RTX 3090

[gemini-code-assist]

Code Review

This pull request introduces support for Lightning LoRA acceleration for video models, specifically targeting MiniMax-H3. It adds capabilities to download, cache, and load Lightning LoRA checkpoints from Hugging Face or ModelScope, resolves configuration versions, and applies the appropriate scheduler shifts. Feedback suggests validating model support for lightning acceleration before caching to fail fast, and raising an explicit error if a lightning version is specified without a corresponding model path to prevent silent fallback.

[OliverBryant]

Nice work overall — the checkpoint validation here is genuinely thorough (rank consistency, paired A/B tensors, shape checks against the instantiated adapter, and a post-load_state_dict confirmation that the adapter tensors were actually consumed). cache_lightning also mirrors cache_gguf closely, which keeps the hub handling consistent.

Three findings below, one of which I'd like to see addressed before merge (the addmm dtype issue).

Separately, and not a problem with this PR: this PR triggered a full four-group GPU CI run (llm embedding image audio, ~33 min on the T4) even though it only touches video code, docs, and the web UI. xinference/model/video/cache_manager.py falls through to the xinference/model/*) full=true catch-all in .github/workflows/python.yaml, whose comment says video has "no dedicated GPU test group; play it safe and run every group." Since there is no video group in the pytest case at all, that fallback buys zero coverage of the changed code while paying for every other group. I'll open a separate PR adding a xinference/model/video/*) ;; arm (model/flexible/ has the same issue).

[OliverBryant]

Re-reviewed at 5c2b33a7a. I verified both earlier findings are genuinely fixed, not just claimed:

• Mixed dtype — low_rank_states and lora_b.weight are now both cast to result.dtype (with a guard so matching dtypes skip the copy), _check_forward_args is called first, and the no-op result.to(result_dtype) is gone. I re-ran the bf16-base/fp32-LoRA case against the new code path: no exception, max deviation from an fp32 reference 0.0079, which is normal bf16 rounding. The updated test genuinely covers it — bf16 base_layer with fp32 LoRA weights, asserting calls["dtypes"] == (torch.bfloat16,) * 3. That's exactly the case the old fp32-only fixture couldn't catch.
• +1 step semantics — documented in both the PR body and a .. note:: in video.rst, explicitly stating it applies with or without Lightning and that non-Lightning output may differ from earlier versions. Keeping the unified semantics was the right call of the two options I offered.

This pass covered the parts I hadn't yet read closely (config resolution, adapter loading, cache manager, frontend, i18n). Two new findings, one of which is a launch-blocking UI bug.

Nice touches I'll note since they're easy to miss: all four locales (en/zh/ja/ko) have the new keys, cache_lightning correctly threads lightning_model_revision separately from model_revision (unlike cache_gguf, which reuses the base revision), and the _skip_peft_gptqmodel_probes context manager restores the original probes in a finally.

[OliverBryant]

Re-reviewed at e804b0eef. Both findings from the previous round are genuinely fixed — verified against the code, not just the replies:

none placeholder / blank path — transformFormToFetch now drops lightning_version === 'none' and also drops a whitespace-only lightning_model_path via .trim() === ''. That covers both the reported bug and the lightning_model_path follow-up I raised in the same comment, so declining Lightning in the UI now launches the base model cleanly.

Target-module matching — now module_name == target or module_name.endswith(f".{target}"). I re-ran my comparison table against PEFT's check_target_module_exists semantics; the two now agree on every case I tested, including the previously divergent add_to_q, audio_ff.net.2, and add_k_proj, plus the bare to_q exact-match case. The new test_minimax_h3_rejects_partial_lora_target_match asserts add_to_q hits the precise "Unsupported ... target module" diagnostic rather than the misleading incompatible-checkpoint path — that's a real regression test for the exact defect, not just a green assertion. The valid fixture is also corrected to realistic rank-4 shapes (A=(4, 128), B=(128, 4), r: 4).

This pass also covered the pieces I hadn't read before — gen_docs.py, the Jinja template, and the CLI path. Nothing further to flag:

• The gen_docs.py refactor from if 'huggingface' in model_src to model_src.get('huggingface') or model_src.get('modelscope') is strictly more robust: an empty huggingface dict previously raised KeyError on ['model_id'] and now falls back to ModelScope. Same result on all normal inputs.
• The committed minimax-h3.rst matches what the updated template renders, so the generated docs are in sync.
• --lightning_version is documented in several places; confirmed the CLI's ignore_unknown_options=True kwargs passthrough forwards it to create_video_model_instance, so the documented commands work as written.

No further findings from me — the implementation, tests, docs, and UI wiring look consistent at this commit. LGTM.

One CI note, unrelated to the code: because this PR touches xinference/model/video/, it keeps triggering a full four-group GPU run (llm embedding image audio) through the xinference/model/*) full=true catch-all in python.yaml, even though no GPU group contains a video test. I'll send that gating fix as a separate PR so it doesn't hold this one up.

[OliverBryant]

LGTM