[PERF] Ascend NPU: MindIE-SD attention backend and VAE-parallel enablement

[yx0716]

Description

Ascend NPU acceleration on top of #42, in two parts:

1. An optional MindIE-SD attention backend (AttnImplType.MINDIE_ATTN) that routes dense attention through mindiesd.attention_forward (auto-tuned Ascend kernels including LaserAttention), with measured end-to-end gains of 1.10–1.18× over the [FEATURE] Run Wan2.2 pipelines on Ascend NPU #42 baseline on compute-dominated configurations.
2. Two fixes that make spatially parallel VAE decode usable: repair the enable_vae_parallel input contract for the Wan2.2 48-channel VAE (currently broken on every platform), and isolate HCCL socket ranges per worker group so concurrent groups can coexist on NPU hosts.

Default behavior is unchanged on every platform: the backend is strictly opt-in, and the VAE/HCCL fixes are inert outside enable_vae_parallel / multi-group NPU runs.

Motivation

• [FEATURE] Run Wan2.2 pipelines on Ascend NPU #42 established the Ascend baseline and showed denoising is the dominant, well-scaling cost; its attention runs on torch_npu SDPA. mindiesd (open-sourced MindIE-SD) provides faster Ascend attention kernels behind a stable API — measured 1.29–1.53× over SDPA at wan-typical shapes.
• Wan22VideoVAE.decode's parallel branch calls torch.stack on its input, assuming a list, but VAEStage.decode_video passes a batched tensor; any enable_vae_parallel run of a Wan2.2-VAE pipeline fails with TypeError today, on CUDA as well as NPU.
• Concurrent worker groups on NPU collide on HCCL's default data-plane socket range (EJ0003); a distinct per-group HCCL_IF_BASE_PORT (mirroring the existing per-group MASTER_PORT) removes the collision.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Performance improvement
• Code refactoring
• Documentation update
• Other (please describe):

Changes Made

• telefuser/core/config.py, telefuser/ops/attention/backends.py, telefuser/ops/attention/attention_impl.py, docs/en/attention.md: add AttnImplType.MINDIE_ATTN with a sageattention-style availability probe (optional mindiesd import, NPU platform only), a BNSD dispatch branch, and documentation. This intentionally adds one public enum value; no dependency is added to pyproject.toml (optional import, like sageattention).
• tests/unit/ops/test_attention_backends.py: mock-based dispatch unit test for the new backend (runs on CPU CI).
• telefuser/models/wan22_video_vae.py: normalize decode's input before the parallel-branch torch.stack — batched tensors pass through, lists are stacked as before.
• telefuser/worker/parallel_worker.py: allocate a distinct HCCL_IF_BASE_PORT per spawned worker group on NPU platforms; other platforms untouched.

Testing

• Unit tests pass (pytest tests/)
• Manual testing performed
• Benchmarks added/updated (if applicable)

Test commands:

pytest tests/unit -q
# backend selected via the existing config surface:
#   pipe_config.dit_config.attention_config = AttentionConfig.dense_attention(AttnImplType.MINDIE_ATTN)
python examples/wan_video/wan22_t2v_5b.py --gpu_num 1 --resolution 720p
python examples/wan_video/wan22_t2v_5b.py --gpu_num 4 --resolution 720p

• tests/unit/ops/test_attention_backends.py 11 passed (includes the new dispatch test; mock-based, platform-independent); full tests/unit green.
• Numerical parity: MINDIE_ATTN output is bit-identical to TORCH_SDPA at probed BNSD bf16 shapes on Atlas 910B.
• Multi-process path verified: the enum travels through the pickled stage config, so 4-card denoising workers select the backend correctly.
• Single-group 4-card regression with the per-group HCCL base port applied: timing at parity with the [FEATURE] Run Wan2.2 pipelines on Ascend NPU #42 baseline.
• VAE-parallel on our particular Ascend host still stops inside torch_npu's cross-process device layer after these fixes (driver/firmware mismatch, unrelated to this change — see Additional Notes); the input fix itself is platform-independent.

Performance Impact

Measured against the #42 baseline (same host, same unmodified example, same commands; Atlas 910B 8× 910B2, CANN 8.2, torch 2.9.0 + torch_npu 2.9.0.post1, 50 denoising steps, MODEL_CPU_OFFLOAD):

Workload	Cards	#42 baseline (TORCH_SDPA)	This PR (MINDIE_ATTN)	Speedup
480p, 121 frames	1	180.9 s	162.6 s	1.11×
480p, 121 frames	4 (cfg=2 × ulysses=2)	87.0 s	103.0 s	0.84× — regression, see guidance
720p, 121 frames	1	477.8 s	405.0 s	1.18×
720p, 121 frames	4 (cfg=2 × ulysses=2)	173.9 s	157.6 s	1.10×

Kernel microbench (BNSD bf16, head_dim 128): mindiesd attention is 1.29–1.53× faster than torch_npu SDPA across S = 12.5k–45.8k, at both 24 and 12 heads (the ulysses head-sharded shape).

Lossless stack (same task, same quality; configuration only): MINDIE_ATTN plus resident weights (NO_CPU_OFFLOAD instead of the example's MODEL_CPU_OFFLOAD) measures 407.6 s on 1 card (1.17×) and 151.9 s on 4 cards (1.14×) at 720p/121 frames. The weight-residency gain is visible at 4 cards (−5.7 s) and within run noise at 1 card.

For reference only (not part of this PR's performance claims): combining the above with the existing in-tree RIFE VFI stage (generate 61 frames, interpolate to 121) measures 193.1 s / 94.4 s — but that changes the generation itself (the model's temporal prior operates at half the frame rate and interpolation reconstructs the rest), so it is a quality trade-off that needs content-level motion validation rather than a lossless optimization.

Guidance: the backend is opt-in and recommended for compute-dominated configurations (single-card, and multi-card at high resolution / long schedules). The 480p 4-card regression persists even though the kernel itself is faster at exactly those shapes, which points at an interaction with the ulysses async-communication overlap when per-step compute is small; TASK_QUEUE_ENABLE=2 does not recover it. Default TORCH_SDPA behavior is unchanged, and the VAE/HCCL fixes have no effect on standard single-group runs (verified at parity).

Related Issues

Complements #42 (Ascend NPU enablement); all baseline numbers above are from #42's Performance Impact section. Independently mergeable.

Additional Notes

• Why no new unit test for the VAE fix: constructing a Wan22VideoVAE requires full model weights and the parallel branch needs an initialized process group; the normalization is a two-line input guard exercised by the example path.
• On our Ascend host (driver 25.2.0 vs CANN 8.2, with repeated "driver and firmware packages do not match" warnings) full dual-worker-group execution still fails inside torch_npu's cross-process device layer after the HCCL port isolation; we validated up to communicator setup and expect driver-matched installs (and CUDA) to run the full path. Expected impact once VAE decode parallelizes, from [FEATURE] Run Wan2.2 pipelines on Ascend NPU #42's measured serial sections: ≈2.08× → ≈2.6× (480p) and ≈2.75× → ≈3.3× (720p) on 4 cards.
• mindiesd is available from PyPI (pip install mindiesd); when absent or off-NPU, the probe leaves the backend unavailable and nothing changes.

GPU Architecture Support

• SM80 (Ampere, Ada Lovelace)
• SM90 (Hopper H100)
• SM100+ (Blackwell)

No CUDA kernels are added or modified. The new backend calls prebuilt mindiesd kernels on Ascend NPU only; CUDA behavior is unchanged.

Checklist

• Code follows the project's coding standards (ruff)
• Pre-commit hooks pass (pre-commit run --all-files — ruff + ruff-format; also covered by the lint CI job)
• All tests pass (pytest tests/)
• New tests added for new functionality (test_attention_backends.py dispatch test)
• Documentation updated (docs/en/attention.md, docstrings)
• Commit messages are clear and descriptive
• PR title follows the convention: [TYPE] Brief description