Windows/ROCm: Qwen3.6-35B-A3B NVFP4 serving on gfx1101 (RDNA3)

[rba]

Get nvidia/Qwen3.6-35B-A3B-NVFP4 running end to end on Windows 11 + ROCm on an AMD RX 7800 XT (gfx1101 / RDNA3), with MoE offload and MoE prefill overlap.

Environment

• GPU: AMD Radeon RX 7800 XT, 16 GiB, gfx1101
• OS: Windows 11
• Python: 3.12.10
• PyTorch: 2.13.0+rocm10.1.0a20260822
• HIP: 7.16.26332
• Triton / triton-windows: 3.7.1
• Checkpoint: nvidia/Qwen3.6-35B-A3B-NVFP4 (MoE offload)

What changes

• MoE NVFP4 prefill (the blocker): On RDNA3 the grouped NVFP4 prefill Triton kernel fails to launch (HIP error 719) and stalls the warm-up. fused_experts_nvfp4() now routes to fused_experts_decode_nvfp4_marlin on gfx11, which has equivalent semantics and bounded memory use.
• FP8 linear: e4m3_native() now returns False on ROCm HIP (the reported capability is not an NVIDIA SM level), and a new _scaled_mm_supported() restricts torch._scaled_mm to gfx94/gfx95 on ROCm (else sm_89+), so RDNA takes the dequant/emulation path.
• GDN causal conv: pass host-known max_seq_len / batch into causal_conv1d_varlen() to avoid a host sync on the prefill path.
• Model loading: download_hf_weight() also fetches model.safetensors.index.json (sharded checkpoints), and drop_page_cache() is a no-op where os.posix_fadvise is unavailable (Windows).
• dist scripts: harden the Windows install/run/stop scripts (venv pip, a required vcvarsall.bat check with a clear error, full ROCm/HIP environment, robust log cleanup, and process-tree shutdown).

Run

. .\dist\install.ps1

.\dist\run-server.ps1 -Arch gfx1101 -Model "nvidia/Qwen3.6-35B-A3B-NVFP4" -KVPages 4096 `
  -ExtraArgs @('--moe-backend','offload','--moe-cache-auto','--expert-load','serial','--cuda-graph-max-bs','0')

-RocmPath is auto-detected; pass it explicitly if auto-detection cannot find the ROCm install.

Validation

• pytest tests/models/test_models_loader.py tests/kernels/test_fp8_pertensor_linear.py tests/kernels/test_e4m3_compat.py → 36 passed, 16 skipped
• pytest tests/moe/test_nvfp4_backends.py tests/moe/test_safe_offload_copy.py → 17 passed, 6 skipped
• Server reaches ready; /v1/models returns the model and /v1/chat/completions streams a stable reply.
• Prefill warm-up: ~22.4 s with MoE overlap (the path that previously hung), ~43.8 s with overlap disabled.

Notes

• --disable-moe-prefill-overlap is no longer required.
• No NVIDIA regression coverage here: the FP8/NVFP4 gating only widens the fallback path on unsupported GPUs, and the MoE change is scoped to gfx11.