Skip to content

fix(dp): wait forward before gather_req - #1506

Open
WANDY666 wants to merge 1 commit into
mainfrom
fix/dp-collective-overlap-ordering
Open

fix(dp): wait forward before gather_req#1506
WANDY666 wants to merge 1 commit into
mainfrom
fix/dp-collective-overlap-ordering

Conversation

@WANDY666

@WANDY666 WANDY666 commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Serialize the next DP collective behind GPU work already queued on the previous overlap stream.
  • Prevent NCCL all-gathers from overlapping outstanding DeepEP/MTP kernels.
  • Preserve request-preparation overlap without adding a host-side synchronization.

Root cause

With overlap enabled, the previous infer thread can release forward before its GPU work finishes. The next infer-loop iteration may then enqueue DP all-gathers on the current stream while the prior DeepEP/MTP kernels are still running. Adding an explicit stream dependency restores the required ordering.

This PR isolates the fix from d5b04bf onto the latest main; the resulting commit is b98f6802.

Testing

  • Focused mocked DP overlap-ordering regression test passed in the H100 LightLLM environment.
  • Changed module passes Python byte-compilation.
  • git diff --check passes.
@WANDY666
fix(dp): serialize DP collectives after overlap forward
b98f680 
The next infer thread may launch DP all-gathers before the previous
overlap-stream forward finishes, causing NCCL to overlap with DeepEP/MTP
kernels. Make the current stream wait before starting DP collectives
while preserving overlap for request preparation.
@WANDY666 WANDY666 changed the title fix(dp): serialize DP collectives after overlap forward Aug 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant

@WANDY666