Skip to content

fix(pd): guard p_node before charging prefill load - #1454

Open
DivyaNarahari97 wants to merge 1 commit into
ModelTC:mainfrom
DivyaNarahari97:fix/pd-master-p-node-guard
Open

fix(pd): guard p_node before charging prefill load#1454
DivyaNarahari97 wants to merge 1 commit into
ModelTC:mainfrom
DivyaNarahari97:fix/pd-master-p-node-guard

Conversation

@DivyaNarahari97

Copy link
Copy Markdown

_generate_one() dereferenced p_node to accumulate dispatched_prompt_chars one line above the "if not p_node or not d_node" check, so that guard could never run -- a None p_node raised AttributeError first, losing the intended error message and the abort path's context.

Moving the increment below the guard alone would introduce a second problem: the finally block decrements whenever p_node is not None and the load was not yet released, which after the move can be reached without the increment ever having happened (e.g. a valid p_node paired with a None d_node), silently skewing the node's in-flight load downward and biasing CacheAware dispatch toward it.

Replace the prefill_load_released flag with prefill_load_charged, which tracks the charge itself rather than an inference about it, so charge and release are symmetric by construction: incremented exactly where the charge happens, cleared exactly where it is refunded, and the finally block refunds if and only if a charge is still outstanding.

_generate_one() dereferenced p_node to accumulate dispatched_prompt_chars
one line *above* the "if not p_node or not d_node" check, so that guard
could never run -- a None p_node raised AttributeError first, losing the
intended error message and the abort path's context.

Moving the increment below the guard alone would introduce a second
problem: the finally block decrements whenever p_node is not None and the
load was not yet released, which after the move can be reached without
the increment ever having happened (e.g. a valid p_node paired with a
None d_node), silently skewing the node's in-flight load downward and
biasing CacheAware dispatch toward it.

Replace the prefill_load_released flag with prefill_load_charged, which
tracks the charge itself rather than an inference about it, so charge and
release are symmetric by construction: incremented exactly where the
charge happens, cleared exactly where it is refunded, and the finally
block refunds if and only if a charge is still outstanding.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant