[FLINK-40167][table] Add EARLY_FIRE join hint surface and option validation by weiqingy · Pull Request #28353 · apache/flink

weiqingy · 2026-06-07T23:12:39Z

This is the first PR of the FLIP-497 implementation, split into a stack of small, independently reviewable PRs under the umbrella issue FLINK-36953. Landing order:

Step	Sub-task	Scope
PR-1a (this PR)	FLINK-40167	EARLY_FIRE hint surface + option validation
PR-1b	FLINK-40168	Thread the hint into the interval join (planner → ExecNode)
PR-2	FLINK-40169	`target` option
PR-3	FLINK-40170	Update-producing changelog mode + insert-only guard
PR-4	FLINK-40171	Runtime early-fire emit + retraction (rowtime + proctime)
PR-5	FLINK-40172	Processing-time early fire on an event-time interval join
PR-6	FLINK-40173	JSON serde + restore coverage
PR-7	FLINK-40174	User-facing documentation

What is the purpose of the change

Introduces the EARLY_FIRE join hint for SQL interval joins: its typed options and key-value validation. The hint is recognized and validated here, but no rule consumes it yet, so it has no planning or runtime effect beyond accepting the new syntax. Threading it into the interval join follows in PR-1b. This keeps the foundational surface layer small and isolated.

Brief change log

Add EarlyFireJoinHintOptions (@PublicEvolving): delay (required), time-mode (optional), and the nested TimeMode enum.
Register EARLY_FIRE in JoinStrategy and FlinkHintStrategies with a key-value option checker (delay must be a positive duration; time-mode must be rowtime/proctime).
Wire the query-hint propagation touchpoints (CapitalizeQueryHintsShuttle, QueryHintsResolver).
Exclude EARLY_FIRE from the generic join-hint test coverage and from config-docs generation.

Verifying this change

This change added tests and can be verified as follows:

Added IntervalJoinTest validation cases (required delay, positive delay, time-mode enum, unknown option). Generic join-hint tests stay green with EARLY_FIRE excluded from their coverage.

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changed class annotated with @Public(Evolving): yes, a new @PublicEvolving EarlyFireJoinHintOptions
The serializers: no
The runtime per-record code paths (performance sensitive): no
Anything that affects deployment or recovery: no
The S3 file system connector: no

Documentation

Does this pull request introduce a new feature? yes (the first step of FLIP-497)
If yes, how is the feature documented? The user-facing documentation lands in the docs PR at the end of this stack (PR-7).

Was generative AI tooling used to co-author this PR?

Yes (please specify the tool below)

Generated-by: Claude Code (Anthropic)

flinkbot · 2026-06-07T23:16:08Z

CI report:

229dcb5 Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

@flinkbot run azure re-run the last Azure build

weiqingy · 2026-06-07T23:57:23Z

Hi @wuchong @xuyangzhong @xccui Could you please help review this PR? Thanks!

MartijnVisser · 2026-06-08T07:28:22Z

@weiqingy This FLIP doesn't have the necessary nr of approved binding votes, so it's not OK to implement this

RocMarshal · 2026-06-08T11:31:11Z

Thanks @weiqingy @MartijnVisser
I'll take a look for the FLIP wiki ASAP.

BWT, @weiqingy

Would you mind helping bumping the discuss email thread and voting email thread?
Also, could we break this pull request down into several smaller patches? It will help reviewers go through the code more efficiently.

Pls let know what's your opinion.

weiqingy · 2026-06-09T06:56:37Z

Hi @MartijnVisser @RocMarshal, thanks for reviewing the PR! I’ve CCed you both on the Discussion, Voting, and Voting Result email thread. I believe we have 3 binding votes, but please let me know if I missed anything.

weiqingy · 2026-06-09T06:59:08Z

could we break this pull request down into several smaller patches? It will help reviewers go through the code more efficiently.

@RocMarshal Sounds good. I’ll follow up and look into how to break this PR down into smaller patches.

MartijnVisser · 2026-06-09T09:30:34Z

I believe we have 3 binding votes, but please let me know if I missed anything.

I can only see 2 binding votes, I've replied in the thread as well

raminqaf

As for now, I recommend to switch to a draft PR. After, the FLIP voting settles we can re-open it.

weiqingy · 2026-06-10T02:57:25Z

Hi @MartijnVisser, @raminqaf, and @RocMarshal,

Thanks to all of you for the reviews, suggestions, and clarification.

@MartijnVisser: Thanks for clarifying the bylaws. To share my thought process on the tally: when I was gathering the results, Xuyang had become an active committer. I assumed his initial "+1 (non-binding)" vote transitioned to a binding vote along with his new status, which would have brought the total to 3 [1]. I appreciate you pointing this out. I agree with your suggestion to restart the process. Since the thread has been open for over a year, having a new round of discussion is the best way to ensure the design is fully aligned with the current state of the project. I will kick off a new [DISCUSS] thread soon. I also replied in the voting email thread to ensure everyone is on the same page [2].

@raminqaf: I will go ahead and convert this PR to a draft while the new FLIP discussion and voting settle, as you recommended.

@RocMarshal: While the FLIP is being re-discussed, I will use the time to look into how to best break this PR down into smaller, more reviewable patches as you suggested.

Thanks again for the guidance, everyone!

[1] https://lists.apache.org/thread/dcn78blz7cqmmyypw897w0dvf6l58cnt
[2] https://lists.apache.org/thread/mw0tqgg1yx0mn8tf6msrgdtqbpdpgtpt

…dation Register the EARLY_FIRE join hint and its typed options (EarlyFireJoinHintOptions: delay, time_mode) with a key-value option checker, and wire the query-hint propagation touchpoints. Exclude EARLY_FIRE from the generic join-hint test coverage and config-docs generation, since it is a key-value hint validated on its own. The hint is recognized and validated here but not yet consumed by any rule; threading it into the interval join follows in a separate change.

weiqingy · 2026-07-19T00:15:36Z

Thanks everyone for the FLIP-497 discussion and vote 🙏

This kicks off the implementation as a stack of 8 small, independently reviewable PRs (roadmap + sub-tasks FLINK-40167–40174 in the description). This first one, PR-1a, only adds the EARLY_FIRE hint surface and option validation — it's inert (no rule consumes the hint yet); threading follows in PR-1b.

@xuyangzhong @xccui @RocMarshal — when you have a moment, I'd appreciate your review here. Feedback on the overall split/granularity is welcome too.

…time-mode Flink option keys use hyphens (e.g. output-mode, fixed-delay); rename the EARLY_FIRE hint's time_mode option key to time-mode to match, and update the affected tests.

…idation tests The runtime uses delay.toMillis(), so a sub-millisecond delay truncates to zero. Make the contract explicit: require at least 1 millisecond in the DELAY description and the checker error message. Add coverage for a sub-millisecond delay, list-style options (only key-value is supported), and case-insensitive hint-name capitalization that preserves the key-value options.

…test class Move the EARLY_FIRE hint surface and option-validation tests out of the Scala IntervalJoinTest into a dedicated Java EarlyFireJoinHintTest (under plan/hints/stream, mirroring StateTtlHintTest), and relocate their plan golden accordingly. IntervalJoinTest keeps only its original tests.

Wrap the assertThatThrownBy chain in testEarlyFireListOptionsRejected to satisfy Spotless.

RocMarshal

LGTM +1

Thanks @weiqingy for the hard work.

Pls let us wait for a few days for more comments.

@snuyanzin Could you help take a look if you had the free time? Thank you.

RocMarshal · 2026-07-25T01:11:34Z

Given the lack of further responses over the past three business days, we will proceed with merging this PR to keep things moving. If there are additional suggestions, please feel free to submit a hotfix or follow-up PR.

Thanks!

raminqaf reviewed Jun 9, 2026

View reviewed changes

github-actions Bot added the community-reviewed PR has been reviewed by the community. label Jun 9, 2026

weiqingy marked this pull request as draft June 10, 2026 02:58

weiqingy force-pushed the FLINK-36953-impl branch from fd2d3ac to 3131f4e Compare July 18, 2026 21:41

weiqingy changed the title ~~[FLINK-36953][table] Early fire support for Flink SQL interval join~~ Jul 18, 2026

weiqingy force-pushed the FLINK-36953-impl branch from 3131f4e to 4215571 Compare July 18, 2026 22:12

weiqingy changed the title ~~[FLINK-36953][table] Add EARLY_FIRE join hint surface and planner plumbing~~ Jul 18, 2026

weiqingy changed the title ~~[FLINK-36953][table] Add EARLY_FIRE join hint surface and option validation~~ Jul 18, 2026

weiqingy marked this pull request as ready for review July 18, 2026 23:46

RocMarshal reviewed Jul 20, 2026

View reviewed changes

Comment thread ...table-api-java/src/main/java/org/apache/flink/table/api/config/EarlyFireJoinHintOptions.java Outdated

weiqingy added 2 commits July 20, 2026 18:46

[FLINK-40167][table] Address review: rename time_mode hint option to …

d64c58a

…time-mode Flink option keys use hyphens (e.g. output-mode, fixed-delay); rename the EARLY_FIRE hint's time_mode option key to time-mode to match, and update the affected tests.

RocMarshal reviewed Jul 21, 2026

View reviewed changes

Comment thread ...er/src/test/scala/org/apache/flink/table/planner/plan/stream/sql/join/IntervalJoinTest.scala Outdated

weiqingy added 2 commits July 21, 2026 11:07

[FLINK-40167][table] Fix Spotless line wrapping in EarlyFireJoinHintTest

229dcb5

Wrap the assertThatThrownBy chain in testEarlyFireListOptionsRejected to satisfy Spotless.

RocMarshal approved these changes Jul 22, 2026

View reviewed changes

RocMarshal self-assigned this Jul 22, 2026

weiqingy mentioned this pull request Jul 22, 2026

[FLINK-40168][table] Thread the EARLY_FIRE hint into the interval join #28796

Merged

1 task

RocMarshal merged commit 1ea8cb0 into apache:master Jul 25, 2026

This was referenced Jul 27, 2026

[FLINK-40169][table] Add target option to the EARLY_FIRE hint #28827

Merged

[FLINK-40170][table-planner] Infer update-producing changelog mode for early-fire interval join #28877

Merged

This was referenced Aug 10, 2026

[FLINK-40171][table-runtime] Emit and retract early-fire results in the interval join operator #28952

Merged

[FLINK-40172][table-runtime] Support processing-time early fire on a row-time interval join #28953

Open

weiqingy mentioned this pull request Aug 30, 2026

[FLINK-40173][table-planner] Add restore coverage for early-fire interval join #29045

Draft

1 task

Navigation Menu

Uh oh!

[FLINK-40167][table] Add EARLY_FIRE join hint surface and option validation - #28353

[FLINK-40167][table] Add EARLY_FIRE join hint surface and option validation#28353
RocMarshal merged 5 commits into
apache:masterfrom
weiqingy:FLINK-36953-impl

weiqingy commented Jun 7, 2026 •

edited

Loading

flinkbot commented Jun 7, 2026 •

edited

Loading

weiqingy commented Jun 7, 2026 •

edited

Loading

MartijnVisser commented Jun 8, 2026

RocMarshal commented Jun 8, 2026

weiqingy commented Jun 9, 2026

weiqingy commented Jun 9, 2026

MartijnVisser commented Jun 9, 2026

raminqaf left a comment

weiqingy commented Jun 10, 2026

weiqingy commented Jul 19, 2026

Uh oh!

Uh oh!

RocMarshal left a comment

RocMarshal commented Jul 25, 2026

Labels

5 participants

Uh oh!

Conversation

weiqingy commented Jun 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What is the purpose of the change

Brief change log

Verifying this change

Does this pull request potentially affect one of the following parts:

Documentation

Was generative AI tooling used to co-author this PR?

flinkbot commented Jun 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CI report:

weiqingy commented Jun 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

MartijnVisser commented Jun 8, 2026

RocMarshal commented Jun 8, 2026

weiqingy commented Jun 9, 2026

weiqingy commented Jun 9, 2026

MartijnVisser commented Jun 9, 2026

raminqaf left a comment

Choose a reason for hiding this comment

weiqingy commented Jun 10, 2026

weiqingy commented Jul 19, 2026

Uh oh!

Uh oh!

RocMarshal left a comment

Choose a reason for hiding this comment

RocMarshal commented Jul 25, 2026

Labels

5 participants

weiqingy commented Jun 7, 2026 •

edited

Loading

flinkbot commented Jun 7, 2026 •

edited

Loading

weiqingy commented Jun 7, 2026 •

edited

Loading