Always treat dash-prefixed expression as index exclusion by ywangd · Pull Request #138467 · elastic/elasticsearch

ywangd · 2025-11-24T07:01:40Z

Fixed a bug that dash prefix expressions, including both concrete names and wildcard patterns, are not always treated as index exclusion during index resolution process. The PR also updates relevant document to make the behaviour clear.

Resolves: #64752
Resolves: #83435

elasticsearchmachine · 2025-11-24T07:02:06Z

Hi @ywangd, I've created a changelog YAML for you.

github-actions · 2025-11-24T07:04:18Z

🔍 Preview links for changed docs

docs/reference/elasticsearch/rest-apis/api-conventions.md

github-actions · 2025-11-24T07:04:19Z

ℹ️ Important: Docs version tagging

👋 Thanks for updating the docs! Just a friendly reminder that our docs are now cumulative. This means all 9.x versions are documented on the same page and published off of the main branch, instead of creating separate pages for each minor version.

We use applies_to tags to mark version-specific features and changes.

Expand for a quick overview

When to use applies_to tags:

✅ At the page level to indicate which products/deployments the content applies to (mandatory)
✅ When features change state (e.g. preview, ga) in a specific version
✅ When availability differs across deployments and environments

What NOT to do:

❌ Don't remove or replace information that applies to an older version
❌ Don't add new information that applies to a specific version without an applies_to tag
❌ Don't forget that applies_to tags can be used at the page, section, and inline level

🤔 Need help?

Check out the cumulative docs guidelines
Reach out in the #docs Slack channel

elasticsearchmachine · 2025-11-24T07:06:41Z

Hi @ywangd, I've updated the changelog YAML for you.

elastic#138464)

This commit adds the bytes and break if we get over the limit of the breaker when building global ordinals.

ywangd · 2025-12-01T01:09:32Z

server/src/main/java/org/elasticsearch/action/ResolvedIndexExpressions.java

+                final var iter = expressions.iterator();
+                while (iter.hasNext()) {
+                    final ResolvedIndexExpression current = iter.next();
+                    if (expressionsToExclude.contains(current.original())) {
+                        iter.remove();
+                        continue;
+                    }
+                    final Set<String> localExpressions = current.localExpressions().indices();
                    if (localExpressions.isEmpty()) {
                        continue;
                    }


This change ensures that an expression like foo,bar,-foo work correctly by excluding foo entirely from the ResolvedExpressions. It is so far a CPS only feature and strictly speaking not necessary for stateful code yet. But we would have to mute or skip tests without it. So I decided to include it given the its relative small size.

elasticsearchmachine · 2025-12-01T01:10:46Z

Pinging @elastic/es-security (Team:Security)

ywangd · 2025-12-01T01:17:07Z

Both indexNameExpressionResolver and IndexAbstractionResolver have the problematic exclusion handling. The former can still take effect when security is disabled. The PR fixes the issues in both places.

idegtiarenko

ES|QL will take advantage of this changes once merged as we indirectly use IndexNameExpressionResolver.

Thanks, looking forward for this change to be merged!

DaveCTurner

A user who is snapshotting very-important-index,other-indices*,-*ant* today will, after this change, silently stop having snapshots of very-important-index. I understand that we might like to change this behaviour, even perhaps considering it a bug, but still it's a potentially-data-loss-causing breaking change that at least needs some discussion with the BCC before proceeding.

The original design in which specifically-named indices (no wildcards) are not excluded was a deliberate choice.

ywangd · 2025-12-01T08:18:35Z

@DaveCTurner

needs some discussion with the BCC before proceeding

Indeed. There is an approval proposal for this change. I tagged you in Slack for the relevant message. Thanks for checking!

DaveCTurner

Sorry, I was misreading the effects of this change, it doesn't have the impact I initially thought. I'm ok with it now, sorry for the noise.

server/src/main/java/org/elasticsearch/cluster/metadata/IndexNameExpressionResolver.java

Doesn't have the impact I thought it had.

leemthompo · 2025-12-01T09:20:07Z

docs/changelog/138467.yaml

@@ -0,0 +1,7 @@
+pr: 138467
+summary: "Fixed a bug that dash prefix expressions, including both concrete names\


language nit: I'd rephrase and add examples:

Fixed a bug where dash-prefixed expressions, for both specific index names and wildcard patterns (for example: -system-index, -logs-*), were not consistently excluded during index resolution.

Thanks for the suggestion. Updated in 1f1d757

leemthompo

Just couple language issues. I think the note is OK, but I'll let @DaveCTurner weigh in on whether it's clear enough to his eyes

docs/reference/elasticsearch/rest-apis/api-conventions.md

docs/changelog/138467.yaml

DaveCTurner · 2025-12-02T08:29:25Z

docs/reference/elasticsearch/rest-apis/api-conventions.md

+{applies_to}`stack: ga 9.3` A dash-prefixed (`-`) expression is always treated as an exclusion.
+
+::::{note}
+In previous versions, such expression is sometimes _not_ treated as an exclusion due to a bug.


Can we use removed 9.3 here?

{applies_to}`stack: removed 9.3`

I think that'd be nicer than a ::::{note} which highlights this whole paragraph as if it's important.

I think removed is aimed more at features that have been deprecated and eventually removed 🤔

Pinging the experts @florent-leborgne @shainaraskas @colleenmcginnis who will have more ideas about the appropriate pattern to use here.

nicer than a ::::{note} which highlights this whole paragraph as if it's important.

Yes, I think we could just remove the note too and that might suffice here, as the new behavior now has the applies_to tag

I think that just pulling this out of a note would work. I think until we have ranges (eta 9.3), that's the best choice

Thanks. I removed the note annotation as suggested fb4b24a

Co-authored-by: Liam Thompson <leemthompo@gmail.com>

leemthompo

Thanks for iterating on the docs!

elasticsearchmachine · 2025-12-08T03:50:05Z

Hi @ywangd, I've updated the changelog YAML for you.

ywangd added 3 commits November 24, 2025 16:56

No need for prior wildcard to recognize index exclusion.

1587e31

fix IndexNameExpressionResolver and update tests

5f61356

update doc

7a23bd1

ywangd added >bug :Security/Security Security issues without another label v9.3.0 labels Nov 24, 2025

Update docs/changelog/138467.yaml

ef23f6a

ywangd and others added 2 commits November 24, 2025 18:06

update changelog

96dca41

Update docs/changelog/138467.yaml

2d9de15

ywangd and others added 5 commits November 24, 2025 18:30

update

fcb0b13

Always prefer YES over NOT_PREFERRED when allocating unassigned shards (

0e6ec7e

elastic#138464)

Break on FieldData when building global ordinals (elastic#108875)

a9350d6

This commit adds the bytes and break if we get over the limit of the breaker when building global ordinals.

Merge remote-tracking branch 'origin/main' into index-exclusion-bug-fix

bb3c900

Merge remote-tracking branch 'origin/main' into index-exclusion-bug-fix

abb935e

ywangd commented Dec 1, 2025

View reviewed changes

ywangd marked this pull request as ready for review December 1, 2025 01:10

elasticsearchmachine added the Team:Security Meta label for security team label Dec 1, 2025

ywangd requested review from piergm, quux00 and tvernum December 1, 2025 01:10

idegtiarenko approved these changes Dec 1, 2025

View reviewed changes

DaveCTurner previously requested changes Dec 1, 2025

View reviewed changes

DaveCTurner reviewed Dec 1, 2025

View reviewed changes

server/src/main/java/org/elasticsearch/cluster/metadata/IndexNameExpressionResolver.java Outdated Show resolved Hide resolved

leemthompo reviewed Dec 1, 2025

View reviewed changes

ywangd added 4 commits December 2, 2025 14:57

Merge remote-tracking branch 'origin/main' into index-exclusion-bug-fix

dc32905

fix comment

5a3aaba

update changelog

1f1d757

update docs

79a9f71

ywangd requested a review from leemthompo December 2, 2025 04:25

tweak

cda53c4

leemthompo reviewed Dec 2, 2025

View reviewed changes

docs/reference/elasticsearch/rest-apis/api-conventions.md Outdated Show resolved Hide resolved

docs/changelog/138467.yaml Outdated Show resolved Hide resolved

DaveCTurner reviewed Dec 2, 2025

View reviewed changes

ywangd and others added 3 commits December 4, 2025 17:51

Apply suggestions from code review

5494067

Co-authored-by: Liam Thompson <leemthompo@gmail.com>

remove note annotation

fb4b24a

Merge remote-tracking branch 'origin/main' into index-exclusion-bug-fix

b3cc5b8

leemthompo approved these changes Dec 4, 2025

View reviewed changes

tvernum approved these changes Dec 8, 2025

View reviewed changes

Merge branch 'main' into index-exclusion-bug-fix

bdcad40

ywangd added the auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) label Dec 8, 2025

Update docs/changelog/138467.yaml

e514ada

add issues back to changelog

c393f16

elasticsearchmachine merged commit 92750be into elastic:main Dec 8, 2025
40 checks passed

ywangd deleted the index-exclusion-bug-fix branch December 8, 2025 05:09

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Always treat dash-prefixed expression as index exclusion#138467

Always treat dash-prefixed expression as index exclusion#138467
elasticsearchmachine merged 22 commits intoelastic:mainfrom
ywangd:index-exclusion-bug-fix

ywangd commented Nov 24, 2025 •

edited

Loading

elasticsearchmachine commented Nov 24, 2025

github-actions bot commented Nov 24, 2025 •

edited

Loading

github-actions bot commented Nov 24, 2025

When to use applies_to tags:

What NOT to do:

elasticsearchmachine commented Nov 24, 2025

ywangd Dec 1, 2025

elasticsearchmachine commented Dec 1, 2025

ywangd commented Dec 1, 2025

idegtiarenko left a comment

DaveCTurner left a comment •

edited

Loading

ywangd commented Dec 1, 2025

DaveCTurner left a comment

Uh oh!

leemthompo Dec 1, 2025 •

edited

Loading

ywangd Dec 2, 2025

leemthompo left a comment

Uh oh!

Uh oh!

DaveCTurner Dec 2, 2025

leemthompo Dec 2, 2025

shainaraskas Dec 2, 2025

ywangd Dec 4, 2025

leemthompo left a comment

elasticsearchmachine commented Dec 8, 2025

Uh oh!

Labels

9 participants

		@@ -0,0 +1,7 @@
		pr: 138467
		summary: "Fixed a bug that dash prefix expressions, including both concrete names\

Conversation

ywangd commented Nov 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

elasticsearchmachine commented Nov 24, 2025

github-actions bot commented Nov 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔍 Preview links for changed docs

github-actions bot commented Nov 24, 2025

ℹ️ Important: Docs version tagging

When to use applies_to tags:

What NOT to do:

🤔 Need help?

elasticsearchmachine commented Nov 24, 2025

ywangd Dec 1, 2025

Choose a reason for hiding this comment

elasticsearchmachine commented Dec 1, 2025

ywangd commented Dec 1, 2025

idegtiarenko left a comment

Choose a reason for hiding this comment

DaveCTurner left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

ywangd commented Dec 1, 2025

DaveCTurner left a comment

Choose a reason for hiding this comment

Uh oh!

leemthompo Dec 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

ywangd Dec 2, 2025

Choose a reason for hiding this comment

leemthompo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

DaveCTurner Dec 2, 2025

Choose a reason for hiding this comment

leemthompo Dec 2, 2025

Choose a reason for hiding this comment

shainaraskas Dec 2, 2025

Choose a reason for hiding this comment

ywangd Dec 4, 2025

Choose a reason for hiding this comment

leemthompo left a comment

Choose a reason for hiding this comment

elasticsearchmachine commented Dec 8, 2025

Uh oh!

Labels

9 participants

ywangd commented Nov 24, 2025 •

edited

Loading

github-actions bot commented Nov 24, 2025 •

edited

Loading

DaveCTurner left a comment •

edited

Loading

leemthompo Dec 1, 2025 •

edited

Loading