Avoid unnecessary determinization in index pattern conflict checks#128362
Merged
jimczi merged 7 commits intoelastic:mainfrom Jun 2, 2025
Merged
Avoid unnecessary determinization in index pattern conflict checks#128362jimczi merged 7 commits intoelastic:mainfrom
jimczi merged 7 commits intoelastic:mainfrom
Conversation
Starting with Lucene 10, `CharacterRunAutomaton` is no longer determinized automatically. In Elasticsearch 9, we adapted to this by eagerly determinizing automatons early (via `Regex#simpleMatchToAutomaton`). However, this introduced regression: operations like index template conflict checks, which only require intersection testing, now pay the cost of determinization—an expensive step that wasn’t needed before. In some cases, especially when many wildcard patterns are involved, determinization can even fail due to state explosion. This change removes the unnecessary determinization, restoring the pre-9.0 behavior and allowing valid index templates with many patterns to be registered again.
jimczi
added
>bug
:Data Management/Indices APIs
Collaborator
|
Pinging @elastic/es-data-management (Team:Data Management) |
elasticsearchmachine
added
the
Team:Data Management (obsolete)
Collaborator
|
Hi @jimczi, I've created a changelog YAML for you. |
javanna
reviewed
May 23, 2025
pawankartik-elastic
approved these changes
May 27, 2025
Contributor
pawankartik-elastic
left a comment
pawankartik-elastic
left a comment
There was a problem hiding this comment.
LGTM overall. Leaving 2 tiny comments.
Member
|
This should be backported to 9.0.x right? Its existed ever since the lucene 10 upgrade. |
jimczi
added a commit
to jimczi/elasticsearch
that referenced
this pull request
Jun 2, 2025
…lastic#128362) Starting with Lucene 10, `CharacterRunAutomaton` is no longer determinized automatically. In Elasticsearch 9, we adapted to this by eagerly determinizing automatons early (via `Regex#simpleMatchToAutomaton`). However, this introduced regression: operations like index template conflict checks, which only require intersection testing, now pay the cost of determinization—an expensive step that wasn’t needed before. In some cases, especially when many wildcard patterns are involved, determinization can even fail due to state explosion. This change removes the unnecessary determinization, restoring the pre-9.0 behavior and allowing valid index templates with many patterns to be registered again.
elasticsearchmachine
pushed a commit
that referenced
this pull request
Jun 2, 2025
…128362) (#128759) Starting with Lucene 10, `CharacterRunAutomaton` is no longer determinized automatically. In Elasticsearch 9, we adapted to this by eagerly determinizing automatons early (via `Regex#simpleMatchToAutomaton`). However, this introduced regression: operations like index template conflict checks, which only require intersection testing, now pay the cost of determinization—an expensive step that wasn’t needed before. In some cases, especially when many wildcard patterns are involved, determinization can even fail due to state explosion. This change removes the unnecessary determinization, restoring the pre-9.0 behavior and allowing valid index templates with many patterns to be registered again.
mridula-s109
pushed a commit
that referenced
this pull request
Jun 2, 2025
…128362) Starting with Lucene 10, `CharacterRunAutomaton` is no longer determinized automatically. In Elasticsearch 9, we adapted to this by eagerly determinizing automatons early (via `Regex#simpleMatchToAutomaton`). However, this introduced regression: operations like index template conflict checks, which only require intersection testing, now pay the cost of determinization—an expensive step that wasn’t needed before. In some cases, especially when many wildcard patterns are involved, determinization can even fail due to state explosion. This change removes the unnecessary determinization, restoring the pre-9.0 behavior and allowing valid index templates with many patterns to be registered again.
mridula-s109
pushed a commit
to mridula-s109/elasticsearch
that referenced
this pull request
Jun 3, 2025
…lastic#128362) Starting with Lucene 10, `CharacterRunAutomaton` is no longer determinized automatically. In Elasticsearch 9, we adapted to this by eagerly determinizing automatons early (via `Regex#simpleMatchToAutomaton`). However, this introduced regression: operations like index template conflict checks, which only require intersection testing, now pay the cost of determinization—an expensive step that wasn’t needed before. In some cases, especially when many wildcard patterns are involved, determinization can even fail due to state explosion. This change removes the unnecessary determinization, restoring the pre-9.0 behavior and allowing valid index templates with many patterns to be registered again.
joshua-adams-1
pushed a commit
to joshua-adams-1/elasticsearch
that referenced
this pull request
Jun 3, 2025
…lastic#128362) Starting with Lucene 10, `CharacterRunAutomaton` is no longer determinized automatically. In Elasticsearch 9, we adapted to this by eagerly determinizing automatons early (via `Regex#simpleMatchToAutomaton`). However, this introduced regression: operations like index template conflict checks, which only require intersection testing, now pay the cost of determinization—an expensive step that wasn’t needed before. In some cases, especially when many wildcard patterns are involved, determinization can even fail due to state explosion. This change removes the unnecessary determinization, restoring the pre-9.0 behavior and allowing valid index templates with many patterns to be registered again.
Samiul-TheSoccerFan
pushed a commit
to Samiul-TheSoccerFan/elasticsearch
that referenced
this pull request
Jun 5, 2025
…lastic#128362) Starting with Lucene 10, `CharacterRunAutomaton` is no longer determinized automatically. In Elasticsearch 9, we adapted to this by eagerly determinizing automatons early (via `Regex#simpleMatchToAutomaton`). However, this introduced regression: operations like index template conflict checks, which only require intersection testing, now pay the cost of determinization—an expensive step that wasn’t needed before. In some cases, especially when many wildcard patterns are involved, determinization can even fail due to state explosion. This change removes the unnecessary determinization, restoring the pre-9.0 behavior and allowing valid index templates with many patterns to be registered again.
cbuescher
added a commit
to cbuescher/elasticsearch
that referenced
this pull request
Sep 5, 2025
We already fixed an issue with this in elastic#128362 but apparently another instance of the unnecessary determinization was hiding elsewhere and in its current state throws exceptions starting with Lucene 10 on complext patterns. This change adds the same fix as elastic#128362 and adds a test that would have triggered this. Closes elastic#133652
cbuescher
added a commit
that referenced
this pull request
Sep 8, 2025
We already fixed an issue with this in #128362 but apparently another instance of the unnecessary determinization was hiding elsewhere and in its current state throws exceptions starting with Lucene 10 on complext patterns. This change adds the same fix as #128362 and adds a test that would have triggered this. Closes #133652
cbuescher
added a commit
to cbuescher/elasticsearch
that referenced
this pull request
Sep 8, 2025
We already fixed an issue with this in elastic#128362 but apparently another instance of the unnecessary determinization was hiding elsewhere and in its current state throws exceptions starting with Lucene 10 on complext patterns. This change adds the same fix as elastic#128362 and adds a test that would have triggered this. Closes elastic#133652
cbuescher
added a commit
to cbuescher/elasticsearch
that referenced
this pull request
Sep 8, 2025
We already fixed an issue with this in elastic#128362 but apparently another instance of the unnecessary determinization was hiding elsewhere and in its current state throws exceptions starting with Lucene 10 on complext patterns. This change adds the same fix as elastic#128362 and adds a test that would have triggered this. Closes elastic#133652
elasticsearchmachine
pushed a commit
that referenced
this pull request
Sep 8, 2025
We already fixed an issue with this in #128362 but apparently another instance of the unnecessary determinization was hiding elsewhere and in its current state throws exceptions starting with Lucene 10 on complext patterns. This change adds the same fix as #128362 and adds a test that would have triggered this. Closes #133652
elasticsearchmachine
pushed a commit
that referenced
this pull request
Sep 8, 2025
) * Fix exceptions in index pattern conflict checks (#134231) We already fixed an issue with this in #128362 but apparently another instance of the unnecessary determinization was hiding elsewhere and in its current state throws exceptions starting with Lucene 10 on complext patterns. This change adds the same fix as #128362 and adds a test that would have triggered this. Closes #133652 * Fix test compilation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Starting with Lucene 10,
CharacterRunAutomatonis no longer determinized automatically.In Elasticsearch 9, we adapted to this by eagerly determinizing automatons early (via
Regex#simpleMatchToAutomaton). However, this introduced regression: operations like index template conflict checks, which only require intersection testing, now pay the cost of determinization, an expensive step that wasn’t needed before. In some cases, especially when many wildcard patterns are involved, determinization can even fail due to state explosion.This change removes the unnecessary determinization for index patterns conflict check, restoring the pre-9.0 behavior and allowing valid index templates with many patterns to be registered again.
closes: #127972