ESQL: Fix sneaky bug in single value query#127146
Merged
nik9000 merged 5 commits intoelastic:mainfrom Apr 22, 2025
Merged
Conversation
Fixes a sneaky bug in single value query that happens when run against
a `keyword` field that:
* Is defined on every field
* Contains the same number of distinct values as documents
The simplest way to reproduce this is to build a single shard index
with two documents:
```
{"a": "foo"}
{"a": ["foo", "bar"]}
```
I don't think this is particularly likely in production, but it's quite
likely in tests. Which is where I hit this - in the serverless tests we
index an index with four documents into three shards and two of the
documents look just like this. So about 1/3 or the time we triggered
this bug.
Mechanically this is triggered by the `SingleValueMatchQuery`
incorrectly rewriting itself to `MatchAll` in the scenario above. This
fixes that.
elasticsearchmachine
added
the
Team:Analytics
Collaborator
|
Hi @nik9000, I've created a changelog YAML for you. |
Collaborator
|
Pinging @elastic/es-analytical-engine (Team:Analytics) |
nik9000
added
auto-backport
Comment on lines
218
to
229
| if (v instanceof Double n) { | ||
| fields.add(new DoubleField("foo", n, Field.Store.NO)); | ||
| } else if (v instanceof Float n) { | ||
| fields.add(new DoubleField("foo", n, Field.Store.NO)); | ||
| } else if (v instanceof Number n) { | ||
| long l = n.longValue(); | ||
| fields.add(new LongField("foo", l, Field.Store.NO)); | ||
| } else if (v instanceof String s) { | ||
| fields.add(new KeywordField("foo", v.toString(), Field.Store.NO)); | ||
| } else { | ||
| throw new UnsupportedOperationException(); | ||
| } |
Contributor
There was a problem hiding this comment.
NIT: could be simplified with switch like in #120875
…ug' into esql_single_value_match_query_bug
This was referenced Apr 22, 2025
nik9000
added a commit
to nik9000/elasticsearch
that referenced
this pull request
Apr 22, 2025
Fixes a sneaky bug in single value query that happens when run against
a `keyword` field that:
* Is defined on every field
* Contains the same number of distinct values as documents
The simplest way to reproduce this is to build a single shard index
with two documents:
```
{"a": "foo"}
{"a": ["foo", "bar"]}
```
I don't think this is particularly likely in production, but it's quite
likely in tests. Which is where I hit this - in the serverless tests we
index an index with four documents into three shards and two of the
documents look just like this. So about 1/3 or the time we triggered
this bug.
Mechanically this is triggered by the `SingleValueMatchQuery`
incorrectly rewriting itself to `MatchAll` in the scenario above. This
fixes that.
Collaborator
nik9000
added a commit
to nik9000/elasticsearch
that referenced
this pull request
Apr 22, 2025
Fixes a sneaky bug in single value query that happens when run against
a `keyword` field that:
* Is defined on every field
* Contains the same number of distinct values as documents
The simplest way to reproduce this is to build a single shard index
with two documents:
```
{"a": "foo"}
{"a": ["foo", "bar"]}
```
I don't think this is particularly likely in production, but it's quite
likely in tests. Which is where I hit this - in the serverless tests we
index an index with four documents into three shards and two of the
documents look just like this. So about 1/3 or the time we triggered
this bug.
Mechanically this is triggered by the `SingleValueMatchQuery`
incorrectly rewriting itself to `MatchAll` in the scenario above. This
fixes that.
nik9000
added a commit
to nik9000/elasticsearch
that referenced
this pull request
Apr 22, 2025
Fixes a sneaky bug in single value query that happens when run against
a `keyword` field that:
* Is defined on every field
* Contains the same number of distinct values as documents
The simplest way to reproduce this is to build a single shard index
with two documents:
```
{"a": "foo"}
{"a": ["foo", "bar"]}
```
I don't think this is particularly likely in production, but it's quite
likely in tests. Which is where I hit this - in the serverless tests we
index an index with four documents into three shards and two of the
documents look just like this. So about 1/3 or the time we triggered
this bug.
Mechanically this is triggered by the `SingleValueMatchQuery`
incorrectly rewriting itself to `MatchAll` in the scenario above. This
fixes that.
elasticsearchmachine
pushed a commit
that referenced
this pull request
Apr 22, 2025
Fixes a sneaky bug in single value query that happens when run against
a `keyword` field that:
* Is defined on every field
* Contains the same number of distinct values as documents
The simplest way to reproduce this is to build a single shard index
with two documents:
```
{"a": "foo"}
{"a": ["foo", "bar"]}
```
I don't think this is particularly likely in production, but it's quite
likely in tests. Which is where I hit this - in the serverless tests we
index an index with four documents into three shards and two of the
documents look just like this. So about 1/3 or the time we triggered
this bug.
Mechanically this is triggered by the `SingleValueMatchQuery`
incorrectly rewriting itself to `MatchAll` in the scenario above. This
fixes that.
elasticsearchmachine
pushed a commit
that referenced
this pull request
Apr 22, 2025
* ESQL: Fix sneaky bug in single value query (#127146) Fixes a sneaky bug in single value query that happens when run against a `keyword` field that: * Is defined on every field * Contains the same number of distinct values as documents The simplest way to reproduce this is to build a single shard index with two documents: ``` {"a": "foo"} {"a": ["foo", "bar"]} ``` I don't think this is particularly likely in production, but it's quite likely in tests. Which is where I hit this - in the serverless tests we index an index with four documents into three shards and two of the documents look just like this. So about 1/3 or the time we triggered this bug. Mechanically this is triggered by the `SingleValueMatchQuery` incorrectly rewriting itself to `MatchAll` in the scenario above. This fixes that. * Revert "Switch" This reverts commit cc7805d.
elasticsearchmachine
pushed a commit
that referenced
this pull request
Apr 22, 2025
* ESQL: Fix sneaky bug in single value query (#127146) Fixes a sneaky bug in single value query that happens when run against a `keyword` field that: * Is defined on every field * Contains the same number of distinct values as documents The simplest way to reproduce this is to build a single shard index with two documents: ``` {"a": "foo"} {"a": ["foo", "bar"]} ``` I don't think this is particularly likely in production, but it's quite likely in tests. Which is where I hit this - in the serverless tests we index an index with four documents into three shards and two of the documents look just like this. So about 1/3 or the time we triggered this bug. Mechanically this is triggered by the `SingleValueMatchQuery` incorrectly rewriting itself to `MatchAll` in the scenario above. This fixes that. * Revert "Switch" This reverts commit cc7805d.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes a sneaky bug in single value query that happens when run against a
keywordfield that:The simplest way to reproduce this is to build a single shard index with two documents:
I don't think this is particularly likely in production, but it's quite likely in tests. Which is where I hit this - in the serverless tests we index an index with four documents into three shards and two of the documents look just like this. So about 1/3 or the time we triggered this bug.
Mechanically this is triggered by the
SingleValueMatchQueryincorrectly rewriting itself toMatchAllin the scenario above. This fixes that.