[ES|QL] Introduce AggregateMetricDoubleBlock by limotova · Pull Request #127299 · elastic/elasticsearch

limotova · 2025-04-24T02:37:23Z

This commit adds a new Block type named AggregateMetricDoubleBlock and
refactors AggregateMetricDouble to use it instead of CompositeBlock.

The reasoning behind this is that there were many areas where we lost
the distinction between a generic CompositeBlock vs using CompositeBlock
specifically for AggregateMetricDoubles, and we felt it was best to
split them into their own types.

limotova · 2025-04-28T17:40:02Z

...in/esql/compute/src/main/java/org/elasticsearch/compute/data/AggregateMetricDoubleBlock.java

+    }
+
+    @Override
+    public int getTotalValueCount() {


This is the same as CompositeBlock but I'm not sure if this is what we want for AggregateMetricDouble?

dnhatn

I've left several suggestions, but the PR looks great. Thanks Larisa!

x-pack/plugin/esql/compute/src/main/java/org/elasticsearch/compute/data/Block.java

...in/esql/compute/src/main/java/org/elasticsearch/compute/data/AggregateMetricDoubleBlock.java

x-pack/plugin/esql/compute/src/main/java/org/elasticsearch/compute/data/CompositeBlock.java

...lugin/esql/compute/src/main/java/org/elasticsearch/compute/operator/topn/ValueExtractor.java

x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/action/ResponseXContentUtils.java

...g/elasticsearch/xpack/esql/expression/function/scalar/convert/FromAggregateMetricDouble.java

...pack/esql/expression/function/scalar/convert/ToStringFromAggregateMetricDoubleEvaluator.java

…nd moving functions

dnhatn

Great work, Larisa!

...in/esql/compute/src/main/java/org/elasticsearch/compute/data/AggregateMetricDoubleBlock.java

...org/elasticsearch/xpack/esql/expression/function/scalar/convert/ToAggregateMetricDouble.java

...in/esql/compute/src/main/java/org/elasticsearch/compute/data/AggregateMetricDoubleBlock.java

elasticsearchmachine · 2025-04-29T22:29:14Z

Pinging @elastic/es-analytical-engine (Team:Analytics)

elasticsearchmachine · 2025-04-29T22:29:15Z

Hi @limotova, I've created a changelog YAML for you.

elasticsearchmachine · 2025-04-29T22:32:12Z

Pinging @elastic/es-storage-engine (Team:StorageEngine)

kkrik-es · 2025-04-30T08:26:43Z

...in/esql/compute/src/main/java/org/elasticsearch/compute/data/AggregateMetricDoubleBlock.java

+
+    @Override
+    public boolean mayHaveNulls() {
+        return Stream.of(minBlock, maxBlock, sumBlock, countBlock).anyMatch(Block::mayHaveNulls);


Nit: I thought there was a push to skip streams as they're unnecessarily expensive?

Backports the following commits to 8.19: - [ES|QL] Introduce AggregateMetricDoubleBlock (elastic#127299)

* [8.19] [ES|QL] Introduce AggregateMetricDoubleBlock (#127299) Backports the following commits to 8.19: - [ES|QL] Introduce AggregateMetricDoubleBlock (#127299) * borrow test skip from b563145

In elastic#127623 we backported elastic#127299 and added a backport transport version for it - `ESQL_AGGREGATE_METRIC_DOUBLE_BLOCK_8_19` aka `8_841_0_24`. This brings that version forwards to `main` and adds support for parsing streams with that version. In elastic#127639 we backported elastic#126401 and added a backport transport version for it - `PINNED_RETRIEVER_8_19` aka `8_841_0_23`. This brings that version forwards to `main` and adds support for parsing streams with that versions. In elastic#127633 we a claimed a backport transport version to backport elastic#127314 - `INTRODUCE_FAILURES_LIFECYCLE_BACKPORT_8_19` aka `8_841_0_23`. That's the same versions as `PINNED_RETRIEVER_8_19`. It's just that this one is in `main` and `PINNED_RETRIEVER_8_19` is in `8.19`. To allow me to bring `PINNED_RETRIEVER_8_19` for wards I've had to revert elastic#127633. Closes elastic#127667

In #127623 we backported #127299 and added a backport transport version for it - `ESQL_AGGREGATE_METRIC_DOUBLE_BLOCK_8_19` aka `8_841_0_24`. This brings that version forwards to `main` and adds support for parsing streams with that version. In #127639 we backported #126401 and added a backport transport version for it - `PINNED_RETRIEVER_8_19` aka `8_841_0_23`. This brings that version forwards to `main` and adds support for parsing streams with that versions. In #127633 we a claimed a backport transport version to backport #127314 - `INTRODUCE_FAILURES_LIFECYCLE_BACKPORT_8_19` aka `8_841_0_23`. That's the same versions as `PINNED_RETRIEVER_8_19`. It's just that this one is in `main` and `PINNED_RETRIEVER_8_19` is in `8.19`. To allow me to bring `PINNED_RETRIEVER_8_19` for wards I've had to revert #127633. Closes #127667

In elastic#127623 we backported elastic#127299 and added a backport transport version for it - `ESQL_AGGREGATE_METRIC_DOUBLE_BLOCK_8_19` aka `8_841_0_24`. This brings that version forwards to `main` and adds support for parsing streams with that version. In elastic#127639 we backported elastic#126401 and added a backport transport version for it - `PINNED_RETRIEVER_8_19` aka `8_841_0_23`. This brings that version forwards to `main` and adds support for parsing streams with that versions. In elastic#127633 we a claimed a backport transport version to backport elastic#127314 - `INTRODUCE_FAILURES_LIFECYCLE_BACKPORT_8_19` aka `8_841_0_23`. That's the same versions as `PINNED_RETRIEVER_8_19`. It's just that this one is in `main` and `PINNED_RETRIEVER_8_19` is in `8.19`. To allow me to bring `PINNED_RETRIEVER_8_19` for wards I've had to revert elastic#127633. Closes elastic#127667

elasticsearchmachine added the v9.1.0 label Apr 24, 2025

limotova force-pushed the agg-metric-block branch 8 times, most recently from 066a38d to daba3fc Compare April 28, 2025 05:12

agg metric block

865a57b

limotova force-pushed the agg-metric-block branch from daba3fc to 865a57b Compare April 28, 2025 16:21

limotova mentioned this pull request Apr 28, 2025

Add support for Downsampling in ES|QL #120759

Closed

15 tasks

limotova changed the title ~~agg metric block~~ Apr 28, 2025

limotova requested a review from dnhatn April 28, 2025 17:33

limotova commented Apr 28, 2025

View reviewed changes

dnhatn reviewed Apr 28, 2025

View reviewed changes

limotova added 3 commits April 28, 2025 17:51

convert AggMetricBlock and CompositeBlock in readTypedBlock, todos, a…

6ce22f3

…nd moving functions

fix serialize composite block test

09497b3

Merge branch 'main' into agg-metric-block

52d9fb1

limotova requested a review from dnhatn April 29, 2025 07:48

dnhatn approved these changes Apr 29, 2025

View reviewed changes

changes

caef023

limotova added >enhancement :Analytics/ES|QL AKA ESQL labels Apr 29, 2025

limotova marked this pull request as ready for review April 29, 2025 22:28

elasticsearchmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Apr 29, 2025

Update docs/changelog/127299.yaml

ebe6a37

Merge branch 'main' into agg-metric-block

eeb4acc

limotova added the :StorageEngine/TSDB You know, for Metrics label Apr 29, 2025

elasticsearchmachine added the Team:StorageEngine label Apr 29, 2025

limotova requested review from kkrik-es and nik9000 April 30, 2025 03:26

kkrik-es reviewed Apr 30, 2025

View reviewed changes

kkrik-es approved these changes Apr 30, 2025

View reviewed changes

Merge branch 'main' into agg-metric-block

8acf39e

limotova merged commit 4241842 into elastic:main May 1, 2025
17 checks passed

limotova deleted the agg-metric-block branch May 1, 2025 05:23

limotova mentioned this pull request May 2, 2025

[8.19] [ES|QL] Introduce AggregateMetricDoubleBlock (#127299) #127623

Merged

limotova added a commit to limotova/elasticsearch that referenced this pull request May 2, 2025

[8.19] [ES|QL] Introduce AggregateMetricDoubleBlock (elastic#127299)

c86702d

Backports the following commits to 8.19: - [ES|QL] Introduce AggregateMetricDoubleBlock (elastic#127299)

limotova added a commit to limotova/elasticsearch that referenced this pull request May 2, 2025

[8.19] [ES|QL] Introduce AggregateMetricDoubleBlock (elastic#127299)

a142e1d

Backports the following commits to 8.19: - [ES|QL] Introduce AggregateMetricDoubleBlock (elastic#127299)

limotova added a commit to limotova/elasticsearch that referenced this pull request May 2, 2025

[8.19] [ES|QL] Introduce AggregateMetricDoubleBlock (elastic#127299)

4f1f5a7

Backports the following commits to 8.19: - [ES|QL] Introduce AggregateMetricDoubleBlock (elastic#127299)

limotova mentioned this pull request May 2, 2025

[ES|QL] Adjust bwc for AggregateMetricDoubleBlock #127624

Closed

nik9000 mentioned this pull request May 2, 2025

ESQL: Fix transport versions #127668

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ES|QL] Introduce AggregateMetricDoubleBlock#127299

[ES|QL] Introduce AggregateMetricDoubleBlock#127299
limotova merged 8 commits intoelastic:mainfrom
limotova:agg-metric-block

limotova commented Apr 24, 2025 •

edited

Loading

limotova Apr 28, 2025

dnhatn left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dnhatn left a comment

Uh oh!

Uh oh!

Uh oh!

elasticsearchmachine commented Apr 29, 2025

elasticsearchmachine commented Apr 29, 2025

elasticsearchmachine commented Apr 29, 2025

kkrik-es Apr 30, 2025

Uh oh!

Labels

4 participants

Conversation

limotova commented Apr 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

limotova Apr 28, 2025

Choose a reason for hiding this comment

dnhatn left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dnhatn left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

elasticsearchmachine commented Apr 29, 2025

elasticsearchmachine commented Apr 29, 2025

elasticsearchmachine commented Apr 29, 2025

kkrik-es Apr 30, 2025

Choose a reason for hiding this comment

Uh oh!

Labels

4 participants

limotova commented Apr 24, 2025 •

edited

Loading