Skip to content

Improve rolling up metrics#124739

Merged
martijnvg merged 4 commits intoelastic:mainfrom
martijnvg:downsample_metric_produces_remove_metric_abstraction
Mar 13, 2025
Merged

Improve rolling up metrics#124739
martijnvg merged 4 commits intoelastic:mainfrom
martijnvg:downsample_metric_produces_remove_metric_abstraction

Conversation

@martijnvg
Copy link
Member

Remove unneeded Metric abstraction in MetricFieldProducer, which speeds-up rolling up gauges/counters.

@martijnvg martijnvg added >enhancement :StorageEngine/Downsampling Downsampling (replacement for rollups) - Turn fine-grained time-based data into coarser-grained data v8.18.1 v8.19.0 v9.0.1 v9.1.0 labels Mar 13, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-storage-engine (Team:StorageEngine)

@elasticsearchmachine
Copy link
Collaborator

Hi @martijnvg, I've created a changelog YAML for you.

case min -> min = Math.min(value, min);
case max -> max = Math.max(value, max);
case sum -> sum.add(value);
case value_count -> count += Math.round(value);
Copy link
Contributor

@kkrik-es kkrik-es Mar 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be count++ ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this implementation is used only when rolling up downsampled indices. In that case we need to sum the value_count field of all aggregate metric double fields. I will add a comment to make this clearer.

Copy link
Contributor

@kkrik-es kkrik-es left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one comment, ptal.

@martijnvg martijnvg enabled auto-merge (squash) March 13, 2025 13:37
@martijnvg martijnvg merged commit d81c799 into elastic:main Mar 13, 2025
17 checks passed
martijnvg added a commit to martijnvg/elasticsearch that referenced this pull request Mar 13, 2025
Remove unneeded Metric abstraction in MetricFieldProducer, which speeds-up rolling up gauges/counters.
martijnvg added a commit to martijnvg/elasticsearch that referenced this pull request Mar 13, 2025
Remove unneeded Metric abstraction in MetricFieldProducer, which speeds-up rolling up gauges/counters.
martijnvg added a commit to martijnvg/elasticsearch that referenced this pull request Mar 13, 2025
Remove unneeded Metric abstraction in MetricFieldProducer, which speeds-up rolling up gauges/counters.
jfreden pushed a commit to jfreden/elasticsearch that referenced this pull request Mar 13, 2025
Remove unneeded Metric abstraction in MetricFieldProducer, which speeds-up rolling up gauges/counters.
elasticsearchmachine pushed a commit that referenced this pull request Mar 13, 2025
Remove unneeded Metric abstraction in MetricFieldProducer, which speeds-up rolling up gauges/counters.
elasticsearchmachine pushed a commit that referenced this pull request Mar 13, 2025
Remove unneeded Metric abstraction in MetricFieldProducer, which speeds-up rolling up gauges/counters.
martijnvg added a commit that referenced this pull request Mar 13, 2025
Backporting #124739 to 8.18 branch.
Remove unneeded Metric abstraction in MetricFieldProducer, which speeds-up rolling up gauges/counters.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

>enhancement :StorageEngine/Downsampling Downsampling (replacement for rollups) - Turn fine-grained time-based data into coarser-grained data Team:StorageEngine v8.18.1 v8.19.0 v9.0.1 v9.1.0

3 participants