Skip to content

[ML] Do not match time series counter fields with aggs in wizards#153021

Merged
jgowdyelastic merged 4 commits intoelastic:mainfrom
jgowdyelastic:remove-time-series-counter-fields-from-job-wizards
Mar 22, 2023
Merged

[ML] Do not match time series counter fields with aggs in wizards#153021
jgowdyelastic merged 4 commits intoelastic:mainfrom
jgowdyelastic:remove-time-series-counter-fields-from-job-wizards

Conversation

@jgowdyelastic
Copy link
Copy Markdown
Member

@jgowdyelastic jgowdyelastic commented Mar 9, 2023

Time series counter metric fields are treated as non aggregatable and are not matched with aggregations in the new_job_caps endpoint.
This removes them from the detector dropdowns in all wizards where we match functions(aggs) to fields.
e.g.
image

Note, the fields are not entirely removed from the new_job_caps response. So they are still available in other dropdowns.

This fixes an issue where having counter fields available for selection would cause an error.

@jgowdyelastic jgowdyelastic mentioned this pull request Mar 9, 2023
4 tasks
@jgowdyelastic jgowdyelastic self-assigned this Mar 9, 2023
@jgowdyelastic jgowdyelastic marked this pull request as ready for review March 9, 2023 15:25
@jgowdyelastic jgowdyelastic requested a review from a team as a code owner March 9, 2023 15:25
@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/ml-ui (:ml)

@droberts195 droberts195 changed the title [ML] Dot not match time series counter fields with aggs in wizards Mar 9, 2023
Comment thread x-pack/plugins/ml/server/models/job_service/new_job_caps/field_service.ts Outdated
@droberts195
Copy link
Copy Markdown

It might be possible to do something better than this for 8.8. elastic/elasticsearch#93884 is adding a list of supported aggregations. There's a CCS complication but a suggestion of how to deal with that in elastic/elasticsearch#93884 (comment).

Depending on the resolution of the CCS issue we should do one of the following:

  • Merge this PR for 8.8, then do something better in 8.9 by making use of the list of available aggregations per field
  • Make use of the list of available aggregations per field in 8.8, assuming something sensible is being returned in the case of CCS against 8.7 clusters
@kibana-ci
Copy link
Copy Markdown

💚 Build Succeeded

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
ml 3.4MB 3.4MB +32.0B
Unknown metric groups

ESLint disabled line counts

id before after diff
securitySolution 433 436 +3

Total ESLint disabled count

id before after diff
securitySolution 513 516 +3

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @jgowdyelastic

Copy link
Copy Markdown
Contributor

@peteharverson peteharverson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested and LGTM.

As @droberts195 suggested, once elastic/elasticsearch#93884 is in, we should look to show these field types and check the available aggregations.

@jgowdyelastic jgowdyelastic merged commit 28a70da into elastic:main Mar 22, 2023
@kibanamachine kibanamachine added the backport:skip This PR does not require backporting label Mar 22, 2023
qn895 added a commit that referenced this pull request Jan 13, 2026
## Summary

Previously, we [removed](#153021)
`counter` metric fields from the Anomaly detection dropdown because it
wasn't supported. Now, it is supported. This PR re-adds support for this
field type.

<img width="1717" height="917" alt="Screenshot 2026-01-07 at 14 24 49"
src="https://github.com/user-attachments/assets/771fed3b-2c1b-4dc5-adf2-ce4fa43a0ab1"
/>
<img width="1717" height="917" alt="Screenshot 2026-01-07 at 14 23 44"
src="https://github.com/user-attachments/assets/ff82f2ca-fb65-4e78-9e01-a85dc03b6d63"
/>


### Checklist

Check the PR satisfies following conditions. 

Reviewers should verify this PR satisfies this list as well.

- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
- [ ] Review the [backport
guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing)
and apply applicable `backport:*` labels.

### Identify risks

Does this PR introduce any risks? For example, consider risks like hard
to test bugs, performance regression, potential of data loss.

Describe the risk, its severity, and mitigation for each identified
risk. Invite stakeholders and evaluate how to proceed before merging.

- [ ] [See some risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
- [ ] ...

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Jan 13, 2026
…c#248187)

## Summary

Previously, we [removed](elastic#153021)
`counter` metric fields from the Anomaly detection dropdown because it
wasn't supported. Now, it is supported. This PR re-adds support for this
field type.

<img width="1717" height="917" alt="Screenshot 2026-01-07 at 14 24 49"
src="https://github.com/user-attachments/assets/771fed3b-2c1b-4dc5-adf2-ce4fa43a0ab1"
/>
<img width="1717" height="917" alt="Screenshot 2026-01-07 at 14 23 44"
src="https://github.com/user-attachments/assets/ff82f2ca-fb65-4e78-9e01-a85dc03b6d63"
/>

### Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
- [ ] Review the [backport
guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing)
and apply applicable `backport:*` labels.

### Identify risks

Does this PR introduce any risks? For example, consider risks like hard
to test bugs, performance regression, potential of data loss.

Describe the risk, its severity, and mitigation for each identified
risk. Invite stakeholders and evaluate how to proceed before merging.

- [ ] [See some risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
- [ ] ...

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
(cherry picked from commit cb89bf6)
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Jan 13, 2026
…c#248187)

## Summary

Previously, we [removed](elastic#153021)
`counter` metric fields from the Anomaly detection dropdown because it
wasn't supported. Now, it is supported. This PR re-adds support for this
field type.

<img width="1717" height="917" alt="Screenshot 2026-01-07 at 14 24 49"
src="https://github.com/user-attachments/assets/771fed3b-2c1b-4dc5-adf2-ce4fa43a0ab1"
/>
<img width="1717" height="917" alt="Screenshot 2026-01-07 at 14 23 44"
src="https://github.com/user-attachments/assets/ff82f2ca-fb65-4e78-9e01-a85dc03b6d63"
/>

### Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
- [ ] Review the [backport
guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing)
and apply applicable `backport:*` labels.

### Identify risks

Does this PR introduce any risks? For example, consider risks like hard
to test bugs, performance regression, potential of data loss.

Describe the risk, its severity, and mitigation for each identified
risk. Invite stakeholders and evaluate how to proceed before merging.

- [ ] [See some risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
- [ ] ...

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
(cherry picked from commit cb89bf6)
smith pushed a commit to smith/kibana that referenced this pull request Jan 16, 2026
…c#248187)

## Summary

Previously, we [removed](elastic#153021)
`counter` metric fields from the Anomaly detection dropdown because it
wasn't supported. Now, it is supported. This PR re-adds support for this
field type.

<img width="1717" height="917" alt="Screenshot 2026-01-07 at 14 24 49"
src="https://github.com/user-attachments/assets/771fed3b-2c1b-4dc5-adf2-ce4fa43a0ab1"
/>
<img width="1717" height="917" alt="Screenshot 2026-01-07 at 14 23 44"
src="https://github.com/user-attachments/assets/ff82f2ca-fb65-4e78-9e01-a85dc03b6d63"
/>


### Checklist

Check the PR satisfies following conditions. 

Reviewers should verify this PR satisfies this list as well.

- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
- [ ] Review the [backport
guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing)
and apply applicable `backport:*` labels.

### Identify risks

Does this PR introduce any risks? For example, consider risks like hard
to test bugs, performance regression, potential of data loss.

Describe the risk, its severity, and mitigation for each identified
risk. Invite stakeholders and evaluate how to proceed before merging.

- [ ] [See some risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
- [ ] ...

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:skip This PR does not require backporting Feature:Anomaly Detection ML anomaly detection Feature:Data Frame Analytics ML data frame analytics features :ml release_note:fix v8.8.0

7 participants