Skip to content

[Fleet] enable feature flag enableAutoInstallContentPackages#232668

Merged
juliaElastic merged 6 commits intoelastic:mainfrom
juliaElastic:enable-content-package-feature-flag
Aug 27, 2025
Merged

[Fleet] enable feature flag enableAutoInstallContentPackages#232668
juliaElastic merged 6 commits intoelastic:mainfrom
juliaElastic:enable-content-package-feature-flag

Conversation

@juliaElastic
Copy link
Contributor

@juliaElastic juliaElastic commented Aug 22, 2025

Summary

Closes https://github.com/elastic/ingest-dev/issues/5872

Blocked by elastic/elasticsearch#132501

Release note

Enable feature to auto install content packages where data is ingested for the matching datasets defined in the packages' discovery fields.

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

  • Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
  • Documentation was added for features that require explanation or tutorials
  • Unit or functional tests were updated or added to match the most common scenarios
  • If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
  • This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The release_note:breaking label should be applied in these situations.
  • Flaky Test Runner was used on any tests changed
  • The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines
  • Review the backport guidelines and apply applicable backport:* labels.

Identify risks

Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss.

Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging.

@juliaElastic juliaElastic self-assigned this Aug 22, 2025
@juliaElastic juliaElastic added backport:skip This PR does not require backporting release_note:feature Makes this part of the condensed release notes labels Aug 22, 2025
@juliaElastic juliaElastic added the ci:cloud-deploy Create or update a Cloud deployment label Aug 25, 2025
const query = `FROM logs-*,metrics-*,traces-*
| KEEP @timestamp, data_stream.dataset
| WHERE @timestamp > NOW() - ${this.intervalToEsql(this.taskInterval)}
| WHERE @timestamp > NOW() - 15 minutes
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a longer time range in case querying discovery fields takes longer before doing the esql query.
It doesn't matter if there is a time overlap between 2 runs, as already installed packages are filtered out.

@juliaElastic juliaElastic marked this pull request as ready for review August 25, 2025 08:50
@juliaElastic juliaElastic requested a review from a team as a code owner August 25, 2025 08:50
Copy link
Contributor

@jillguyonnet jillguyonnet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀

@botelastic botelastic bot added the Team:Fleet Team label for Observability Data Collection Fleet team label Aug 26, 2025
@elasticmachine
Copy link
Contributor

Pinging @elastic/fleet (Team:Fleet)

this.logger.info(`[AutoInstallContentPackagesTask] ESQL query took: ${response.took}ms`);

const datasetsWithData: string[] = response.values.map((value: any[]) => value[1]);
const body = await esClient.indices.getDataStream({
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed the implementation to read data stream metadata instead of the ingested docs and derive the dataset name from the data stream name.
Related discussion: elastic/elasticsearch#132501 (comment)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested with the kibana pr image and verified that kibana can read the data streams API:
https://buildkite.com/elastic/observability-perf/builds/4668#0198eb26-d3a7-4eef-92c5-a428598f7158

@elasticmachine
Copy link
Contributor

elasticmachine commented Aug 27, 2025

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

cc @juliaElastic

@juliaElastic juliaElastic merged commit 9933241 into elastic:main Aug 27, 2025
12 checks passed
kowalczyk-krzysztof pushed a commit to kowalczyk-krzysztof/kibana that referenced this pull request Aug 30, 2025
…ic#232668)

## Summary

Closes elastic/ingest-dev#5872

Blocked by elastic/elasticsearch#132501

## Release note

Enable feature to auto install content packages where data is ingested
for the matching datasets defined in the packages' discovery fields.


### Checklist

Check the PR satisfies following conditions. 

Reviewers should verify this PR satisfies this list as well.

- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
- [ ] Review the [backport
guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing)
and apply applicable `backport:*` labels.

### Identify risks

Does this PR introduce any risks? For example, consider risks like hard
to test bugs, performance regression, potential of data loss.

Describe the risk, its severity, and mitigation for each identified
risk. Invite stakeholders and evaluate how to proceed before merging.

- [ ] [See some risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
- [ ] ...
qn895 pushed a commit to qn895/kibana that referenced this pull request Sep 2, 2025
…ic#232668)

## Summary

Closes elastic/ingest-dev#5872

Blocked by elastic/elasticsearch#132501

## Release note

Enable feature to auto install content packages where data is ingested
for the matching datasets defined in the packages' discovery fields.


### Checklist

Check the PR satisfies following conditions. 

Reviewers should verify this PR satisfies this list as well.

- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
- [ ] Review the [backport
guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing)
and apply applicable `backport:*` labels.

### Identify risks

Does this PR introduce any risks? For example, consider risks like hard
to test bugs, performance regression, potential of data loss.

Describe the risk, its severity, and mitigation for each identified
risk. Invite stakeholders and evaluate how to proceed before merging.

- [ ] [See some risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
- [ ] ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:skip This PR does not require backporting ci:cloud-deploy Create or update a Cloud deployment release_note:feature Makes this part of the condensed release notes Team:Fleet Team label for Observability Data Collection Fleet team v9.2.0

4 participants