Skip to content

Update data stream deprecations warnings to new format and filter searchable snapshots from response#118562

Merged
lukewhiting merged 13 commits intoelastic:mainfrom
lukewhiting:additional-migration-info-for-ds-reindex
Dec 19, 2024
Merged

Update data stream deprecations warnings to new format and filter searchable snapshots from response#118562
lukewhiting merged 13 commits intoelastic:mainfrom
lukewhiting:additional-migration-info-for-ds-reindex

Conversation

@lukewhiting
Copy link
Contributor

@lukewhiting lukewhiting commented Dec 12, 2024

This pull request includes changes to the DataStreamDeprecationChecks class and it's corresponding test class to implement the metadata format requested by Kibana to assist in making the cluster upgrade process smoother.

It also makes the following smaller changes:

  • Adds the reindex_required flag to the non Data Streams index migration warning.
  • Makes all migration / deprecation code use the same shared logic for determining if in an index needs migrating
  • Updates wording of error messages to say the correct ES versions
  • Filters searchable snapshots out of the list of indices needing migration
@lukewhiting lukewhiting added :StorageEngine/Data streams Data streams and their lifecycles auto-backport Automatically create backport pull requests when merged v9.0.0 v8.18.0 labels Dec 12, 2024
@lukewhiting lukewhiting force-pushed the additional-migration-info-for-ds-reindex branch 2 times, most recently from 94db41c to fdfcc44 Compare December 12, 2024 13:42
@lukewhiting lukewhiting force-pushed the additional-migration-info-for-ds-reindex branch from ba12172 to e02283f Compare December 16, 2024 16:54
@lukewhiting lukewhiting marked this pull request as ready for review December 16, 2024 16:55
@elasticsearchmachine elasticsearchmachine added the Team:Data Management (obsolete) DO NOT USE. This team no longer exists. label Dec 16, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-data-management (Team:Data Management)

@lukewhiting
Copy link
Contributor Author

Note to self: Add on hiding searchable snapshots from the deprecations API (DS and regular index) and from the migrations API

return new DeprecationIssue(
DeprecationIssue.Level.CRITICAL,
"Old data stream with a compatibility version < 8.0",
"Old data stream with a compatibility version < 9.0",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have updated the wording here. Is this correct for 9.0 (Main)? I'm guessing it will need changing back for the backport?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah for main it is correct (for now -- who knows what will be correct when we actually get to 10.0), and for 8.x it'll need to read < 8.0.

}

private static boolean creationVersionBeforeMinimumWritableVersion(IndexMetadata metadata) {
return metadata.getCreationVersion().before(MINIMUM_WRITEABLE_VERSION_AFTER_UPGRADE);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previously this was onOrBefore but I think before is correct here in that you can write to the minimum version so should not result in a warning or migration?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops. You're right!

@lukewhiting lukewhiting changed the title Update data stream deprecations warnings to new format Dec 18, 2024
@elasticsearchmachine
Copy link
Collaborator

Hi @lukewhiting, I've created a changelog YAML for you.

@lukewhiting
Copy link
Contributor Author

lukewhiting commented Dec 18, 2024

@masseyke @parkertimmins Have pushed up the changes we discussed yesterday RE @dakrone asking us to filter snapshots and centralising the predicate.

Copy link
Member

@masseyke masseyke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lukewhiting lukewhiting removed the auto-backport Automatically create backport pull requests when merged label Dec 18, 2024
@lukewhiting lukewhiting enabled auto-merge (squash) December 18, 2024 16:08
@lukewhiting lukewhiting merged commit 5487927 into elastic:main Dec 19, 2024
lukewhiting added a commit to lukewhiting/elasticsearch that referenced this pull request Dec 19, 2024
…rchable snapshots from response (elastic#118562)

* Update data stream deprecations warnings to new format

* Add reindex_required flag to index version deprecation notice response

* PR Changes

* Move all deprecation checks to use a shared predicate which also excludes snapshots

* Update docs/changelog/118562.yaml

* Tests for excluding snapshots

* PR Changes - Remove leftover comment

(cherry picked from commit 5487927)
lukewhiting added a commit that referenced this pull request Dec 19, 2024
…er sea… (#119097)

* Update data stream deprecations warnings to new format and filter searchable snapshots from response (#118562)
* Update data stream deprecations warnings to new format

* Add reindex_required flag to index version deprecation notice response

* PR Changes

* Move all deprecation checks to use a shared predicate which also excludes snapshots

* Update docs/changelog/118562.yaml

* Tests for excluding snapshots

* PR Changes - Remove leftover comment

(cherry picked from commit 5487927)

* Update docs/changelog/119097.yaml
@lukewhiting lukewhiting deleted the additional-migration-info-for-ds-reindex branch December 19, 2024 16:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

>enhancement :StorageEngine/Data streams Data streams and their lifecycles Team:Data Management (obsolete) DO NOT USE. This team no longer exists. v8.18.0 v9.0.0

4 participants