Add support for specifying reindexing script for system index migration

[alexey-ivanov-es]

Adds support for setting a reindexing script for system index migration during /_migration/system_features task.
Script is set using SystemIndexDescriptor.Builder.setMigrationScript method.

Changes in this PR will be backported to 8.x branch

[alexey-ivanov-es]

This change here is only to illustrate approach. I'll revert it later and create a separate PR for 8.x branch after all other changes is backported there. The script will only be in 8.18, not in 9.0

[alexey-ivanov-es]

Verified manually 7.0.1 (-> 7.17.0) -> 8.18:
POST http://localhost:9200/_migration/system_features:

{
    "accepted": true,
    "features": [
        {
            "feature_name": "geoip"
        },
        {
            "feature_name": "watcher"
        }
    ]
}

types field was removed from the document created in v7

[elasticsearchmachine]

Hi @alexey-ivanov-es, I've created a changelog YAML for you.

[elasticsearchmachine]

Pinging @elastic/es-core-infra (Team:Core/Infra)

[JVerwolf]

LGTM. There is some unused code to clean up, and I left a comment on the Javadoc, but overall this looks good.

I was thinking that this reindexing script is tied to the migration between specific versions, in this case V7->V8. I wonder if we should make that relationship explicit somehow? Otherwise, during the next migration from V8->V9, any descriptor with a script will likely be broken by default.

[JVerwolf]

WDYT of adding some additional details here?
e.g.:

    /**
    * An optional reindexing script to use when migrating an index from V7->V8. This script can be used to modify documents
    * before they are added to the new index. For example, it could be used to migrate off deprecated features.
    * <p> 
    * See https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html#reindex-scripts
    */

[alexey-ivanov-es]

ok, I'll add more details here, however, I would try to avoid mentioning any specific versions 🙂 Thank you!

[JVerwolf]

Is this used?

[alexey-ivanov-es]

In JavaDoc, line 151. I think it is ok

[JVerwolf]

Should we call this setV7toV8MigrationScript? This way, we can deprecate it with a annotation later, and easily track usages of outdated scripts prior to the next migration. For the next migration, we'd then add setV8toV9MigrationScript, and so on.

[alexey-ivanov-es]

I don't really want to mention any specific versions in the name of the method, because otherwise we will have to do additional job for every major release and I think it is better to avoid it. Also #119001 (comment)

[alexey-ivanov-es]

I was thinking that this reindexing script is tied to the migration between specific versions, in this case V7->V8. I wonder if we should make that relationship explicit somehow? Otherwise, during the next migration from V8->V9, any descriptor with a script will likely be broken by default.

@JVerwolf it is. I thought about having some kind of Map<Version, Script>, but it would complicate things unnecessarily: the reindexing script is only required in the last (two) minor of previous major (in our case in 8.18 and 8.19 for migration to 9.0). Therefore, the script should only exist in those minors and be introduced in the "last minor" branch after it diverges from the "next major" (main) branch. This way, there will be no need to update/remove the script - it will only exist in the branches where it should be applied. I'll add a note to the JavaDoc to clarify this

[elasticsearchmachine]

💔 Backport failed

Status	Branch	Result
❌	8.x	Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 119001

[alexey-ivanov-es]

8.x PR: #120333