Add index and reindex request settings to speed up reindex#119780
Add index and reindex request settings to speed up reindex#119780parkertimmins merged 6 commits intoelastic:mainfrom
Conversation
parkertimmins
added
>enhancement
:StorageEngine/Data streams
elasticsearchmachine
added
the
Team:Data Management (obsolete)
|
Pinging @elastic/es-data-management (Team:Data Management) |
|
Hi @parkertimmins, I've created a changelog YAML for you. |
- if source index was read-only, setting update needs to happen before test is made read only - default test was failing due to a template causing number_of_replica to not come from setting default
| // random_index_template sets value for number_of_replicas, remove template so default value is used instead | ||
| assertAcked( | ||
| indicesAdmin().execute(TransportDeleteIndexTemplateAction.TYPE, new DeleteIndexTemplateRequest("random_index_template")) | ||
| ); |
There was a problem hiding this comment.
This is a bit hacky, but not sure of a better way to force number_of_replicas to use the default value.
| builder.copy(setting, settingsBefore); | ||
| } else { | ||
| // otherwise, delete from dest index so that it loads from the settings default | ||
| builder.putNull(setting); |
There was a problem hiding this comment.
Alternatively, we could get the current setting value using one of the methods that falls back to the default, then set this value on the dest index. But I prefer the current version as it avoids adding a settings explicitly to the dest index which had been unset (and using the default) on the source index.
masseyke
left a comment
masseyke
left a comment
There was a problem hiding this comment.
We talked offline about maybe using a higher-priority template rather than deleting random_index_template, but I think either one gives the same result (and either one requires that you know that randomly sometimes a random_index_template changes your default replicas.
parkertimmins
added
auto-backport
💚 All backports created successfully
Questions ?Please refer to the Backport tool documentation |
…19780) - set slices:auto on the reindex request - set refresh_interval: -1 on destination index before reindexing into it - set number_of_replicas: 0 on destination index before reindexing into it - reset refresh_interval and number_of_replicas to previous value or default after reindex (cherry picked from commit c4024dc)
…119992) - set slices:auto on the reindex request - set refresh_interval: -1 on destination index before reindexing into it - set number_of_replicas: 0 on destination index before reindexing into it - reset refresh_interval and number_of_replicas to previous value or default after reindex (cherry picked from commit c4024dc)
3 participants
Just a couple optimizations to speed up the reindexing of a single data stream index: