Skip to content

Serverless filtering create from#137850

Merged
seanzatzdev merged 6 commits intoelastic:mainfrom
seanzatzdev:serverless-filtering-create-from
Nov 12, 2025
Merged

Serverless filtering create from#137850
seanzatzdev merged 6 commits intoelastic:mainfrom
seanzatzdev:serverless-filtering-create-from

Conversation

@seanzatzdev
Copy link
Contributor

@seanzatzdev seanzatzdev commented Nov 10, 2025

Some index settings are not available in Serverless, however, currently the /_create_from API does not work correctly, because it tries to copy some user-unsettable settings, e.g.

illegal_argument_exception: Settings [index.routing.allocation.include._tier_preference,index.number_of_replicas,index.number_of_shards] are not available when running in serverless mode

This PR fixes this for Serverless so the API can be used there:

POST _create_from/source/dest-index
{
    "acknowledged": true,
    "shards_acknowledged": true,
    "index": "dest-index"
}

I've got a follow up PR in the serverless repo that adds tests for this logic on serverless mode: https://github.com/elastic/elasticsearch-serverless/pull/4812

@seanzatzdev seanzatzdev requested a review from Copilot November 10, 2025 21:31
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR fixes the /_create_from API in Serverless mode by filtering out settings that are not available in Serverless environments. Previously, the API attempted to copy user-unsettable settings like index.routing.allocation.include._tier_preference, index.number_of_replicas, and index.number_of_shards, causing failures. The fix adds logic to skip settings that are not serverless-public when running in serverless mode.

Key changes:

  • Added serverless mode detection using DiscoveryNode.isStateless()
  • Implemented filtering to exclude non-serverless-public settings when copying from source index

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@seanzatzdev seanzatzdev marked this pull request as ready for review November 11, 2025 18:34
@seanzatzdev seanzatzdev requested a review from masseyke November 11, 2025 18:34
@elasticsearchmachine elasticsearchmachine added the needs:triage Requires assignment of a team area label label Nov 11, 2025
@seanzatzdev seanzatzdev added >bug :Data Management/Indices APIs DO NOT USE. Use ":Distributed/Indices APIs" or ":StorageEngine/Templates" instead. labels Nov 11, 2025
@elasticsearchmachine elasticsearchmachine added Team:Data Management (obsolete) DO NOT USE. This team no longer exists. and removed needs:triage Requires assignment of a team area label labels Nov 11, 2025
@elasticsearchmachine
Copy link
Collaborator

Hi @seanzatzdev, I've created a changelog YAML for you.

@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-data-management (Team:Data Management)

@seanzatzdev seanzatzdev requested a review from a team November 12, 2025 15:47
Copy link
Contributor

@lukewhiting lukewhiting left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍🏻

@seanzatzdev seanzatzdev merged commit c6cab2e into elastic:main Nov 12, 2025
34 checks passed
@seanzatzdev seanzatzdev deleted the serverless-filtering-create-from branch November 12, 2025 16:09
@elasticsearchmachine
Copy link
Collaborator

💔 Backport failed

Status Branch Result
9.2
8.19 Commit could not be cherrypicked due to conflicts
9.1 Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 137850

seanzatzdev added a commit to seanzatzdev/elasticsearch that referenced this pull request Nov 12, 2025
* add serverless filtering in _create_from transport action

* Moved serverless test to serverless repo

* Update docs/changelog/137850.yaml
@seanzatzdev seanzatzdev restored the serverless-filtering-create-from branch November 12, 2025 16:14
seanzatzdev added a commit that referenced this pull request Nov 12, 2025
@seanzatzdev
Copy link
Contributor Author

seanzatzdev commented Nov 12, 2025

My first time using the backport label, seem to have missed those failures... Going to resolve this now

elasticsearchmachine pushed a commit that referenced this pull request Nov 12, 2025
* add serverless filtering in _create_from transport action

* Moved serverless test to serverless repo

* Update docs/changelog/137850.yaml
@seanzatzdev
Copy link
Contributor Author

💔 Some backports could not be created

Status Branch Result
9.1
8.19 An unhandled error occurred. Please see the logs for details

Manual backport

To create the backport manually run:

backport --pr 137850

Questions ?

Please refer to the Backport tool documentation

seanzatzdev added a commit to seanzatzdev/elasticsearch that referenced this pull request Nov 12, 2025
* add serverless filtering in _create_from transport action

* Moved serverless test to serverless repo

* Update docs/changelog/137850.yaml

(cherry picked from commit c6cab2e)
@seanzatzdev
Copy link
Contributor Author

💚 All backports created successfully

Status Branch Result
8.19

Questions ?

Please refer to the Backport tool documentation

seanzatzdev added a commit to seanzatzdev/elasticsearch that referenced this pull request Nov 12, 2025
* add serverless filtering in _create_from transport action

* Moved serverless test to serverless repo

* Update docs/changelog/137850.yaml

(cherry picked from commit c6cab2e)

# Conflicts:
#	x-pack/plugin/migrate/src/main/java/org/elasticsearch/xpack/migrate/action/CreateIndexFromSourceTransportAction.java
elasticsearchmachine pushed a commit that referenced this pull request Nov 13, 2025
* Serverless filtering create from (#137850)

* add serverless filtering in _create_from transport action

* Moved serverless test to serverless repo

* Update docs/changelog/137850.yaml

(cherry picked from commit c6cab2e)

* [CI] Auto commit changes from spotless

---------

Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
elasticsearchmachine pushed a commit that referenced this pull request Nov 13, 2025
* Serverless filtering create from (#137850)

* add serverless filtering in _create_from transport action

* Moved serverless test to serverless repo

* Update docs/changelog/137850.yaml

(cherry picked from commit c6cab2e)

# Conflicts:
#	x-pack/plugin/migrate/src/main/java/org/elasticsearch/xpack/migrate/action/CreateIndexFromSourceTransportAction.java

* [CI] Auto commit changes from spotless

---------

Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-backport Automatically create backport pull requests when merged backport pending >bug :Data Management/Indices APIs DO NOT USE. Use ":Distributed/Indices APIs" or ":StorageEngine/Templates" instead. Team:Data Management (obsolete) DO NOT USE. This team no longer exists. v8.19.8 v9.1.8 v9.2.2 v9.3.0

4 participants