Bug fix: the filter of a data stream alias is not always properly removed. by gmarouli · Pull Request #139679 · elastic/elasticsearch

gmarouli · 2025-12-17T10:28:41Z

A data stream alias that has more than 1 data stream does not handle correctly the removal of a filtered data stream alias. It currently retains the filter even after the alias is removed.

# Create 2 data streams

PUT _data_stream/logs-yyy-default
PUT _data_stream/logs-xxx-default

# Add a filtered alias to one and a simple alias to the other.
POST _aliases
{
  "actions": [
    {
      "add": {
        "index": "logs-yyy-default",
        "alias": "myalias",
        "filter": {
          "term": {
            "type": "mytype"
          }
        }
      }
    },
    {
      "add": {
        "index": "logs-xxx-default",
        "alias": "myalias"
      }
    }
  ]
}

# Remove the filtered alias
POST _aliases
{
  "actions": [
    {
      "remove": {
        "index": "logs-yyy-default",
        "alias": "myalias"
      }
    }
  ]
}

# Add back the alias but WITHOUT filter
POST _aliases
{
  "actions": [
    {
      "add": {
        "index": "logs-yyy-default",
        "alias": "myalias"
       }
    }
  ]
}

The last alias will contain the initial filter while it should not. This PR fixes that.

Furthermore, we edit the update and removal of data streams from an alias to be able to handle orphaned aliases.

…emoved

elasticsearchmachine · 2025-12-17T10:29:07Z

Pinging @elastic/es-data-management (Team:Data Management)

elasticsearchmachine · 2025-12-17T10:29:31Z

Hi @gmarouli, I've created a changelog YAML for you.

Copilot

Pull request overview

This PR fixes a bug where filtered data stream aliases were not properly removed when an alias was deleted and then re-added without a filter. The filter from the original alias would persist incorrectly. Additionally, the PR adds handling for orphaned alias filters (filters that exist for data streams no longer in the alias).

Key Changes:

Modified DataStreamAlias.update() and DataStreamAlias.removeDataStream() to properly clean up filters when removing data streams or updating aliases without filters
Added capability flag fix_filtered_data_stream_alias_removal to track this fix
Added comprehensive test coverage for filtered alias removal and orphaned filter cleanup

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
DataStreamAlias.java	Core logic fix to properly remove filters when updating or removing data streams from aliases
DataStreamAliasTests.java	Added unit tests for filtered data stream removal and orphaned filter cleanup
RestIndicesAliasesAction.java	Added capability flag to indicate the bug fix is available
140_data_stream_aliases.yml	Added integration test verifying filtered alias removal behavior
139679.yaml	Changelog entry documenting the bug fix

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-12-17T10:33:19Z

server/src/main/java/org/elasticsearch/cluster/metadata/DataStreamAlias.java

+        if (filterAsMap != null || dataStreamToFilterMap.containsKey(dataStream)) {
            CompressedXContent previousFilter = dataStreamToFilterMap.get(dataStream);
-            if (previousFilter == null) {
+            if (previousFilter == null || filterAsMap == null) {


The condition previousFilter == null || filterAsMap == null could be clearer. Consider splitting this into separate conditions with explicit comments explaining when each case occurs (adding a new filter vs. removing an existing filter).

lukewhiting

Looks good to me 👍🏻 Thanks for sorting this!

elasticsearchmachine · 2025-12-17T13:57:52Z

💚 Backport successful

Status	Branch	Result
✅	9.1
✅	9.2
✅	8.19

…oved. (elastic#139679) A data stream alias that has more than 1 data stream does not handle correctly the removal of a filtered data stream alias. It currently retains the filter even after the alias is removed. ``` # Create 2 data streams PUT _data_stream/logs-yyy-default PUT _data_stream/logs-xxx-default # Add a filtered alias to one and a simple alias to the other. POST _aliases { "actions": [ { "add": { "index": "logs-yyy-default", "alias": "myalias", "filter": { "term": { "type": "mytype" } } } }, { "add": { "index": "logs-xxx-default", "alias": "myalias" } } ] } # Remove the filtered alias POST _aliases { "actions": [ { "remove": { "index": "logs-yyy-default", "alias": "myalias" } } ] } # Add back the alias but WITHOUT filter POST _aliases { "actions": [ { "add": { "index": "logs-yyy-default", "alias": "myalias" } } ] } ``` The last alias will contain the initial filter while it should not. This PR fixes that. Furthermore, we edit the update and removal of data streams from an alias to be able to handle orphaned aliases.

…oved. (#139679) (#139688) A data stream alias that has more than 1 data stream does not handle correctly the removal of a filtered data stream alias. It currently retains the filter even after the alias is removed. ``` # Create 2 data streams PUT _data_stream/logs-yyy-default PUT _data_stream/logs-xxx-default # Add a filtered alias to one and a simple alias to the other. POST _aliases { "actions": [ { "add": { "index": "logs-yyy-default", "alias": "myalias", "filter": { "term": { "type": "mytype" } } } }, { "add": { "index": "logs-xxx-default", "alias": "myalias" } } ] } # Remove the filtered alias POST _aliases { "actions": [ { "remove": { "index": "logs-yyy-default", "alias": "myalias" } } ] } # Add back the alias but WITHOUT filter POST _aliases { "actions": [ { "add": { "index": "logs-yyy-default", "alias": "myalias" } } ] } ``` The last alias will contain the initial filter while it should not. This PR fixes that. Furthermore, we edit the update and removal of data streams from an alias to be able to handle orphaned aliases.

…oved. (#139679) (#139689) A data stream alias that has more than 1 data stream does not handle correctly the removal of a filtered data stream alias. It currently retains the filter even after the alias is removed. ``` # Create 2 data streams PUT _data_stream/logs-yyy-default PUT _data_stream/logs-xxx-default # Add a filtered alias to one and a simple alias to the other. POST _aliases { "actions": [ { "add": { "index": "logs-yyy-default", "alias": "myalias", "filter": { "term": { "type": "mytype" } } } }, { "add": { "index": "logs-xxx-default", "alias": "myalias" } } ] } # Remove the filtered alias POST _aliases { "actions": [ { "remove": { "index": "logs-yyy-default", "alias": "myalias" } } ] } # Add back the alias but WITHOUT filter POST _aliases { "actions": [ { "add": { "index": "logs-yyy-default", "alias": "myalias" } } ] } ``` The last alias will contain the initial filter while it should not. This PR fixes that. Furthermore, we edit the update and removal of data streams from an alias to be able to handle orphaned aliases.

…oved. (#139679) (#139691) A data stream alias that has more than 1 data stream does not handle correctly the removal of a filtered data stream alias. It currently retains the filter even after the alias is removed. ``` # Create 2 data streams PUT _data_stream/logs-yyy-default PUT _data_stream/logs-xxx-default # Add a filtered alias to one and a simple alias to the other. POST _aliases { "actions": [ { "add": { "index": "logs-yyy-default", "alias": "myalias", "filter": { "term": { "type": "mytype" } } } }, { "add": { "index": "logs-xxx-default", "alias": "myalias" } } ] } # Remove the filtered alias POST _aliases { "actions": [ { "remove": { "index": "logs-yyy-default", "alias": "myalias" } } ] } # Add back the alias but WITHOUT filter POST _aliases { "actions": [ { "add": { "index": "logs-yyy-default", "alias": "myalias" } } ] } ``` The last alias will contain the initial filter while it should not. This PR fixes that. Furthermore, we edit the update and removal of data streams from an alias to be able to handle orphaned aliases.

gmarouli added 4 commits December 17, 2025 11:40

Add tests to capture the bug

126db02

Ensure that a data stream alias filter is removed when the alias is r…

5a2f997

…emoved

Add test that handle already orphaned filters

f82328d

Ensure update and remove handle orphaned filters

1ac7ebe

gmarouli added >bug :StorageEngine/Data streams Data streams and their lifecycles auto-backport Automatically create backport pull requests when merged branch:9.2 branch:9.1 branch:8.19 labels Dec 17, 2025

elasticsearchmachine added Team:Data Management (obsolete) DO NOT USE. This team no longer exists. v9.1.10 v9.2.4 v9.3.0 v8.19.10 labels Dec 17, 2025

elasticsearchmachine removed branch:9.2 branch:9.1 branch:8.19 labels Dec 17, 2025

Update docs/changelog/139679.yaml

47ffda9

lukewhiting requested a review from Copilot December 17, 2025 10:32

Copilot AI reviewed Dec 17, 2025

View reviewed changes

lukewhiting approved these changes Dec 17, 2025

View reviewed changes

Fix re-adding an existing alias

aa1eccb

gmarouli added the auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) label Dec 17, 2025

elasticsearchmachine merged commit 7e84a3c into elastic:main Dec 17, 2025
35 checks passed

gmarouli deleted the fix-data-stream-filtered-alias-clean-up branch December 17, 2025 13:56

This was referenced Dec 17, 2025

[9.1] Bug fix: the filter of a data stream alias is not always properly removed. (#139679) #139688

Merged

[9.2] Bug fix: the filter of a data stream alias is not always properly removed. (#139679) #139689

Merged

gmarouli mentioned this pull request Dec 17, 2025

[8.19] Bug fix: the filter of a data stream alias is not always properly removed. (#139679) #139691

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bug fix: the filter of a data stream alias is not always properly removed.#139679

Bug fix: the filter of a data stream alias is not always properly removed.#139679
elasticsearchmachine merged 6 commits intoelastic:mainfrom
gmarouli:fix-data-stream-filtered-alias-clean-up

gmarouli commented Dec 17, 2025

elasticsearchmachine commented Dec 17, 2025

elasticsearchmachine commented Dec 17, 2025

Copilot AI left a comment

Copilot AI Dec 17, 2025

lukewhiting left a comment

Uh oh!

elasticsearchmachine commented Dec 17, 2025

Labels

4 participants

Conversation

gmarouli commented Dec 17, 2025

elasticsearchmachine commented Dec 17, 2025

elasticsearchmachine commented Dec 17, 2025

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Copilot AI Dec 17, 2025

Choose a reason for hiding this comment

lukewhiting left a comment

Choose a reason for hiding this comment

Uh oh!

elasticsearchmachine commented Dec 17, 2025

💚 Backport successful

Labels

4 participants