Skip to content

[Data Streams] Fix displayed index mode#215683

Merged
ElenaStoeva merged 10 commits intoelastic:mainfrom
ElenaStoeva:data-streams/fix-index-mode-logic
Apr 16, 2025
Merged

[Data Streams] Fix displayed index mode#215683
ElenaStoeva merged 10 commits intoelastic:mainfrom
ElenaStoeva:data-streams/fix-index-mode-logic

Conversation

@ElenaStoeva
Copy link
Contributor

@ElenaStoeva ElenaStoeva commented Mar 24, 2025

Fixes #208671

Summary

Before this PR, the displayed index mode of the data streams was determined based on the index mode of the associated index template. However, the index mode can also be set through the component template, so that logic is not reliable and can cause incorrectly displayed index mode like described in #208671.

In this PR, we replace this logic with the recently added index_mode field to the Es Get Data Streams API (see elastic/elasticsearch#122486).

How to test:

  1. Create a component template with a LogsDB index mode (you can also test with other index modes):
PUT _component_template/my-component-template
{
  "template": {
    "settings": {
      "index": {
        "mode": "logsdb"
      }
    }
  }
}
  1. Create an index template that is composed of the component template above:
PUT _index_template/my-index-template
{
  "index_patterns": [
    "my-ds-*"
  ],
  "data_stream": {},
  "composed_of": [
    "my-component-template"
  ]
}
  1. Create a data stream that matched the index pattern from the index template above:
PUT _data_stream/my-ds-1
  1. Go to the data streams table and verify that the index mode is displayed correctly in the table.
Screenshot 2025-03-24 at 18 12 04
  1. Click on the created data stream and verify that the displayed index mode in the details panel is correct:
Screenshot 2025-03-06 at 14 36 12
@ElenaStoeva ElenaStoeva added Feature:Index Management Index and index templates UI Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more t// release_note:skip Skip the PR/issue when compiling release notes backport:prev-major labels Mar 24, 2025
@ElenaStoeva ElenaStoeva self-assigned this Mar 24, 2025
@ElenaStoeva ElenaStoeva requested a review from a team as a code owner March 24, 2025 12:35
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-management (Team:Kibana Management)

@ElenaStoeva
Copy link
Contributor Author

Marking as a draft for now as I noticed that the index mode field from the Es API is not behaving as expected - it doesn't get updated after we update the index mode of the index template/component template:

Screen.Recording.2025-03-25.at.11.01.33.mov

Will follow up on this.

@ElenaStoeva ElenaStoeva marked this pull request as draft March 25, 2025 11:15
@ElenaStoeva ElenaStoeva marked this pull request as ready for review April 7, 2025 17:21
@ElenaStoeva
Copy link
Contributor Author

Reopening as it's okay if the displayed index mode is not immediately updated after a change of index mode of the index template as per discussion with @elastic/es-data-management.

@ElenaStoeva ElenaStoeva force-pushed the data-streams/fix-index-mode-logic branch from 93e1b5c to 973ef80 Compare April 9, 2025 10:54
@SoniaSanzV
Copy link
Contributor

@elasticmachine merge upstream

Copy link
Contributor

@SoniaSanzV SoniaSanzV left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tested locally, lgtm :)

@ElenaStoeva ElenaStoeva enabled auto-merge (squash) April 15, 2025 13:46
@ElenaStoeva ElenaStoeva added backport:skip This PR does not require backporting v9.1.0 and removed backport:prev-major labels Apr 16, 2025
@ElenaStoeva ElenaStoeva merged commit dec6a17 into elastic:main Apr 16, 2025
25 checks passed
@elasticmachine
Copy link
Contributor

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

  • [job] [logs] FTR Configs #40 / security APIs - Login Selector Login Selector SAML should be able to have many pending SP initiated logins all successfully succeed

Metrics [docs]

✅ unchanged

History

cc @ElenaStoeva

akowalska622 pushed a commit to akowalska622/kibana that referenced this pull request Apr 17, 2025
Fixes elastic#208671

## Summary

Before this PR, the displayed index mode of the data streams was
determined based on the index mode of the associated index template.
However, the index mode can also be set through the component template,
so that logic is not reliable and can cause incorrectly displayed index
mode like described in elastic#208671.

In this PR, we replace this logic with the recently added `index_mode`
field to the Es Get Data Streams API (see
elastic/elasticsearch#122486).

**How to test:**
1. Create a component template with a LogsDB index mode (you can also
test with other index modes):
```
PUT _component_template/my-component-template
{
  "template": {
    "settings": {
      "index": {
        "mode": "logsdb"
      }
    }
  }
}
```
2. Create an index template that is composed of the component template
above:
```
PUT _index_template/my-index-template
{
  "index_patterns": [
    "my-ds-*"
  ],
  "data_stream": {},
  "composed_of": [
    "my-component-template"
  ]
}
```
3. Create a data stream that matched the index pattern from the index
template above:
```
PUT _data_stream/my-ds-1
```
4. Go to the data streams table and verify that the index mode is
displayed correctly in the table.

<img width="1165" alt="Screenshot 2025-03-24 at 18 12 04"
src="https://github.com/user-attachments/assets/ea211c14-3d03-49c7-ace7-88b15e294d1f"
/>


5. Click on the created data stream and verify that the displayed index
mode in the details panel is correct:

<img width="1165" alt="Screenshot 2025-03-06 at 14 36 12"
src="https://github.com/user-attachments/assets/954864e2-ae2a-4cb8-9eef-2c5f8b417f52"
/>

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
@mattkime mattkime added backport:version Backport to applied version labels and removed v8.18.1 v9.0.1 v8.17.6 backport:skip This PR does not require backporting labels May 21, 2025
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.19

https://github.com/elastic/kibana/actions/runs/15170415637

@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.19

https://github.com/elastic/kibana/actions/runs/15170415587

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request May 21, 2025
Fixes elastic#208671

## Summary

Before this PR, the displayed index mode of the data streams was
determined based on the index mode of the associated index template.
However, the index mode can also be set through the component template,
so that logic is not reliable and can cause incorrectly displayed index
mode like described in elastic#208671.

In this PR, we replace this logic with the recently added `index_mode`
field to the Es Get Data Streams API (see
elastic/elasticsearch#122486).

**How to test:**
1. Create a component template with a LogsDB index mode (you can also
test with other index modes):
```
PUT _component_template/my-component-template
{
  "template": {
    "settings": {
      "index": {
        "mode": "logsdb"
      }
    }
  }
}
```
2. Create an index template that is composed of the component template
above:
```
PUT _index_template/my-index-template
{
  "index_patterns": [
    "my-ds-*"
  ],
  "data_stream": {},
  "composed_of": [
    "my-component-template"
  ]
}
```
3. Create a data stream that matched the index pattern from the index
template above:
```
PUT _data_stream/my-ds-1
```
4. Go to the data streams table and verify that the index mode is
displayed correctly in the table.

<img width="1165" alt="Screenshot 2025-03-24 at 18 12 04"
src="https://github.com/user-attachments/assets/ea211c14-3d03-49c7-ace7-88b15e294d1f"
/>

5. Click on the created data stream and verify that the displayed index
mode in the details panel is correct:

<img width="1165" alt="Screenshot 2025-03-06 at 14 36 12"
src="https://github.com/user-attachments/assets/954864e2-ae2a-4cb8-9eef-2c5f8b417f52"
/>

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
(cherry picked from commit dec6a17)
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
8.19

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request May 21, 2025
# Backport

This will backport the following commits from `main` to `8.19`:
- [[Data Streams] Fix displayed index mode
(#215683)](#215683)

<!--- Backport version: 9.6.6 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Elena
Stoeva","email":"59341489+ElenaStoeva@users.noreply.github.com"},"sourceCommit":{"committedDate":"2025-04-16T13:09:33Z","message":"[Data
Streams] Fix displayed index mode (#215683)\n\nFixes
https://github.com/elastic/kibana/issues/208671\n\n## Summary\n\nBefore
this PR, the displayed index mode of the data streams was\ndetermined
based on the index mode of the associated index template.\nHowever, the
index mode can also be set through the component template,\nso that
logic is not reliable and can cause incorrectly displayed index\nmode
like described in https://github.com/elastic/kibana/issues/208671.\n\nIn
this PR, we replace this logic with the recently added
`index_mode`\nfield to the Es Get Data Streams API
(see\nhttps://github.com/elastic/elasticsearch/pull/122486).\n\n**How to
test:**\n1. Create a component template with a LogsDB index mode (you
can also\ntest with other index modes):\n```\nPUT
_component_template/my-component-template\n{\n \"template\": {\n
\"settings\": {\n \"index\": {\n \"mode\": \"logsdb\"\n }\n }\n
}\n}\n```\n2. Create an index template that is composed of the component
template\nabove:\n```\nPUT _index_template/my-index-template\n{\n
\"index_patterns\": [\n \"my-ds-*\"\n ],\n \"data_stream\": {},\n
\"composed_of\": [\n \"my-component-template\"\n ]\n}\n```\n3. Create a
data stream that matched the index pattern from the index\ntemplate
above:\n```\nPUT _data_stream/my-ds-1\n```\n4. Go to the data streams
table and verify that the index mode is\ndisplayed correctly in the
table.\n\n<img width=\"1165\" alt=\"Screenshot 2025-03-24 at 18 12
04\"\nsrc=\"https://github.com/user-attachments/assets/ea211c14-3d03-49c7-ace7-88b15e294d1f\"\n/>\n\n\n5.
Click on the created data stream and verify that the displayed
index\nmode in the details panel is correct:\n\n<img width=\"1165\"
alt=\"Screenshot 2025-03-06 at 14 36
12\"\nsrc=\"https://github.com/user-attachments/assets/954864e2-ae2a-4cb8-9eef-2c5f8b417f52\"\n/>\n\n---------\n\nCo-authored-by:
Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"dec6a17ec02ffea5669ffcf8431009aef1e99cfa","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Feature:Index
Management","Team:Kibana
Management","release_note:skip","backport:version","v9.1.0","v8.19.0"],"title":"[Data
Streams] Fix displayed index
mode","number":215683,"url":"https://github.com/elastic/kibana/pull/215683","mergeCommit":{"message":"[Data
Streams] Fix displayed index mode (#215683)\n\nFixes
https://github.com/elastic/kibana/issues/208671\n\n## Summary\n\nBefore
this PR, the displayed index mode of the data streams was\ndetermined
based on the index mode of the associated index template.\nHowever, the
index mode can also be set through the component template,\nso that
logic is not reliable and can cause incorrectly displayed index\nmode
like described in https://github.com/elastic/kibana/issues/208671.\n\nIn
this PR, we replace this logic with the recently added
`index_mode`\nfield to the Es Get Data Streams API
(see\nhttps://github.com/elastic/elasticsearch/pull/122486).\n\n**How to
test:**\n1. Create a component template with a LogsDB index mode (you
can also\ntest with other index modes):\n```\nPUT
_component_template/my-component-template\n{\n \"template\": {\n
\"settings\": {\n \"index\": {\n \"mode\": \"logsdb\"\n }\n }\n
}\n}\n```\n2. Create an index template that is composed of the component
template\nabove:\n```\nPUT _index_template/my-index-template\n{\n
\"index_patterns\": [\n \"my-ds-*\"\n ],\n \"data_stream\": {},\n
\"composed_of\": [\n \"my-component-template\"\n ]\n}\n```\n3. Create a
data stream that matched the index pattern from the index\ntemplate
above:\n```\nPUT _data_stream/my-ds-1\n```\n4. Go to the data streams
table and verify that the index mode is\ndisplayed correctly in the
table.\n\n<img width=\"1165\" alt=\"Screenshot 2025-03-24 at 18 12
04\"\nsrc=\"https://github.com/user-attachments/assets/ea211c14-3d03-49c7-ace7-88b15e294d1f\"\n/>\n\n\n5.
Click on the created data stream and verify that the displayed
index\nmode in the details panel is correct:\n\n<img width=\"1165\"
alt=\"Screenshot 2025-03-06 at 14 36
12\"\nsrc=\"https://github.com/user-attachments/assets/954864e2-ae2a-4cb8-9eef-2c5f8b417f52\"\n/>\n\n---------\n\nCo-authored-by:
Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"dec6a17ec02ffea5669ffcf8431009aef1e99cfa"}},"sourceBranch":"main","suggestedTargetBranches":["8.19"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/215683","number":215683,"mergeCommit":{"message":"[Data
Streams] Fix displayed index mode (#215683)\n\nFixes
https://github.com/elastic/kibana/issues/208671\n\n## Summary\n\nBefore
this PR, the displayed index mode of the data streams was\ndetermined
based on the index mode of the associated index template.\nHowever, the
index mode can also be set through the component template,\nso that
logic is not reliable and can cause incorrectly displayed index\nmode
like described in https://github.com/elastic/kibana/issues/208671.\n\nIn
this PR, we replace this logic with the recently added
`index_mode`\nfield to the Es Get Data Streams API
(see\nhttps://github.com/elastic/elasticsearch/pull/122486).\n\n**How to
test:**\n1. Create a component template with a LogsDB index mode (you
can also\ntest with other index modes):\n```\nPUT
_component_template/my-component-template\n{\n \"template\": {\n
\"settings\": {\n \"index\": {\n \"mode\": \"logsdb\"\n }\n }\n
}\n}\n```\n2. Create an index template that is composed of the component
template\nabove:\n```\nPUT _index_template/my-index-template\n{\n
\"index_patterns\": [\n \"my-ds-*\"\n ],\n \"data_stream\": {},\n
\"composed_of\": [\n \"my-component-template\"\n ]\n}\n```\n3. Create a
data stream that matched the index pattern from the index\ntemplate
above:\n```\nPUT _data_stream/my-ds-1\n```\n4. Go to the data streams
table and verify that the index mode is\ndisplayed correctly in the
table.\n\n<img width=\"1165\" alt=\"Screenshot 2025-03-24 at 18 12
04\"\nsrc=\"https://github.com/user-attachments/assets/ea211c14-3d03-49c7-ace7-88b15e294d1f\"\n/>\n\n\n5.
Click on the created data stream and verify that the displayed
index\nmode in the details panel is correct:\n\n<img width=\"1165\"
alt=\"Screenshot 2025-03-06 at 14 36
12\"\nsrc=\"https://github.com/user-attachments/assets/954864e2-ae2a-4cb8-9eef-2c5f8b417f52\"\n/>\n\n---------\n\nCo-authored-by:
Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"dec6a17ec02ffea5669ffcf8431009aef1e99cfa"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Elena Stoeva <59341489+ElenaStoeva@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
akowalska622 pushed a commit to akowalska622/kibana that referenced this pull request May 29, 2025
Fixes elastic#208671

## Summary

Before this PR, the displayed index mode of the data streams was
determined based on the index mode of the associated index template.
However, the index mode can also be set through the component template,
so that logic is not reliable and can cause incorrectly displayed index
mode like described in elastic#208671.

In this PR, we replace this logic with the recently added `index_mode`
field to the Es Get Data Streams API (see
elastic/elasticsearch#122486).

**How to test:**
1. Create a component template with a LogsDB index mode (you can also
test with other index modes):
```
PUT _component_template/my-component-template
{
  "template": {
    "settings": {
      "index": {
        "mode": "logsdb"
      }
    }
  }
}
```
2. Create an index template that is composed of the component template
above:
```
PUT _index_template/my-index-template
{
  "index_patterns": [
    "my-ds-*"
  ],
  "data_stream": {},
  "composed_of": [
    "my-component-template"
  ]
}
```
3. Create a data stream that matched the index pattern from the index
template above:
```
PUT _data_stream/my-ds-1
```
4. Go to the data streams table and verify that the index mode is
displayed correctly in the table.

<img width="1165" alt="Screenshot 2025-03-24 at 18 12 04"
src="https://github.com/user-attachments/assets/ea211c14-3d03-49c7-ace7-88b15e294d1f"
/>


5. Click on the created data stream and verify that the displayed index
mode in the details panel is correct:

<img width="1165" alt="Screenshot 2025-03-06 at 14 36 12"
src="https://github.com/user-attachments/assets/954864e2-ae2a-4cb8-9eef-2c5f8b417f52"
/>

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
@ElenaStoeva ElenaStoeva deleted the data-streams/fix-index-mode-logic branch January 31, 2026 19:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:version Backport to applied version labels Feature:Index Management Index and index templates UI release_note:skip Skip the PR/issue when compiling release notes Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more t// v8.19.0 v9.1.0

5 participants