[Discover] Managed data view improvement#223451
Conversation
## Summary Ref: elastic/security-team#12791 Enable `newDataViewPickerEnabled` and fresh kibana build ### Before  ### After Label is cut off in security pages because of the `Managed` label, will address this in a separate PR (likely in #223451)   ### Checklist - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
## Summary Ref: elastic/security-team#12791 Enable `newDataViewPickerEnabled` and fresh kibana build ### Before  ### After Label is cut off in security pages because of the `Managed` label, will address this in a separate PR (likely in elastic#223451)   ### Checklist - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) (cherry picked from commit 26a14f3)
…) (#225174) # Backport This will backport the following commits from `main` to `8.19`: - [[Security Solution] Fix default data view name mismatch (#224333)](#224333) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"christineweng","email":"18648970+christineweng@users.noreply.github.com"},"sourceCommit":{"committedDate":"2025-06-24T21:05:52Z","message":"[Security Solution] Fix default data view name mismatch (#224333)\n\n## Summary\n\nRef: https://github.com/elastic/security-team/issues/12791\n\nEnable `newDataViewPickerEnabled` and fresh kibana build\n\n### Before\n\n\n\n\n### After\n\nLabel is cut off in security pages because of the `Managed` label, will\naddress this in a separate PR (likely in\nhttps://github.com//pull/223451)\n\n\n\n\n\n\n\n\n### Checklist\n\n- [x] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"26a14f36bf8208caa8836b54607514810378372b","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Threat Hunting:Investigations","backport:version","v9.1.0","v8.19.0"],"title":"[Security Solution] Fix default data view name mismatch","number":224333,"url":"https://github.com/elastic/kibana/pull/224333","mergeCommit":{"message":"[Security Solution] Fix default data view name mismatch (#224333)\n\n## Summary\n\nRef: https://github.com/elastic/security-team/issues/12791\n\nEnable `newDataViewPickerEnabled` and fresh kibana build\n\n### Before\n\n\n\n\n### After\n\nLabel is cut off in security pages because of the `Managed` label, will\naddress this in a separate PR (likely in\nhttps://github.com//pull/223451)\n\n\n\n\n\n\n\n\n### Checklist\n\n- [x] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"26a14f36bf8208caa8836b54607514810378372b"}},"sourceBranch":"main","suggestedTargetBranches":["8.19"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/224333","number":224333,"mergeCommit":{"message":"[Security Solution] Fix default data view name mismatch (#224333)\n\n## Summary\n\nRef: https://github.com/elastic/security-team/issues/12791\n\nEnable `newDataViewPickerEnabled` and fresh kibana build\n\n### Before\n\n\n\n\n### After\n\nLabel is cut off in security pages because of the `Managed` label, will\naddress this in a separate PR (likely in\nhttps://github.com//pull/223451)\n\n\n\n\n\n\n\n\n### Checklist\n\n- [x] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"26a14f36bf8208caa8836b54607514810378372b"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: christineweng <18648970+christineweng@users.noreply.github.com>
## Summary Ref: elastic/security-team#12791 Enable `newDataViewPickerEnabled` and fresh kibana build ### Before  ### After Label is cut off in security pages because of the `Managed` label, will address this in a separate PR (likely in elastic#223451)   ### Checklist - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
9aceec8 to
ddeaf14
Compare
...atform/plugins/shared/data_view_editor/public/components/data_view_editor_flyout_content.tsx
Outdated
Show resolved
Hide resolved
src/platform/plugins/shared/data_view_editor/public/components/footer/footer.tsx
Outdated
Show resolved
Hide resolved
src/platform/plugins/shared/data_view_editor/public/components/footer/footer.tsx
Outdated
Show resolved
Hide resolved
src/platform/plugins/shared/data_views/common/data_views/abstract_data_views.ts
Outdated
Show resolved
Hide resolved
| data_view: serviceKey === SERVICE_KEY ? dataViewSpecSchema : schema.never(), | ||
| index_pattern: | ||
| serviceKey === SERVICE_KEY_LEGACY ? dataViewSpecSchema : schema.never(), | ||
| managed: schema.maybe(schema.boolean({ defaultValue: false })), |
There was a problem hiding this comment.
@davismcphee who would we need to update on the docs side about documenting this change in a release note? Not sure what the process is for platform here..
There was a problem hiding this comment.
I don't think we should expose managed in the public API. Is it needed? I'm under the impression all internal requests run through the content management API that uses the saved object client under the hood.
There was a problem hiding this comment.
I'm not too familiar with internal/external API, is removing this line sufficient to keep it internal?
There was a problem hiding this comment.
Yep, should be. Sorry for the delayed response here!
There was a problem hiding this comment.
I think it should be, and maybe removing from the schema in src/platform/plugins/shared/data_views/server/rest_api_routes/schema.ts too.
...s/shared/discover/public/application/main/components/sidebar/discover_sidebar_responsive.tsx
Show resolved
Hide resolved
| const savedDataViewRefs = savedDataViews | ||
| const availableDataViewRefs = savedDataViews | ||
| ? savedDataViews | ||
| : (await data.dataViews.getIdsWithTitle()) ?? []; |
There was a problem hiding this comment.
Not really on you here, but should this be wrapped in a try catch? fyi @davismcphee
There was a problem hiding this comment.
More of a question for the Presentation team as Unified Search owners, but it definitely could be. Even something like data.dataViews.getIdsWithTitle().catch(() => []) would stop things from blowing up, although I guess it must not fail much as is or we'd probably hear about it 😄
src/platform/plugins/shared/unified_search/public/dataview_picker/change_dataview.tsx
Show resolved
Hide resolved
src/platform/plugins/shared/unified_search/public/dataview_picker/change_dataview.tsx
Outdated
Show resolved
Hide resolved
src/platform/plugins/shared/unified_search/public/dataview_picker/change_dataview.tsx
Outdated
Show resolved
Hide resolved
davismcphee
left a comment
There was a problem hiding this comment.
Great job! The code changes are looking good overall. I left some questions and feedback in addition to what @michaelolo24 already added, but it seems like we're on the right track. I have a few comments around the UX, but I'll save them for the One Discover sync so we can discuss.
A couple of things I noticed when testing locally:
A user is still able to implicitly modify a managed data view through the field editor flyout and persist the changes. I think for this case, we can probably just do a check for managed and completely disable the ability to save, with a tooltip along the lines of Fields cannot be edited on managed data views. Duplicate the data view in order to make changes.:

It looks like managed data views can still be edited from data view management. This might be related to the isEdit thing added in this PR. We may need to figure out a way to handle this without each data view editor consumer having to explicitly pass isEdit since it can be error prone if missed:

src/platform/plugins/shared/data_views/common/data_views/abstract_data_views.ts
Outdated
Show resolved
Hide resolved
src/platform/plugins/shared/data_views/common/data_views/data_views.ts
Outdated
Show resolved
Hide resolved
src/platform/plugins/shared/unified_search/public/dataview_picker/change_dataview.tsx
Outdated
Show resolved
Hide resolved
src/platform/plugins/shared/unified_search/public/dataview_picker/change_dataview.tsx
Outdated
Show resolved
Hide resolved
| const savedDataViewRefs = savedDataViews | ||
| const availableDataViewRefs = savedDataViews | ||
| ? savedDataViews | ||
| : (await data.dataViews.getIdsWithTitle()) ?? []; |
There was a problem hiding this comment.
More of a question for the Presentation team as Unified Search owners, but it definitely could be. Even something like data.dataViews.getIdsWithTitle().catch(() => []) would stop things from blowing up, although I guess it must not fail much as is or we'd probably hear about it 😄
src/platform/plugins/shared/unified_search/public/dataview_picker/change_dataview.tsx
Show resolved
Hide resolved
| name: 'All logs', | ||
| title: 'logs-*,dataset-logs-*-*', | ||
| timeFieldName: '@timestamp', | ||
| managed: true, |
There was a problem hiding this comment.
On a totally unrelated note since I just happened to notice it, @elastic/obs-ux-logs-team shouldn't this just be { id: ALL_LOGS_DATA_VIEW_ID } like it is here?
There was a problem hiding this comment.
@thomheymann @weltenwort is this a leftover from #209565? Or we need this for redirects?
There was a problem hiding this comment.
@davismcphee Yeh, looks like All Logs is used as a fallback which should be using the ALL_LOGS_DATA_VIEW_ID data view.
@gbamparop This code was introduced as part of #215867 to pass app state from Logs Stream component to Discover.
There was a problem hiding this comment.
Agreed. Although I suspect this is a dead code path since the conditional earlier in that function already seems to cover all cases and we'd never reach that fall-through. For cleanliness's sake one could turn the 'all' branch into the default to remove this.
ee414c0 to
1a3bbd8
Compare
1a3bbd8 to
f4a8630
Compare
### Summary This PR adds a `managed` property to `DataView` type. It leveraged the existing `managed` property in saved objects, when a data view is created, `managed` is also passed to SO creation. Major changes: - Kibana managed data views (from integrations, discover and security) now have a `Managed` tag. The tags are present consistently across applications - When viewing an existing data view, user is able to duplicate the data view via the new `Duplicate` button - When clicking `Managed this data view` for a managed data view, the form is disabled from editing. User has the option to click `Duplicate`, which will open a new flyout with the indices populated - New fields cannot be saved in a managed data view - In `Stack management` -> `Data views`, editing a managed data view also shows a disabled form. `Managed` tag is added. - Out of the scope of this PR: `Duplicate` is not available in stack management -> data views -> edit. If the data view is managed, the form is disabled and save button is not present https://github.com/user-attachments/assets/16401792-489f-462b-9e4e-b20a74e2bfaa ### Data view editor flyout logic As the variation of the editor form increase from 2 scenarios to 4, additional logic is added. A small refactoring was made to `onDataViewCreated` to standardized the props, so that it can be reused by the duplicate logic (when duplicating, it is essentially the same behavior as creating a data view. <img width="1086" height="345" alt="image" src="https://github.com/user-attachments/assets/b4ccfd17-6745-4c2a-89f7-06500bddb365" /> ### UI updates <img width="1436" height="718" alt="image" src="https://github.com/user-attachments/assets/70026a65-aa91-4cb4-96b5-5be6798bc26b" /> <img width="1494" height="865" alt="image" src="https://github.com/user-attachments/assets/bd4a9125-1aba-476b-b459-fb09c4a08be3" /> <img width="1144" height="590" alt="image" src="https://github.com/user-attachments/assets/3e647288-bb21-439d-8672-81ee21b86523" /> ### Checklist - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Davis McPhee <davis.mcphee@elastic.co>
## Summary When testing the managed dataViews pr here: github.com//pull/223451 @christineweng made a great catch that while caching dataViews based on the index pattern may seem beneficial to avoid caching the same set of fields over and over again, two dataViews can share the same index pattern list, but have a different field list based on any runtime fields set on the dataView instance. Given this, the safest approach is to just rely on the `DataViewId` to cache the browser fields. Expectations / Concessions: 1. AdHoc dataViews will not be cached 2. The same index pattern list for two different dataViews will now cache two different browserField objects 3. To not overload the browser memory, we will set a cacge size limit of 10.
…ic#234381) ## Summary When testing the managed dataViews pr here: github.com/elastic/pull/223451 @christineweng made a great catch that while caching dataViews based on the index pattern may seem beneficial to avoid caching the same set of fields over and over again, two dataViews can share the same index pattern list, but have a different field list based on any runtime fields set on the dataView instance. Given this, the safest approach is to just rely on the `DataViewId` to cache the browser fields. Expectations / Concessions: 1. AdHoc dataViews will not be cached 2. The same index pattern list for two different dataViews will now cache two different browserField objects 3. To not overload the browser memory, we will set a cacge size limit of 10. (cherry picked from commit 9d9e643) # Conflicts: # x-pack/solutions/security/plugins/security_solution/public/cases/components/case_events/table.tsx # x-pack/solutions/security/plugins/security_solution/public/data_view_manager/components/data_view_picker/index.tsx # x-pack/solutions/security/plugins/security_solution/public/data_view_manager/hooks/use_browser_fields.ts # x-pack/solutions/security/plugins/security_solution/public/data_view_manager/utils/security_browser_fields_manager.ts # x-pack/solutions/security/plugins/security_solution/public/timelines/components/fields_browser/index.tsx
…ic#234381) ## Summary When testing the managed dataViews pr here: github.com/elastic/pull/223451 @christineweng made a great catch that while caching dataViews based on the index pattern may seem beneficial to avoid caching the same set of fields over and over again, two dataViews can share the same index pattern list, but have a different field list based on any runtime fields set on the dataView instance. Given this, the safest approach is to just rely on the `DataViewId` to cache the browser fields. Expectations / Concessions: 1. AdHoc dataViews will not be cached 2. The same index pattern list for two different dataViews will now cache two different browserField objects 3. To not overload the browser memory, we will set a cacge size limit of 10.
…ic#234381) ## Summary When testing the managed dataViews pr here: github.com/elastic/pull/223451 @christineweng made a great catch that while caching dataViews based on the index pattern may seem beneficial to avoid caching the same set of fields over and over again, two dataViews can share the same index pattern list, but have a different field list based on any runtime fields set on the dataView instance. Given this, the safest approach is to just rely on the `DataViewId` to cache the browser fields. Expectations / Concessions: 1. AdHoc dataViews will not be cached 2. The same index pattern list for two different dataViews will now cache two different browserField objects 3. To not overload the browser memory, we will set a cacge size limit of 10. (cherry picked from commit 9d9e643) # Conflicts: # x-pack/solutions/security/plugins/security_solution/public/cases/components/case_events/table.tsx # x-pack/solutions/security/plugins/security_solution/public/data_view_manager/components/data_view_picker/index.tsx # x-pack/solutions/security/plugins/security_solution/public/data_view_manager/hooks/use_browser_fields.ts # x-pack/solutions/security/plugins/security_solution/public/data_view_manager/utils/security_browser_fields_manager.ts # x-pack/solutions/security/plugins/security_solution/public/timelines/components/fields_browser/index.tsx
…ic#234381) ## Summary When testing the managed dataViews pr here: github.com/elastic/pull/223451 @christineweng made a great catch that while caching dataViews based on the index pattern may seem beneficial to avoid caching the same set of fields over and over again, two dataViews can share the same index pattern list, but have a different field list based on any runtime fields set on the dataView instance. Given this, the safest approach is to just rely on the `DataViewId` to cache the browser fields. Expectations / Concessions: 1. AdHoc dataViews will not be cached 2. The same index pattern list for two different dataViews will now cache two different browserField objects 3. To not overload the browser memory, we will set a cacge size limit of 10. (cherry picked from commit 9d9e643) # Conflicts: # x-pack/solutions/security/plugins/security_solution/public/cases/components/case_events/table.tsx # x-pack/solutions/security/plugins/security_solution/public/data_view_manager/components/data_view_picker/index.tsx # x-pack/solutions/security/plugins/security_solution/public/data_view_manager/hooks/use_browser_fields.ts # x-pack/solutions/security/plugins/security_solution/public/data_view_manager/utils/security_browser_fields_manager.ts # x-pack/solutions/security/plugins/security_solution/public/timelines/components/fields_browser/index.tsx
…ic#234381) ## Summary When testing the managed dataViews pr here: github.com/elastic/pull/223451 @christineweng made a great catch that while caching dataViews based on the index pattern may seem beneficial to avoid caching the same set of fields over and over again, two dataViews can share the same index pattern list, but have a different field list based on any runtime fields set on the dataView instance. Given this, the safest approach is to just rely on the `DataViewId` to cache the browser fields. Expectations / Concessions: 1. AdHoc dataViews will not be cached 2. The same index pattern list for two different dataViews will now cache two different browserField objects 3. To not overload the browser memory, we will set a cacge size limit of 10.
…ic#234381) ## Summary When testing the managed dataViews pr here: github.com/elastic/pull/223451 @christineweng made a great catch that while caching dataViews based on the index pattern may seem beneficial to avoid caching the same set of fields over and over again, two dataViews can share the same index pattern list, but have a different field list based on any runtime fields set on the dataView instance. Given this, the safest approach is to just rely on the `DataViewId` to cache the browser fields. Expectations / Concessions: 1. AdHoc dataViews will not be cached 2. The same index pattern list for two different dataViews will now cache two different browserField objects 3. To not overload the browser memory, we will set a cacge size limit of 10.
…ic#234381) ## Summary When testing the managed dataViews pr here: github.com/elastic/pull/223451 @christineweng made a great catch that while caching dataViews based on the index pattern may seem beneficial to avoid caching the same set of fields over and over again, two dataViews can share the same index pattern list, but have a different field list based on any runtime fields set on the dataView instance. Given this, the safest approach is to just rely on the `DataViewId` to cache the browser fields. Expectations / Concessions: 1. AdHoc dataViews will not be cached 2. The same index pattern list for two different dataViews will now cache two different browserField objects 3. To not overload the browser memory, we will set a cacge size limit of 10.
### Summary This PR adds a `managed` property to `DataView` type. It leveraged the existing `managed` property in saved objects, when a data view is created, `managed` is also passed to SO creation. Major changes: - Kibana managed data views (from integrations, discover and security) now have a `Managed` tag. The tags are present consistently across applications - When viewing an existing data view, user is able to duplicate the data view via the new `Duplicate` button - When clicking `Managed this data view` for a managed data view, the form is disabled from editing. User has the option to click `Duplicate`, which will open a new flyout with the indices populated - New fields cannot be saved in a managed data view - In `Stack management` -> `Data views`, editing a managed data view also shows a disabled form. `Managed` tag is added. - Out of the scope of this PR: `Duplicate` is not available in stack management -> data views -> edit. If the data view is managed, the form is disabled and save button is not present https://github.com/user-attachments/assets/16401792-489f-462b-9e4e-b20a74e2bfaa ### Data view editor flyout logic As the variation of the editor form increase from 2 scenarios to 4, additional logic is added. A small refactoring was made to `onDataViewCreated` to standardized the props, so that it can be reused by the duplicate logic (when duplicating, it is essentially the same behavior as creating a data view. <img width="1086" height="345" alt="image" src="https://github.com/user-attachments/assets/b4ccfd17-6745-4c2a-89f7-06500bddb365" /> ### UI updates <img width="1436" height="718" alt="image" src="https://github.com/user-attachments/assets/70026a65-aa91-4cb4-96b5-5be6798bc26b" /> <img width="1494" height="865" alt="image" src="https://github.com/user-attachments/assets/bd4a9125-1aba-476b-b459-fb09c4a08be3" /> <img width="1144" height="590" alt="image" src="https://github.com/user-attachments/assets/3e647288-bb21-439d-8672-81ee21b86523" /> ### Checklist - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Davis McPhee <davis.mcphee@elastic.co>
## Summary When testing the managed dataViews pr here: github.com//pull/223451 @christineweng made a great catch that while caching dataViews based on the index pattern may seem beneficial to avoid caching the same set of fields over and over again, two dataViews can share the same index pattern list, but have a different field list based on any runtime fields set on the dataView instance. Given this, the safest approach is to just rely on the `DataViewId` to cache the browser fields. Expectations / Concessions: 1. AdHoc dataViews will not be cached 2. The same index pattern list for two different dataViews will now cache two different browserField objects 3. To not overload the browser memory, we will set a cacge size limit of 10.
…234381) (#236482) # Backport This will backport the following commits from `main` to `9.1`: - [[Security Solution][Investigations] - Fix browser fields cache (#234381)](#234381) <!--- Backport version: 10.0.0 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Michael Olorunnisola","email":"michael.olorunnisola@elastic.co"},"sourceCommit":{"committedDate":"2025-09-25T14:24:18Z","message":"[Security Solution][Investigations] - Fix browser fields cache (#234381)\n\n## Summary\n\nWhen testing the managed dataViews pr here:\ngithub.com//pull/223451 @christineweng made a great catch\nthat while caching dataViews based on the index pattern may seem\nbeneficial to avoid caching the same set of fields over and over again,\ntwo dataViews can share the same index pattern list, but have a\ndifferent field list based on any runtime fields set on the dataView\ninstance. Given this, the safest approach is to just rely on the\n`DataViewId` to cache the browser fields.\n\nExpectations / Concessions:\n\n1. AdHoc dataViews will not be cached\n2. The same index pattern list for two different dataViews will now\ncache two different browserField objects\n3. To not overload the browser memory, we will set a cacge size limit of\n10.","sha":"9d9e643ad08b86549883c538fe6f13b2562019e9","branchLabelMapping":{"^v9.2.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["bug","release_note:fix","Team:Threat Hunting:Investigations","ci:cloud-deploy","ci:project-deploy-security","backport:version","v9.2.0","v8.19.5","v9.1.5"],"title":"[Security Solution][Investigations] - Fix browser fields cache","number":234381,"url":"https://github.com/elastic/kibana/pull/234381","mergeCommit":{"message":"[Security Solution][Investigations] - Fix browser fields cache (#234381)\n\n## Summary\n\nWhen testing the managed dataViews pr here:\ngithub.com//pull/223451 @christineweng made a great catch\nthat while caching dataViews based on the index pattern may seem\nbeneficial to avoid caching the same set of fields over and over again,\ntwo dataViews can share the same index pattern list, but have a\ndifferent field list based on any runtime fields set on the dataView\ninstance. Given this, the safest approach is to just rely on the\n`DataViewId` to cache the browser fields.\n\nExpectations / Concessions:\n\n1. AdHoc dataViews will not be cached\n2. The same index pattern list for two different dataViews will now\ncache two different browserField objects\n3. To not overload the browser memory, we will set a cacge size limit of\n10.","sha":"9d9e643ad08b86549883c538fe6f13b2562019e9"}},"sourceBranch":"main","suggestedTargetBranches":["8.19","9.1"],"targetPullRequestStates":[{"branch":"main","label":"v9.2.0","branchLabelMappingKey":"^v9.2.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/234381","number":234381,"mergeCommit":{"message":"[Security Solution][Investigations] - Fix browser fields cache (#234381)\n\n## Summary\n\nWhen testing the managed dataViews pr here:\ngithub.com//pull/223451 @christineweng made a great catch\nthat while caching dataViews based on the index pattern may seem\nbeneficial to avoid caching the same set of fields over and over again,\ntwo dataViews can share the same index pattern list, but have a\ndifferent field list based on any runtime fields set on the dataView\ninstance. Given this, the safest approach is to just rely on the\n`DataViewId` to cache the browser fields.\n\nExpectations / Concessions:\n\n1. AdHoc dataViews will not be cached\n2. The same index pattern list for two different dataViews will now\ncache two different browserField objects\n3. To not overload the browser memory, we will set a cacge size limit of\n10.","sha":"9d9e643ad08b86549883c538fe6f13b2562019e9"}},{"branch":"8.19","label":"v8.19.5","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"9.1","label":"v9.1.5","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> --------- Co-authored-by: Philippe Oberti <philippe.oberti@elastic.co>
…#234381) (#236484) # Backport This will backport the following commits from `main` to `8.19`: - [[Security Solution][Investigations] - Fix browser fields cache (#234381)](#234381) <!--- Backport version: 10.0.0 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Michael Olorunnisola","email":"michael.olorunnisola@elastic.co"},"sourceCommit":{"committedDate":"2025-09-25T14:24:18Z","message":"[Security Solution][Investigations] - Fix browser fields cache (#234381)\n\n## Summary\n\nWhen testing the managed dataViews pr here:\ngithub.com//pull/223451 @christineweng made a great catch\nthat while caching dataViews based on the index pattern may seem\nbeneficial to avoid caching the same set of fields over and over again,\ntwo dataViews can share the same index pattern list, but have a\ndifferent field list based on any runtime fields set on the dataView\ninstance. Given this, the safest approach is to just rely on the\n`DataViewId` to cache the browser fields.\n\nExpectations / Concessions:\n\n1. AdHoc dataViews will not be cached\n2. The same index pattern list for two different dataViews will now\ncache two different browserField objects\n3. To not overload the browser memory, we will set a cacge size limit of\n10.","sha":"9d9e643ad08b86549883c538fe6f13b2562019e9","branchLabelMapping":{"^v9.2.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["bug","release_note:fix","Team:Threat Hunting:Investigations","ci:cloud-deploy","ci:project-deploy-security","backport:version","v9.2.0","v8.19.5","v9.1.5"],"title":"[Security Solution][Investigations] - Fix browser fields cache","number":234381,"url":"https://github.com/elastic/kibana/pull/234381","mergeCommit":{"message":"[Security Solution][Investigations] - Fix browser fields cache (#234381)\n\n## Summary\n\nWhen testing the managed dataViews pr here:\ngithub.com//pull/223451 @christineweng made a great catch\nthat while caching dataViews based on the index pattern may seem\nbeneficial to avoid caching the same set of fields over and over again,\ntwo dataViews can share the same index pattern list, but have a\ndifferent field list based on any runtime fields set on the dataView\ninstance. Given this, the safest approach is to just rely on the\n`DataViewId` to cache the browser fields.\n\nExpectations / Concessions:\n\n1. AdHoc dataViews will not be cached\n2. The same index pattern list for two different dataViews will now\ncache two different browserField objects\n3. To not overload the browser memory, we will set a cacge size limit of\n10.","sha":"9d9e643ad08b86549883c538fe6f13b2562019e9"}},"sourceBranch":"main","suggestedTargetBranches":["8.19","9.1"],"targetPullRequestStates":[{"branch":"main","label":"v9.2.0","branchLabelMappingKey":"^v9.2.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/234381","number":234381,"mergeCommit":{"message":"[Security Solution][Investigations] - Fix browser fields cache (#234381)\n\n## Summary\n\nWhen testing the managed dataViews pr here:\ngithub.com//pull/223451 @christineweng made a great catch\nthat while caching dataViews based on the index pattern may seem\nbeneficial to avoid caching the same set of fields over and over again,\ntwo dataViews can share the same index pattern list, but have a\ndifferent field list based on any runtime fields set on the dataView\ninstance. Given this, the safest approach is to just rely on the\n`DataViewId` to cache the browser fields.\n\nExpectations / Concessions:\n\n1. AdHoc dataViews will not be cached\n2. The same index pattern list for two different dataViews will now\ncache two different browserField objects\n3. To not overload the browser memory, we will set a cacge size limit of\n10.","sha":"9d9e643ad08b86549883c538fe6f13b2562019e9"}},{"branch":"8.19","label":"v8.19.5","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"9.1","label":"v9.1.5","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> --------- Co-authored-by: PhilippeOberti <philippe.oberti@elastic.co>
This PR: - adds mention of "managed" data views to the permissions section of the the data views page - adds instructions for duplicating a managed data view, since the duplicate option isn't available from everywhere in Kibana Closes: elastic/kibana#223451 @christineweng @davismcphee let me know if you think this is clear enough. I don't think it's super valuable to mention this in every app's documentation given that the UI is pretty self-explanatory in this case, but if you think otherwise, please let me know. --------- Co-authored-by: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com>
### Summary This PR adds a `managed` property to `DataView` type. It leveraged the existing `managed` property in saved objects, when a data view is created, `managed` is also passed to SO creation. Major changes: - Kibana managed data views (from integrations, discover and security) now have a `Managed` tag. The tags are present consistently across applications - When viewing an existing data view, user is able to duplicate the data view via the new `Duplicate` button - When clicking `Managed this data view` for a managed data view, the form is disabled from editing. User has the option to click `Duplicate`, which will open a new flyout with the indices populated - New fields cannot be saved in a managed data view - In `Stack management` -> `Data views`, editing a managed data view also shows a disabled form. `Managed` tag is added. - Out of the scope of this PR: `Duplicate` is not available in stack management -> data views -> edit. If the data view is managed, the form is disabled and save button is not present https://github.com/user-attachments/assets/16401792-489f-462b-9e4e-b20a74e2bfaa ### Data view editor flyout logic As the variation of the editor form increase from 2 scenarios to 4, additional logic is added. A small refactoring was made to `onDataViewCreated` to standardized the props, so that it can be reused by the duplicate logic (when duplicating, it is essentially the same behavior as creating a data view. <img width="1086" height="345" alt="image" src="https://github.com/user-attachments/assets/b4ccfd17-6745-4c2a-89f7-06500bddb365" /> ### UI updates <img width="1436" height="718" alt="image" src="https://github.com/user-attachments/assets/70026a65-aa91-4cb4-96b5-5be6798bc26b" /> <img width="1494" height="865" alt="image" src="https://github.com/user-attachments/assets/bd4a9125-1aba-476b-b459-fb09c4a08be3" /> <img width="1144" height="590" alt="image" src="https://github.com/user-attachments/assets/3e647288-bb21-439d-8672-81ee21b86523" /> ### Checklist - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Davis McPhee <davis.mcphee@elastic.co>
…ic#234381) ## Summary When testing the managed dataViews pr here: github.com/elastic/pull/223451 @christineweng made a great catch that while caching dataViews based on the index pattern may seem beneficial to avoid caching the same set of fields over and over again, two dataViews can share the same index pattern list, but have a different field list based on any runtime fields set on the dataView instance. Given this, the safest approach is to just rely on the `DataViewId` to cache the browser fields. Expectations / Concessions: 1. AdHoc dataViews will not be cached 2. The same index pattern list for two different dataViews will now cache two different browserField objects 3. To not overload the browser memory, we will set a cacge size limit of 10.
…3370) This PR: - adds mention of "managed" data views to the permissions section of the the data views page - adds instructions for duplicating a managed data view, since the duplicate option isn't available from everywhere in Kibana Closes: elastic/kibana#223451 @christineweng @davismcphee let me know if you think this is clear enough. I don't think it's super valuable to mention this in every app's documentation given that the UI is pretty self-explanatory in this case, but if you think otherwise, please let me know. --------- Co-authored-by: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com>
## Summary Fixes validation error when creating custom threshold rules with data view objects by adding the missing `managed` field to `dataViewSpecSchema`. ## Problem When creating a custom threshold rule with a data view object in `searchConfiguration.index`, the validation was failing with: `params invalid: [searchConfiguration.index]: types that failed validation: [searchConfiguration.index.0]: expected value of type [string] but got [Object] [searchConfiguration.index.1.managed]: definition for this key is missing` The DataView's `toSpec()` method includes a `managed` field (added in #223451), but this field was not part of the `dataViewSpecSchema` validation, causing rule creation to fail. ## Solution Added the `managed` field as an optional boolean to `dataViewSpecSchema` in `@kbn/response-ops-rule-params`. `managed: schema.maybe(schema.boolean())` This approach explicitly defines the expected field rather than ignoring unknown properties, which: - Maintains strict validation (typos/misconfigured rules will still fail) - Avoids the breaking change implications of toggling `unknowns: 'ignore'` Before: https://github.com/user-attachments/assets/7ad0fd02-9dc6-42fe-b90c-bf87d73cca9d After: https://github.com/user-attachments/assets/385eb375-ac9d-471c-8a2f-118f1e56dd74 --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary Fixes validation error when creating custom threshold rules with data view objects by adding the missing `managed` field to `dataViewSpecSchema`. ## Problem When creating a custom threshold rule with a data view object in `searchConfiguration.index`, the validation was failing with: `params invalid: [searchConfiguration.index]: types that failed validation: [searchConfiguration.index.0]: expected value of type [string] but got [Object] [searchConfiguration.index.1.managed]: definition for this key is missing` The DataView's `toSpec()` method includes a `managed` field (added in elastic#223451), but this field was not part of the `dataViewSpecSchema` validation, causing rule creation to fail. ## Solution Added the `managed` field as an optional boolean to `dataViewSpecSchema` in `@kbn/response-ops-rule-params`. `managed: schema.maybe(schema.boolean())` This approach explicitly defines the expected field rather than ignoring unknown properties, which: - Maintains strict validation (typos/misconfigured rules will still fail) - Avoids the breaking change implications of toggling `unknowns: 'ignore'` Before: https://github.com/user-attachments/assets/7ad0fd02-9dc6-42fe-b90c-bf87d73cca9d After: https://github.com/user-attachments/assets/385eb375-ac9d-471c-8a2f-118f1e56dd74 --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> (cherry picked from commit f6abc44)
## Summary Fixes validation error when creating custom threshold rules with data view objects by adding the missing `managed` field to `dataViewSpecSchema`. ## Problem When creating a custom threshold rule with a data view object in `searchConfiguration.index`, the validation was failing with: `params invalid: [searchConfiguration.index]: types that failed validation: [searchConfiguration.index.0]: expected value of type [string] but got [Object] [searchConfiguration.index.1.managed]: definition for this key is missing` The DataView's `toSpec()` method includes a `managed` field (added in elastic#223451), but this field was not part of the `dataViewSpecSchema` validation, causing rule creation to fail. ## Solution Added the `managed` field as an optional boolean to `dataViewSpecSchema` in `@kbn/response-ops-rule-params`. `managed: schema.maybe(schema.boolean())` This approach explicitly defines the expected field rather than ignoring unknown properties, which: - Maintains strict validation (typos/misconfigured rules will still fail) - Avoids the breaking change implications of toggling `unknowns: 'ignore'` Before: https://github.com/user-attachments/assets/7ad0fd02-9dc6-42fe-b90c-bf87d73cca9d After: https://github.com/user-attachments/assets/385eb375-ac9d-471c-8a2f-118f1e56dd74 --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary Fixes validation error when creating custom threshold rules with data view objects by adding the missing `managed` field to `dataViewSpecSchema`. ## Problem When creating a custom threshold rule with a data view object in `searchConfiguration.index`, the validation was failing with: `params invalid: [searchConfiguration.index]: types that failed validation: [searchConfiguration.index.0]: expected value of type [string] but got [Object] [searchConfiguration.index.1.managed]: definition for this key is missing` The DataView's `toSpec()` method includes a `managed` field (added in elastic#223451), but this field was not part of the `dataViewSpecSchema` validation, causing rule creation to fail. ## Solution Added the `managed` field as an optional boolean to `dataViewSpecSchema` in `@kbn/response-ops-rule-params`. `managed: schema.maybe(schema.boolean())` This approach explicitly defines the expected field rather than ignoring unknown properties, which: - Maintains strict validation (typos/misconfigured rules will still fail) - Avoids the breaking change implications of toggling `unknowns: 'ignore'` Before: https://github.com/user-attachments/assets/7ad0fd02-9dc6-42fe-b90c-bf87d73cca9d After: https://github.com/user-attachments/assets/385eb375-ac9d-471c-8a2f-118f1e56dd74 --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary Fixes validation error when creating custom threshold rules with data view objects by adding the missing `managed` field to `dataViewSpecSchema`. ## Problem When creating a custom threshold rule with a data view object in `searchConfiguration.index`, the validation was failing with: `params invalid: [searchConfiguration.index]: types that failed validation: [searchConfiguration.index.0]: expected value of type [string] but got [Object] [searchConfiguration.index.1.managed]: definition for this key is missing` The DataView's `toSpec()` method includes a `managed` field (added in elastic#223451), but this field was not part of the `dataViewSpecSchema` validation, causing rule creation to fail. ## Solution Added the `managed` field as an optional boolean to `dataViewSpecSchema` in `@kbn/response-ops-rule-params`. `managed: schema.maybe(schema.boolean())` This approach explicitly defines the expected field rather than ignoring unknown properties, which: - Maintains strict validation (typos/misconfigured rules will still fail) - Avoids the breaking change implications of toggling `unknowns: 'ignore'` Before: https://github.com/user-attachments/assets/7ad0fd02-9dc6-42fe-b90c-bf87d73cca9d After: https://github.com/user-attachments/assets/385eb375-ac9d-471c-8a2f-118f1e56dd74 --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Summary
This PR adds a
managedproperty toDataViewtype. It leveraged the existingmanagedproperty in saved objects, when a data view is created,managedis also passed to SO creation.Major changes:
Managedtag. The tags are present consistently across applicationsDuplicatebuttonManaged this data viewfor a managed data view, the form is disabled from editing. User has the option to clickDuplicate, which will open a new flyout with the indices populatedStack management->Data views, editing a managed data view also shows a disabled form.Managedtag is added.Duplicateis not available in stack management -> data views -> edit. If the data view is managed, the form is disabled and save button is not presentScreen.Recording.2025-09-04.at.2.18.03.PM.mov
Data view editor flyout logic
As the variation of the editor form increase from 2 scenarios to 4, additional logic is added. A small refactoring was made to
onDataViewCreatedto standardized the props, so that it can be reused by the duplicate logic (when duplicating, it is essentially the same behavior as creating a data view.UI updates
Checklist
release_note:breakinglabel should be applied in these situations.release_note:*label is applied per the guidelines