[SLO] Fix dashboard filters not applied to SLO embeddable for grouping fields#255746
Conversation
…g fields Dashboard-level filters using fields like `orchestrator.cluster.name` were silently dropped by the SLO embeddable because: 1. The SLO overview embeddable did not subscribe to parent dashboard filters via `useFetchContext`, so global filters were never forwarded. 2. The SLO summary index stores grouping values under a flattened `slo.groupings.*` namespace, but `buildQueryFromFilters` with `ignoreFilterIfFieldNotInIndex: true` discarded filters whose field names didn't match the summary index mapping. This commit fixes both issues: - Merge dashboard filters with embeddable-local filters in the SLO overview factory using `useFetchContext`. - Rewrite non-native filter field names to their `slo.groupings.*` equivalents on the client side before calling `buildQueryFromFilters`, and flip `ignoreFilterIfFieldNotInIndex` to `false` so rewritten filters are always included in the ES query. - Extract summary mapping properties into a shared `common/` module so the rewrite logic derives native fields from the actual mapping definition rather than a hardcoded list. - Remove the now-unnecessary server-side query DSL rewriting from `parseStringFilters`. - Add unit tests for filter rewriting and integration test for the overview endpoint with cluster filters. Closes elastic#198289 Made-with: Cursor
|
Pinging @elastic/actionable-obs-team (Team:actionable-obs) |
|
|
||
| export const SUMMARY_MAPPINGS_TEMPLATE: ClusterPutComponentTemplateRequest = { | ||
| name: SUMMARY_COMPONENT_TEMPLATE_MAPPINGS_NAME, | ||
| template: { | ||
| mappings: { | ||
| properties: { |
There was a problem hiding this comment.
The runtime values of the original inline properties object and SUMMARY_MAPPING_PROPERTIES produce the exact same JSON
There was a problem hiding this comment.
not ideal to move this out the server. kind of scared someone will update the summary_mapping_properties.ts later on. Do we need to export this to common? On what fields do we expect to do filtering on?
| import { SUMMARY_MAPPING_PROPERTIES } from './summary_mapping_properties'; | ||
|
|
||
| const SLO_GROUPINGS_PREFIX = 'slo.groupings.'; | ||
|
|
||
| const SUMMARY_TOP_LEVEL_KEYS = Object.keys(SUMMARY_MAPPING_PROPERTIES); | ||
| const SUMMARY_NATIVE_FIELDS = new Set(SUMMARY_TOP_LEVEL_KEYS); | ||
| const SUMMARY_NATIVE_PREFIXES = SUMMARY_TOP_LEVEL_KEYS.filter((key) => { | ||
| const val = (SUMMARY_MAPPING_PROPERTIES as Record<string, unknown>)[key]; | ||
| return val != null && typeof val === 'object' && 'properties' in val; | ||
| }).map((key) => `${key}.`); |
There was a problem hiding this comment.
I think we should not move the mappings definition.
Can we introduce an allow list of fields that we consider useful, e.g. slo.id, slo.tags, slo.groupings.*, service.*, transaction.*, monitor.*, observer.* ?
I don't think others fields are that useful to filter upon
|
I'll try to review and test this properly this week |
| const mergedFilters = useMemo( | ||
| () => [...(groupFilters?.filters ?? []), ...(fetchContext.filters ?? [])], | ||
| [groupFilters?.filters, fetchContext.filters] | ||
| ); |
There was a problem hiding this comment.
Do you think we could have a rewrite_filter here that transforms the fetchContext.filters by adding the slo.grouping. prefix to every fields? We wouldn't need to make any change in the existing hooks by doing that?
…luster-filter-rewriting Made-with: Cursor # Conflicts: # x-pack/solutions/observability/plugins/slo/public/embeddable/slo/overview/slo_embeddable_factory.tsx
| const mergedFilters = useMemo( | ||
| () => [ | ||
| ...(toStoredFilters(groupFilters?.filters) ?? []), | ||
| ...rewriteFiltersForSloSummary(fetchContext.filters ?? []), |
There was a problem hiding this comment.
Thanks for making this change, it feels much better to handle this only here and leak the transformation inside the hooks used in other places than the embeddable 👍🏻
| if (!field || field.startsWith(SLO_GROUPINGS_PREFIX) || isSummaryNativeField(field)) { | ||
| return filter; | ||
| } |
There was a problem hiding this comment.
I guess SLO_GROUPINGS_PREFIX can be part of SUMMARY_NATIVE_PREFIXES and checked by isSummaryNativeField?
💛 Build succeeded, but was flaky
Failed CI StepsTest Failures
Metrics [docs]Module Count
Async chunks
Page load bundle
History
cc @fkanout |
| .set(adminRoleAuthc.apiKeyHeader) | ||
| .set(internalHeaders) | ||
| .send() | ||
| .expect(200); |
There was a problem hiding this comment.
Interesting that we are testing this API, did the LLM hallucinate the API used behind the overview embeddable? overview embeddable -> /overview API? 😅
I mean it's not bad to test this API (it is used for the stats shown on the listing page), but it's unrelated to your change :)
There was a problem hiding this comment.
“LLM hallucination”? Maybe 😆 My guess is it came from my prompt about tests, since I usually ask for broad coverage, starting with edge cases.
| expect(result.query).toEqual({ term: { 'service.name': 'my-svc' } }); | ||
| }); | ||
|
|
||
| it('leaves native prefixed fields unchanged', () => { |
kdelemme
left a comment
There was a problem hiding this comment.
Looking good, thanks for the changes. One more thing about adding slo.groupings into the SUMMARY_NATIVE_PREFIXES so we can use only isSummaryNativeField...
And the integration test is somewhat irrelevant to the codepath changed but still we can keep it :)
I'm going to test it locally now
kdelemme
left a comment
There was a problem hiding this comment.
Approving to not block you further
…luster-filter-rewriting Made-with: Cursor # Conflicts: # x-pack/solutions/observability/plugins/slo/public/embeddable/slo/overview/slo_embeddable_factory.tsx
…mary slo.groupings.* is already covered by the 'slo.' entry in SUMMARY_NATIVE_PREFIXES, making the explicit startsWith check unnecessary. Made-with: Cursor
ApprovabilityVerdict: Needs human review This bug fix changes runtime query behavior by modifying You can customize Macroscope's approvability policy. Learn more. |
|
Starting backport for target branches: 9.3 |
💔 All backports failed
Manual backportTo create the backport manually run: Questions ?Please refer to the Backport tool documentation |
…g fields (elastic#255746) ## Summary Closes elastic#198289 Dashboard-level filters (e.g. `orchestrator.cluster.name`, `k8s.cluster.name`) were silently dropped when applied to SLO embeddables in global dashboards. Two root causes: 1. The SLO overview embeddable never subscribed to parent dashboard filters — it only used its own internal `groupFilters`, ignoring any filters set via the dashboard's KQL bar or control panels. 2. The SLO summary index stores user-defined grouping values under a flattened `slo.groupings.*` field, but `buildQueryFromFilters` was called with `ignoreFilterIfFieldNotInIndex: true`, which silently discarded any filter whose field name didn't exist in the summary mapping (e.g. `orchestrator.cluster.name` is not a native summary field). ### What changed - **Embeddable filter propagation**: The SLO overview embeddable now merges parent dashboard filters (via `useFetchContext`) with its own local filters before passing them downstream. - **Client-side filter rewriting**: A new shared utility (`common/rewrite_slo_filters.ts`) rewrites Kibana `Filter` objects whose field is not a native summary field — prefixing the field name with `slo.groupings.` in both `meta.key` and the query payload. This happens before `buildQueryFromFilters`, which now runs with `ignoreFilterIfFieldNotInIndex: false`. - **Shared mapping properties**: The summary index mapping properties are extracted into `common/summary_mapping_properties.ts`, imported by both the server-side component template and the client-side rewrite logic. This avoids duplication and ensures native-field detection stays in sync with the actual mapping. ### How it works When a dashboard filter like `orchestrator.cluster.name: "prod"` reaches the SLO hooks: 1. `rewriteFiltersForSloSummary` checks the field against the summary mapping properties. 2. `orchestrator` is not a top-level mapping key → the filter is rewritten to `slo.groupings.orchestrator.cluster.name: "prod"`. 3. `buildQueryFromFilters` now generates a valid ES query against the summary index. Native summary fields (e.g. `status`, `slo.tags`, `service.name`) pass through untouched. https://github.com/user-attachments/assets/9e086fa4-7f66-4461-aa60-1a7a0afc9faf ### Files changed | Area | Files | What | |------|-------|------| | Common | `common/summary_mapping_properties.ts` | Extracted mapping properties (single source of truth) | | Common | `common/rewrite_slo_filters.ts` | Client-side filter field rewriting | | Embeddable | `public/embeddable/slo/overview/slo_embeddable_factory.tsx` | Subscribe to dashboard filters via `useFetchContext` | | Hooks | `use_fetch_slo_list.ts`, `use_fetch_slo_groups.ts`, `use_fetch_slos_overview.ts` | Apply `rewriteFiltersForSloSummary` before `buildQueryFromFilters` | | Server | `summary_mappings_template.ts` | Import properties from common | | Tests | `common/rewrite_slo_filters.test.ts` | 14 unit tests for the rewrite logic | | Tests | `transform_generators/common.test.ts` | Updated server-side tests | | Tests | `find_slo_with_cluster_filter.ts` | E2E integration test for cluster filtering | ## Test plan - [x] Unit tests for `rewriteFilterForSloSummary` covering term, match_phrase, exists, bool, native fields, leaf-field subpaths, and edge cases (14 tests) - [x] Server-side `parseStringFilters` tests updated - [x] E2E integration test: create SLO with `orchestrator.cluster.name` grouping, query overview endpoint with cluster filter, verify results - [ ] Manual: create an SLO grouped by `orchestrator.cluster.name`, add SLO overview panel to a dashboard, add a dashboard control for `orchestrator.cluster.name`, verify filtering works Made with [Cursor](https://cursor.com)
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
1 similar comment
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
…g fields (elastic#255746) ## Summary Closes elastic#198289 Dashboard-level filters (e.g. `orchestrator.cluster.name`, `k8s.cluster.name`) were silently dropped when applied to SLO embeddables in global dashboards. Two root causes: 1. The SLO overview embeddable never subscribed to parent dashboard filters — it only used its own internal `groupFilters`, ignoring any filters set via the dashboard's KQL bar or control panels. 2. The SLO summary index stores user-defined grouping values under a flattened `slo.groupings.*` field, but `buildQueryFromFilters` was called with `ignoreFilterIfFieldNotInIndex: true`, which silently discarded any filter whose field name didn't exist in the summary mapping (e.g. `orchestrator.cluster.name` is not a native summary field). ### What changed - **Embeddable filter propagation**: The SLO overview embeddable now merges parent dashboard filters (via `useFetchContext`) with its own local filters before passing them downstream. - **Client-side filter rewriting**: A new shared utility (`common/rewrite_slo_filters.ts`) rewrites Kibana `Filter` objects whose field is not a native summary field — prefixing the field name with `slo.groupings.` in both `meta.key` and the query payload. This happens before `buildQueryFromFilters`, which now runs with `ignoreFilterIfFieldNotInIndex: false`. - **Shared mapping properties**: The summary index mapping properties are extracted into `common/summary_mapping_properties.ts`, imported by both the server-side component template and the client-side rewrite logic. This avoids duplication and ensures native-field detection stays in sync with the actual mapping. ### How it works When a dashboard filter like `orchestrator.cluster.name: "prod"` reaches the SLO hooks: 1. `rewriteFiltersForSloSummary` checks the field against the summary mapping properties. 2. `orchestrator` is not a top-level mapping key → the filter is rewritten to `slo.groupings.orchestrator.cluster.name: "prod"`. 3. `buildQueryFromFilters` now generates a valid ES query against the summary index. Native summary fields (e.g. `status`, `slo.tags`, `service.name`) pass through untouched. https://github.com/user-attachments/assets/9e086fa4-7f66-4461-aa60-1a7a0afc9faf ### Files changed | Area | Files | What | |------|-------|------| | Common | `common/summary_mapping_properties.ts` | Extracted mapping properties (single source of truth) | | Common | `common/rewrite_slo_filters.ts` | Client-side filter field rewriting | | Embeddable | `public/embeddable/slo/overview/slo_embeddable_factory.tsx` | Subscribe to dashboard filters via `useFetchContext` | | Hooks | `use_fetch_slo_list.ts`, `use_fetch_slo_groups.ts`, `use_fetch_slos_overview.ts` | Apply `rewriteFiltersForSloSummary` before `buildQueryFromFilters` | | Server | `summary_mappings_template.ts` | Import properties from common | | Tests | `common/rewrite_slo_filters.test.ts` | 14 unit tests for the rewrite logic | | Tests | `transform_generators/common.test.ts` | Updated server-side tests | | Tests | `find_slo_with_cluster_filter.ts` | E2E integration test for cluster filtering | ## Test plan - [x] Unit tests for `rewriteFilterForSloSummary` covering term, match_phrase, exists, bool, native fields, leaf-field subpaths, and edge cases (14 tests) - [x] Server-side `parseStringFilters` tests updated - [x] E2E integration test: create SLO with `orchestrator.cluster.name` grouping, query overview endpoint with cluster filter, verify results - [ ] Manual: create an SLO grouped by `orchestrator.cluster.name`, add SLO overview panel to a dashboard, add a dashboard control for `orchestrator.cluster.name`, verify filtering works Made with [Cursor](https://cursor.com)
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
…g fields (elastic#255746) ## Summary Closes elastic#198289 Dashboard-level filters (e.g. `orchestrator.cluster.name`, `k8s.cluster.name`) were silently dropped when applied to SLO embeddables in global dashboards. Two root causes: 1. The SLO overview embeddable never subscribed to parent dashboard filters — it only used its own internal `groupFilters`, ignoring any filters set via the dashboard's KQL bar or control panels. 2. The SLO summary index stores user-defined grouping values under a flattened `slo.groupings.*` field, but `buildQueryFromFilters` was called with `ignoreFilterIfFieldNotInIndex: true`, which silently discarded any filter whose field name didn't exist in the summary mapping (e.g. `orchestrator.cluster.name` is not a native summary field). ### What changed - **Embeddable filter propagation**: The SLO overview embeddable now merges parent dashboard filters (via `useFetchContext`) with its own local filters before passing them downstream. - **Client-side filter rewriting**: A new shared utility (`common/rewrite_slo_filters.ts`) rewrites Kibana `Filter` objects whose field is not a native summary field — prefixing the field name with `slo.groupings.` in both `meta.key` and the query payload. This happens before `buildQueryFromFilters`, which now runs with `ignoreFilterIfFieldNotInIndex: false`. - **Shared mapping properties**: The summary index mapping properties are extracted into `common/summary_mapping_properties.ts`, imported by both the server-side component template and the client-side rewrite logic. This avoids duplication and ensures native-field detection stays in sync with the actual mapping. ### How it works When a dashboard filter like `orchestrator.cluster.name: "prod"` reaches the SLO hooks: 1. `rewriteFiltersForSloSummary` checks the field against the summary mapping properties. 2. `orchestrator` is not a top-level mapping key → the filter is rewritten to `slo.groupings.orchestrator.cluster.name: "prod"`. 3. `buildQueryFromFilters` now generates a valid ES query against the summary index. Native summary fields (e.g. `status`, `slo.tags`, `service.name`) pass through untouched. https://github.com/user-attachments/assets/9e086fa4-7f66-4461-aa60-1a7a0afc9faf ### Files changed | Area | Files | What | |------|-------|------| | Common | `common/summary_mapping_properties.ts` | Extracted mapping properties (single source of truth) | | Common | `common/rewrite_slo_filters.ts` | Client-side filter field rewriting | | Embeddable | `public/embeddable/slo/overview/slo_embeddable_factory.tsx` | Subscribe to dashboard filters via `useFetchContext` | | Hooks | `use_fetch_slo_list.ts`, `use_fetch_slo_groups.ts`, `use_fetch_slos_overview.ts` | Apply `rewriteFiltersForSloSummary` before `buildQueryFromFilters` | | Server | `summary_mappings_template.ts` | Import properties from common | | Tests | `common/rewrite_slo_filters.test.ts` | 14 unit tests for the rewrite logic | | Tests | `transform_generators/common.test.ts` | Updated server-side tests | | Tests | `find_slo_with_cluster_filter.ts` | E2E integration test for cluster filtering | ## Test plan - [x] Unit tests for `rewriteFilterForSloSummary` covering term, match_phrase, exists, bool, native fields, leaf-field subpaths, and edge cases (14 tests) - [x] Server-side `parseStringFilters` tests updated - [x] E2E integration test: create SLO with `orchestrator.cluster.name` grouping, query overview endpoint with cluster filter, verify results - [ ] Manual: create an SLO grouped by `orchestrator.cluster.name`, add SLO overview panel to a dashboard, add a dashboard control for `orchestrator.cluster.name`, verify filtering works Made with [Cursor](https://cursor.com)
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
11 similar comments
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |

Summary
Closes #198289
Dashboard-level filters (e.g.
orchestrator.cluster.name,k8s.cluster.name) were silently dropped when applied to SLO embeddables in global dashboards. Two root causes:groupFilters, ignoring any filters set via the dashboard's KQL bar or control panels.slo.groupings.*field, butbuildQueryFromFilterswas called withignoreFilterIfFieldNotInIndex: true, which silently discarded any filter whose field name didn't exist in the summary mapping (e.g.orchestrator.cluster.nameis not a native summary field).What changed
useFetchContext) with its own local filters before passing them downstream.common/rewrite_slo_filters.ts) rewrites KibanaFilterobjects whose field is not a native summary field — prefixing the field name withslo.groupings.in bothmeta.keyand the query payload. This happens beforebuildQueryFromFilters, which now runs withignoreFilterIfFieldNotInIndex: false.common/summary_mapping_properties.ts, imported by both the server-side component template and the client-side rewrite logic. This avoids duplication and ensures native-field detection stays in sync with the actual mapping.How it works
When a dashboard filter like
orchestrator.cluster.name: "prod"reaches the SLO hooks:rewriteFiltersForSloSummarychecks the field against the summary mapping properties.orchestratoris not a top-level mapping key → the filter is rewritten toslo.groupings.orchestrator.cluster.name: "prod".buildQueryFromFiltersnow generates a valid ES query against the summary index.Native summary fields (e.g.
status,slo.tags,service.name) pass through untouched.Screen.Recording.2026-03-03.at.10.55.59.mov
Files changed
common/summary_mapping_properties.tscommon/rewrite_slo_filters.tspublic/embeddable/slo/overview/slo_embeddable_factory.tsxuseFetchContextuse_fetch_slo_list.ts,use_fetch_slo_groups.ts,use_fetch_slos_overview.tsrewriteFiltersForSloSummarybeforebuildQueryFromFilterssummary_mappings_template.tscommon/rewrite_slo_filters.test.tstransform_generators/common.test.tsfind_slo_with_cluster_filter.tsTest plan
rewriteFilterForSloSummarycovering term, match_phrase, exists, bool, native fields, leaf-field subpaths, and edge cases (14 tests)parseStringFilterstests updatedorchestrator.cluster.namegrouping, query overview endpoint with cluster filter, verify resultsorchestrator.cluster.name, add SLO overview panel to a dashboard, add a dashboard control fororchestrator.cluster.name, verify filtering worksMade with Cursor