Skip to content

[cloud_security_posture] Revert ECS field definition removal from misconfiguration transform - #17654

Merged
maxcold merged 1 commit into
elastic:mainfrom
maxcold:csp-transform-remove-dynamic-false
Mar 4, 2026
Merged

[cloud_security_posture] Revert ECS field definition removal from misconfiguration transform#17654
maxcold merged 1 commit into
elastic:mainfrom
maxcold:csp-transform-remove-dynamic-false

Conversation

@maxcold

@maxcold maxcold commented Mar 4, 2026

Copy link
Copy Markdown
Contributor

Proposed commit message

[cloud_security_posture] Revert ECS field definition removal from misconfiguration transform

Restoring explicit ECS mappings (fields/ecs.yml) and dynamic: false to the misconfiguration transform destination index template. This reverts the CSP-specific changes from #17552 (preview07) and the attempted fix in #17637 (preview08).

Why: Removing dynamic: false (preview08) caused cloudbeat's resource.raw fields — raw cloud provider API responses with deeply nested, variable-structure JSON — to be dynamically mapped via ecs@mappings, exceeding total_fields.limit of 1000. The proper fix requires adding enabled: false for resource.raw, result.evidence, and result.expected before dynamic: false can be safely removed. This is tracked in elastic/security-team#16225.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version conditions are aligned across all changed packages.

Author's Checklist

  • Changes are limited to a single package

How to test this PR locally

  1. Install the package via Fleet
  2. Verify the misconfiguration transform destination index template has dynamic: false
  3. Verify ECS fields (e.g., cloud.account.id) are explicitly mapped in the destination index
  4. Verify Group-by Cloud Account ID works in the Findings page

Related issues

…configuration transform

Restoring explicit ECS mappings and `dynamic: false` to prevent field
explosion from `ecs@mappings` dynamic templates. Removing `dynamic: false`
(preview08) caused cloudbeat's `resource.raw` fields to be dynamically
mapped, exceeding the 1000 total fields limit.
@maxcold
maxcold force-pushed the csp-transform-remove-dynamic-false branch from 20be1ae to e9f355d Compare March 4, 2026 10:58
@maxcold maxcold added Integration:cloud_security_posture Security Posture Management bugfix Pull request that fixes a bug issue labels Mar 4, 2026
@maxcold
maxcold marked this pull request as ready for review March 4, 2026 11:00
@maxcold
maxcold requested a review from a team as a code owner March 4, 2026 11:00
@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

@elasticmachine

Copy link
Copy Markdown

💚 Build Succeeded

History

@maxcold
maxcold merged commit 04e6518 into elastic:main Mar 4, 2026
11 checks passed
@maxcold
maxcold deleted the csp-transform-remove-dynamic-false branch March 4, 2026 16:43
@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package cloud_security_posture - 3.3.0-preview09 containing this change is available at https://epr.elastic.co/package/cloud_security_posture/3.3.0-preview09/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package cloud_security_posture - 3.3.0 containing this change is available at https://epr.elastic.co/package/cloud_security_posture/3.3.0/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix Pull request that fixes a bug issue Integration:cloud_security_posture Security Posture Management

3 participants