multiple: update transform mapping

[mmahacek]

{m365_defender, aws, aws_securityhub, google_scc, microsoft_defender_cloud, microsoft_defender_endpoint, prisma_cloud, qualys_vmdr, rapid7_inishgtvm, teneable_io, wiz} Addresses config not addressed by #14809

Proposed commit message

{m365_defender, aws, aws_securityhub, google_scc, microsoft_defender_cloud, microsoft_defender_endpoint, prisma_cloud, qualys_vmdr, rapid7_inishgtvm, teneable_io, wiz} Addresses transform mapping not addressed by #14809

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

• [ ]

How to test this PR locally

Verify component template for the transforms mention in the PR have data_stream.namespace properly mapped as keyword

Related issues

• Closes [Multiple Integrations]: data_stream.namespace mapping error on transforms #19054

Screenshots

n/a

[infra-vault-gh-plugin-prod]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[elastic-vault-github-plugin-prod]

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

[efd6]

I think this is the wrong place to fix this. If it's needed (it is), we should make it possible for a type: constant_keyword to be redefined as a type: keyword. This would be a change in github.com/elastic/elastic-package in the allow list in allowedTypeOverride.

Note that the changes here are bug-fixes (ref) and so should bump patch.

cc @jsoriano

[mmahacek]

@efd6 Thanks for the feedback. It does look like the allowedTypeOverride would benefit from also having {"constant_keyword", "keyword"} defined so that the type could be overridden either direction. I've raised elastic/elastic-package#3583 regarding this.

I updated this PR to bump the versions down from minor to patch, since these packages would need to be rebuilt to get the correct mapping applied after updating the elastic-package anyway.

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: e1bcd2f

History

• 💚 Build #43112 succeeded f19f657
• 💚 Build #43099 succeeded bb72f99
• 💚 Build #43024 succeeded 243ffca

cc @mmahacek

[efd6]

@mmahacek The elastic-package issue appears to be aligned with ecosystem, so I'd suggest that you send that change. It won't be available for a while, so we should also go ahead with this as an interim fix, but I'd like to see the documentation for the fields defined in ECS be copied over to the local definitions here so that there is not a documentation regression. Then the future follow up would be to revert to the external: ecs while retaining the type new type specification.

[mmahacek]

@efd6 I'm not on the dev team, and not fluent in Go, so I'm not going to submit any code changes for the elastic-package repo. I've got a issue reported there for someone on that team to pick up.

It also looks like my permissions on this repo have been updated, so I no longer have push access to my branch. I either need someone to update the AWS version since there's a conflict with another patch that has been pushed, or I need to cancel this PR and re-submit from a fork.

[efd6]

@mmahacek I've sent elastic/elastic-package#3605.