Apply default processors to events from the monitoring receiver

[faec]

Include the beat processor in the OTel Collector and use it to apply default processors (most notably add_host_metadata) to monitoring events generated by collector internal telemetry. See #12577

Checklist

• I have read and understood the pull request guidelines of this project.
• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in ./changelog/fragments using the changelog tool
• I have added an integration test or an E2E test

How to test this PR locally

With this PR, any ingested monitoring metrics with component.id of elasticsearch-default or elasticsearch-monitoring should include the fields from standard processors, most notably the hostname.

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b processors-for-otel-monitoring upstream/processors-for-otel-monitoring
git merge upstream/main
git push upstream processors-for-otel-monitoring

[mergify]

This pull request does not have a backport label. Could you fix it @faec? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-./d./d is the label that automatically backports to the 8./d branch. /d is the digit
• backport-active-all is the label that automatically backports to all active branches.
• backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
• backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[mauri870]

Can we have a test for this? Ideally you could change one of the TestMonitoringReceiver (integration) to assert some fields are present.

[swiatekm]

I agree that an integration test should check this.

[faec]

@mauri870 I couldn't find a practical way to do an integration test that used the OTel manager config code directly, so I added a unit test that makes sure the expected processor config is there and then separately an integration test that makes sure that including a processor with that config template produces the expected event fields.

[elasticmachine]

⏳ Build in-progress, with failures

• Buildkite Build
• Commit: f931ee9

Failed CI Steps

• check-ci
• Unit tests - Ubuntu 22.04 ARM64
• :open-pull-request: :kubernetes: v1.27.16:amd64:slim-wolfi

History

• 💔 Build #34709 failed 92c62be
• 💔 Build #34543 failed c4db28f

cc @faec

[mauri870]

Thanks for adding a test!