Ensure the self-monitoring configuration knows the actual component runtime

[swiatekm]

What does this PR do?

When we generate self-monitoring configuration, we do certain things differently depending on whether a component will run in a beat process or a beat receiver in an otel collector. This PR ensures this information is accurate. Up until now, this decision was based on which runtime the component was configured to use, rather than what it ultimately used. If the component cannot run in the otel runtime - for example because the output is not supported - it would fall back to the process runtime, but this would happen after the self-monitoring configuration was generated, leading to inconsistencies.

This is achieved by making the following changes:

1. Falling back to the process runtime is done as a component modifier instead of a dedicated Coordinator method.
2. Component modifiers are now applied immediately after the components are generated, and before the self-monitoring configuration is generated. As a result, the monitoring manager now sees what runtime components will actually run as.
3. For the self-monitoring components, we check if the output is otel supported and make decisions based on that.

Why is it important?

Checklist

• I have read and understood the pull request guidelines of this project.
• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• [ ] I have made corresponding changes to the documentation
• [ ] I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in ./changelog/fragments using the changelog tool
• [ ] I have added an integration test or an E2E test

How to test this PR locally

Build agent locally and run using the following configuration:

agent:
  logging:
    to_stderr: true
inputs:
- data_stream:
    namespace: default
  id: unique-system-metrics-input
  streams:
  - data_stream:
      dataset: system.cpu
    metricsets:
    - cpu
  type: system/metrics
  use_output: default
outputs:
  default:
    username: elastic
    password: elastic
    hosts:
    - 127.0.0.1:9200
    type: elasticsearch
    indices: []

Verify that all the components are running as beats processes via the status, and that the prometheus monitoring component is not present.

Related issues

• Closes https://github.com/elastic/ingest-dev/issues/6422

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[elasticmachine]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: c20cea5

Failed CI Steps

• Unit tests - macOS 15 ARM
• K8s tests: 1.30.0
• K8s tests: 1.30.0
• Custom ECH Testing
• Custom ECH Testing
• x86_64:sudo: default

History

• 💔 Build #30910 failed 0e8bc78
• 💔 Build #30876 failed 1dc723b
• 💔 Build #30868 failed aba6cc1
• 💔 Build #30865 failed 7e29478

cc @swiatekm

[cmacknz]

Tested locally and works, a couple of very minor comments.

A lot of this is mechanical so we can probably backport after 9.2.2 is released, I don't think we need to rush this into 9.2.2 as long as we are confident there is no reason to with the work around that is already there.