Skip to content

[8.19] (backport #10857) Move component working directory management to coordinator#10924

Merged
swiatekm merged 2 commits into8.19from
mergify/bp/8.19/pr-10857
Oct 30, 2025
Merged

[8.19] (backport #10857) Move component working directory management to coordinator#10924
swiatekm merged 2 commits into8.19from
mergify/bp/8.19/pr-10857

Conversation

@mergify
Copy link
Contributor

@mergify mergify bot commented Oct 30, 2025

What does this PR do?

It makes the coordinator responsible for creating and removing the working directories of components. Until now, CommandRuntime and ServiceRuntime did this on their own, whereas the OtelManager didn't do it at all. We move the logic for creating and removing the directories into the component module itself, and call it from the Coordinator. The new logic is as follows.

  • When we generate a new component model, we ensure each component has a working directory.
  • When we get a STOPPED state for a component AND that component is not present in the component model, we delete its working directory.

The condition that the component must not be present in the current model fixes an issue where the working directory would be deleted when the component was being moved between runtimes.

Why is it important?

  1. Currently, the Otel runtime does not remove the working directories at all. The behavior should be the same no matter where the component runs.
  2. If a component is moved from the process runtime to the otel runtime, its working directory is incorrectly deleted. This causes filestream to re-ingest all logs, for example.

Note that I've included an integration test from #10544, which specifically checks point 2 from above.

Checklist

  • I have read and understood the pull request guidelines of this project.
  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • [ ] I have made corresponding changes to the documentation
  • [ ] I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in ./changelog/fragments using the changelog tool
  • I have added an integration test or an E2E test

How to test this PR locally

Related issues

This is an automatic backport of pull request #10857 done by [Mergify](https://mergify.com).
@swiatekm @leehinman @mergify
Move component working directory management to coordinator (#10857)
0f6268e 
* Move component setup and teardown code to the Component struct

* Move teardown to coordinator

* Integration test to check data re-ingest when switching runtimes

* Add integration test for component working dirs

* Check workdir creation time in tests

* Fix linter warnings

* Fix a minor issue in endpoint integration tests

* Add changelog entry

* Completely remove workdir handling from command runtime

* Fix changelog summary

* Use fleet in the integration test

---------

Co-authored-by: Lee E. Hinman <lee.e.hinman@elastic.co>
(cherry picked from commit 41fd24f)
@mergify mergify bot requested a review from a team as a code owner October 30, 2025 14:25
@mergify mergify bot added the backport label Oct 30, 2025
@mergify mergify bot requested review from michalpristas and ycombinator and removed request for a team October 30, 2025 14:25
@github-actions github-actions bot added bug Something isn't working Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team labels Oct 30, 2025
@elasticmachine
Copy link
Contributor

elasticmachine commented Oct 30, 2025

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)
@swiatekm swiatekm enabled auto-merge (squash) October 30, 2025 20:37
@swiatekm swiatekm merged commit e80c326 into 8.19 Oct 30, 2025
17 checks passed
@swiatekm swiatekm deleted the mergify/bp/8.19/pr-10857 branch October 30, 2025 22:31
@elasticmachine
Copy link
Contributor

elasticmachine commented Oct 30, 2025

💛 Build succeeded, but was flaky

Failed CI Steps

History

cc @swiatekm
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport bug Something isn't working Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team

2 participants

@elasticmachine @swiatekm