fix: don't override log level from state for standalone agents

[pkoutsovasilis]

What does this PR do?

This PR fixes a bug where the log level for standalone Elastic Agent installations was incorrectly restored from the persisted state directory (/var/lib/elastic-agent-standalone/...) instead of being sourced from the current configuration file. This resulted in stale configuration values after updates to the ConfigMap, particularly when the pod was restarted without clearing the state directory.

The change ensures that for standalone mode, the log.level value is always taken from the configuration, not restored from previously persisted agent state. For managed (Fleet) mode, the previous behavior is preserved.

Why is it important?

This fix addresses an issue where changes to logging configuration were not taking effect unless the agent's state directory was manually deleted. This led to operational confusion and could cause logs to be emitted at incorrect verbosity levels.

Ensuring that the standalone agent reliably reflects the current configuration helps users maintain observability and avoid side effects like duplicated logs when restarting pods.

Checklist

• I have read and understood the pull request guidelines of this project.
• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in ./changelog/fragments using the changelog tool
• I have added an integration test or an E2E test

Disruptive User Impact

This change should not cause disruption. It fixes the logic so that standalone agents correctly reflect updated configuration values after pod restarts, without requiring manual state cleanup. Users relying on previously persisted log levels (instead of ConfigMap changes) may see a difference in behavior, but this aligns with the intended functionality.

How to test this PR locally

helm install elastic-agent ./deploy/helm/elastic-agent -n kube-system  --set outputs.default.type=ESPlainAuthBasic --set outputs.default.url='...' --set outputs.default.username='...' --set outputs.default.password='...' --set agent.presets.perNode.agent.logging.level=info

helm upgrade elastic-agent ./deploy/helm/elastic-agent -n kube-system  --set outputs.default.type=ESPlainAuthBasic --set outputs.default.url='...' --set outputs.default.username='...' --set outputs.default.password='...' --set agent.presets.perNode.agent.logging.level=debug

Related issues

• Relates Some config values stale after updating standalone config map #8137

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: efa69be

History

• 💛 Build #23453 was flaky 587af4d

cc @pkoutsovasilis

[swiatekm]

LGTM, though I'd love to have a simple unit test for this function.

[pkoutsovasilis]

LGTM, though I'd love to have a simple unit test for this function.

more than a fair point, lemme check what I can cook for it

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

[pchila]

LGTM

[github-actions]

@Mergifyio backport 8.17 8.18 8.19 9.0 9.1

[mergify]

backport 8.17 8.18 8.19 9.0 9.1

✅ Backports have been created

Details

• #8857 [8.17] (backport #8784) fix: don't override log level from state for standalone agents has been created for branch 8.17
• #8858 [8.18] (backport #8784) fix: don't override log level from state for standalone agents has been created for branch 8.18
• #8859 [8.19] (backport #8784) fix: don't override log level from state for standalone agents has been created for branch 8.19
• #8860 [9.0] (backport #8784) fix: don't override log level from state for standalone agents has been created for branch 9.0
• #8861 [9.1] (backport #8784) fix: don't override log level from state for standalone agents has been created for branch 9.1