Skip to content

[Fleet] add fleet-server version check for SSL secrets#237464

Merged
juliaElastic merged 44 commits intoelastic:mainfrom
juliaElastic:ssl-secret-storage
Dec 8, 2025
Merged

[Fleet] add fleet-server version check for SSL secrets#237464
juliaElastic merged 44 commits intoelastic:mainfrom
juliaElastic:ssl-secret-storage

Conversation

@juliaElastic
Copy link
Contributor

@juliaElastic juliaElastic commented Oct 3, 2025

Summary

Do not merge until elastic/fleet-server#4470 is done, otherwise secrets would be enabled in serverless without fleet-server support.

Closes #237404

Added a separate setting to check if fleet-server supports SSL secrets. Working on the assumption that elastic/fleet-server#4470 will be done in 9.3.

The check ensures that SSL secrets are only used if all fleet-servers are on the supported version or fleet-server is standalone (in serverless).

To verify:

  • As a baseline start kibana without fleet-servers on 9.3 (so version check will not be met)
  • Add a Fleet Server host and fill out all secret fields (SSL keys)
  • Verify that the keys are stored as plain text (except for ssl.es_key, which requires fleet-server 8.12)
  • Add an Agent Binary source, fill out the secret field
  • Verify that the ssl.key is stored as plain text
  • Enroll fleet-server version 9.3 (and remove older ones) or set kibana config xpack.fleet.internal.fleetServerStandalone: true
  • Edit the Fleet Server host and verify that the SSL keys are stored as secrets
  • Edit the Agent Binary source and verify that the SSL key is stored as secrets
image image image image image image

Release note

Enable storing secrets in Fleet Server Host config if fleet-server is running at a minimum supported version.
The secret storage is going to be enabled if there are no Fleet Servers enrolled yet.

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

  • Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
  • Documentation was added for features that require explanation or tutorials
  • Unit or functional tests were updated or added to match the most common scenarios
  • If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
  • This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The release_note:breaking label should be applied in these situations.
  • Flaky Test Runner was used on any tests changed
  • The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines
  • Review the backport guidelines and apply applicable backport:* labels.

Identify risks

Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss.

Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging.

@juliaElastic juliaElastic self-assigned this Oct 3, 2025
@juliaElastic juliaElastic added release_note:skip Skip the PR/issue when compiling release notes backport:version Backport to applied version labels v9.2.0 v9.1.6 labels Oct 3, 2025
kibanamachine and others added 6 commits October 3, 2025 12:18
…t --include-path /api/status --include-path /api/alerting/rule/ --include-path /api/alerting/rules --include-path /api/actions --include-path /api/security/role --include-path /api/spaces --include-path /api/streams --include-path /api/fleet --include-path /api/saved_objects/_import --include-path /api/saved_objects/_export --include-path /api/maintenance_window --include-path /api/agent_builder --update'
… src/core/server/integration_tests/ci_checks'
@juliaElastic juliaElastic marked this pull request as ready for review October 6, 2025 11:58
@juliaElastic juliaElastic requested review from a team as code owners October 6, 2025 11:58
@nchaulet
Copy link
Member

nchaulet commented Oct 6, 2025

Does this mean that every new deployment first fleet server policies will not have ssl secrets, even if the user start a 9.3 as a first fleet server?

@juliaElastic
Copy link
Contributor Author

Does this mean that every new deployment first fleet server policies will not have ssl secrets, even if the user start a 9.3 as a first fleet server?

Hm good point, the check requires a running fleet server on version 9.3 so fleet-server policies at first wouldn't store secrets. Should we change this to allow secrets if there are no running fleet-servers?
Though there is a chance that an older version of fleet-server will be enrolled in self-managed.

@nchaulet
Copy link
Member

nchaulet commented Oct 6, 2025

Hm good point, the check requires a running fleet server on version 9.3 so fleet-server policies at first wouldn't store secrets. Should we change this to allow secrets if there are no running fleet-servers?

In my opinion it will make sense, I think there are low odds a user want to add an older fleet server than the the stack version on a new cluster no? maybe worth having other people thoughts on that cc @kpollich

Copy link
Contributor

@florent-leborgne florent-leborgne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for docs

@botelastic botelastic bot added the Team:Fleet Team label for Observability Data Collection Fleet team label Oct 6, 2025
@elasticmachine
Copy link
Contributor

Pinging @elastic/fleet (Team:Fleet)

@kpollich
Copy link
Member

kpollich commented Oct 6, 2025

Hm good point, the check requires a running fleet server on version 9.3 so fleet-server policies at first wouldn't store secrets. Should we change this to allow secrets if there are no running fleet-servers?

I think at this point in time it makes sense to flip this behavior and start defaulting to secrets being enabled.

@kc13greiner kc13greiner self-requested a review October 6, 2025 21:57
OUTPUT_SECRETS_MINIMUM_FLEET_SERVER_VERSION,
} from '../../../../../../../common/constants';

export type SecretType = 'output' | 'ssl';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought that the output secrets were supported since long time. What is the case for outputs that need the new version of fleet server?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's true, we can probably leave the secret that ends up in the output in the previous version check

Copy link
Contributor Author

@juliaElastic juliaElastic Oct 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated, had to refactor to handle the separate ssl and output secret enabled state in the fleet server host form
e378419

image

tested with a fleet-server 9.1.4 enrolled, and verifying that only the ES key is saved as a secret
image

after unenrolling that and enrolling a fleet-server 9.3.0-SNAPSHOT, update the fleet-server-host and verify that all 3 SSL keys are stored as secret
image

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this change!

@juliaElastic juliaElastic added release_note:fix and removed release_note:skip Skip the PR/issue when compiling release notes labels Oct 7, 2025
@elasticmachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
fleet 2.1MB 2.1MB +248.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
fleet 178.8KB 179.0KB +176.0B

History

cc @juliaElastic

@juliaElastic juliaElastic merged commit 827ae4c into elastic:main Dec 8, 2025
12 checks passed
@kibanamachine kibanamachine added backport:skip This PR does not require backporting and removed backport:version Backport to applied version labels labels Dec 8, 2025
JordanSh pushed a commit to JordanSh/kibana that referenced this pull request Dec 9, 2025
## Summary

Do not merge until elastic/fleet-server#4470
is done, otherwise secrets would be enabled in serverless without
fleet-server support.

Closes elastic#237404

Added a separate setting to check if fleet-server supports SSL secrets.
Working on the assumption that
elastic/fleet-server#4470 will be done in 9.3.

The check ensures that SSL secrets are only used if all fleet-servers
are on the supported version or fleet-server is standalone (in
serverless).

To verify:
- As a baseline start kibana without fleet-servers on 9.3 (so version
check will not be met)
- Add a Fleet Server host and fill out all secret fields (SSL keys)
- Verify that the keys are stored as plain text (except for
`ssl.es_key`, which requires fleet-server 8.12)
- Add an Agent Binary source, fill out the secret field
- Verify that the `ssl.key` is stored as plain text
- Enroll fleet-server version 9.3 (and remove older ones) or set kibana
config `xpack.fleet.internal.fleetServerStandalone: true`
- Edit the Fleet Server host and verify that the SSL keys are stored as
secrets
- Edit the Agent Binary source and verify that the SSL key is stored as
secrets

<img width="509" height="835" alt="image"
src="https://github.com/user-attachments/assets/f80e79b1-4e07-459a-b2c6-75d36e6fe6f3"
/>
<img width="716" height="483" alt="image"
src="https://github.com/user-attachments/assets/51d68b24-1b82-4236-80c6-6bf4589f9aa2"
/>
<img width="506" height="828" alt="image"
src="https://github.com/user-attachments/assets/e2fdc0d9-9e8f-49c4-b993-7afb67663f30"
/>
<img width="711" height="348" alt="image"
src="https://github.com/user-attachments/assets/1348ad7a-dea4-4bf2-ab04-d1a1ad01f421"
/>

<img width="713" height="468" alt="image"
src="https://github.com/user-attachments/assets/32a7b447-fc11-41ce-b594-a39128673a81"
/>
<img width="705" height="375" alt="image"
src="https://github.com/user-attachments/assets/0b658f57-c492-468f-a9b7-325d9134fafa"
/>

## Release note

Enable storing secrets in Fleet Server Host config if fleet-server is
running at a minimum supported version.
The secret storage is going to be enabled if there are no Fleet Servers
enrolled yet.


### Checklist

Check the PR satisfies following conditions. 

Reviewers should verify this PR satisfies this list as well.

- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
- [ ] Review the [backport
guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing)
and apply applicable `backport:*` labels.

### Identify risks

Does this PR introduce any risks? For example, consider risks like hard
to test bugs, performance regression, potential of data loss.

Describe the risk, its severity, and mitigation for each identified
risk. Invite stakeholders and evaluate how to proceed before merging.

- [ ] [See some risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
- [ ] ...

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:skip This PR does not require backporting release_note:fix Team:Fleet Team label for Observability Data Collection Fleet team v9.3.0

9 participants