Skip to content

[Fleet] Expose ssl options for ES and remote ES outputs in UI#208745

Merged
criamico merged 27 commits intoelastic:mainfrom
criamico:207326_es_outputs_ssl
Feb 21, 2025
Merged

[Fleet] Expose ssl options for ES and remote ES outputs in UI#208745
criamico merged 27 commits intoelastic:mainfrom
criamico:207326_es_outputs_ssl

Conversation

@criamico
Copy link
Contributor

@criamico criamico commented Jan 29, 2025

Fixes #207326

Summary

Expose SSL options for outputs of type ES and remote ES outputs to use in case of mTLS configuration.

These options correspond to --elastic-agent-cert, --elastic-agent-cert-key, --certificate-authorities CLI options explained in the docs and they need to be defined only when the user intends to configure mTLS in the system.

Secrets inputs are disabled for now in the UI until the needed changes to fleet server are done.

Testing

  • Create an ES or remote ES output with all the SSL fields compiled
  • Either make it default or assign it as a custom output for an agent policy
  • Check that the agent policy has the input data under fleet section:
Screenshot 2025-02-20 at 11 56 05
Screenshots Screenshot 2025-02-20 at 16 07 38 Screenshot 2025-02-18 at 15 31 54

Checklist

@criamico criamico self-assigned this Jan 29, 2025
@criamico criamico added Team:Fleet Team label for Observability Data Collection Fleet team release_note:feature Makes this part of the condensed release notes v9.0.0 labels Jan 29, 2025
kibanamachine and others added 4 commits January 29, 2025 13:39
…t --include-path /api/status --include-path /api/alerting/rule/ --include-path /api/alerting/rules --include-path /api/actions --include-path /api/security/role --include-path /api/spaces --include-path /api/fleet --include-path /api/dashboards --update'
@criamico
Copy link
Contributor Author

@elasticmachine merge upstream

@elasticmachine
Copy link
Contributor

merge conflict between base and head

@criamico criamico force-pushed the 207326_es_outputs_ssl branch from 7d5e515 to 3c8bfbb Compare February 10, 2025 11:12
@elastic elastic deleted a comment from github-actions bot Feb 10, 2025
@criamico
Copy link
Contributor Author

@elasticmachine merge upstream

elasticmachine and others added 7 commits February 17, 2025 17:13
…t --include-path /api/status --include-path /api/alerting/rule/ --include-path /api/alerting/rules --include-path /api/actions --include-path /api/security/role --include-path /api/spaces --include-path /api/fleet --include-path /api/dashboards --update'
@criamico criamico marked this pull request as ready for review February 18, 2025 15:16
@criamico criamico requested a review from a team as a code owner February 18, 2025 15:16
@elasticmachine
Copy link
Contributor

Pinging @elastic/fleet (Team:Fleet)

@juliaElastic
Copy link
Contributor

As the generated policy goes under a different section then other outputs, I think we'll need to do these changes as well in fleet server. I'll hide the secrets fields in this PR as well, wdyt?

Yeah, we should probably hide them until the fleet-server change is made.

@criamico
Copy link
Contributor Author

@elasticmachine merge upstream

@criamico criamico added v9.1.0 backport:skip This PR does not require backporting and removed v9.0.0 backport:prev-minor labels Feb 20, 2025
@criamico
Copy link
Contributor Author

@elasticmachine merge upstream

@elasticmachine
Copy link
Contributor

merge conflict between base and head

@criamico
Copy link
Contributor Author

@elasticmachine merge upstream

Copy link
Contributor

@juliaElastic juliaElastic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@criamico
Copy link
Contributor Author

@juliaElastic I noticed that I had a small bug in the plain text key due to the fact that we have several secrets in some forms, so disabling the secrets in the whole page is not a good solution. i pushed a change that introduces the feature flag enableSSLSecrets.

@criamico
Copy link
Contributor Author

@elasticmachine merge upstream

@criamico criamico enabled auto-merge (squash) February 21, 2025 16:53
@criamico criamico merged commit ff84c0e into elastic:main Feb 21, 2025
9 checks passed
@elasticmachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
fleet 1199 1200 +1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
fleet 1.7MB 1.7MB +1.7KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
fleet 156.0KB 156.0KB +20.0B

History

cc @criamico

@criamico criamico deleted the 207326_es_outputs_ssl branch February 24, 2025 08:43
JoseLuisGJ pushed a commit to JoseLuisGJ/kibana that referenced this pull request Feb 27, 2025
…c#208745)

Fixes elastic#207326

## Summary
Expose SSL options for outputs of type ES and remote ES outputs to use
in case of mTLS configuration.

These options correspond to `--elastic-agent-cert,
--elastic-agent-cert-key, --certificate-authorities` CLI options
explained [in the
docs](https://www.elastic.co/guide/en/fleet/current/tls-overview.html#mutual-tls-connection)
and they need to be defined only when the user intends to configure mTLS
in the system.

Secrets inputs are disabled for now in the UI until the needed changes
to fleet server are done.

### Testing
- Create an ES or remote ES output with all the SSL fields compiled
- Either make it default or assign it as a custom output for an agent
policy
- Check that the agent policy has the input data under `fleet` section:

<img width="741" alt="Screenshot 2025-02-20 at 11 56 05"
src="https://github.com/user-attachments/assets/bc891bbc-d669-4e31-951f-d045b64328e7"
/>


<details>
  <summary>Screenshots</summary>

<img width="803" alt="Screenshot 2025-02-20 at 16 07 38"
src="https://github.com/user-attachments/assets/e489cd51-6a40-4820-bf39-7670ad2ce69b"
/>
<img width="810" alt="Screenshot 2025-02-18 at 15 31 54"
src="https://github.com/user-attachments/assets/9f155b2b-1feb-424f-b02b-b232e1e4e63e"
/>


</details>


### Checklist
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this pull request Mar 22, 2025
…c#208745)

Fixes elastic#207326

## Summary
Expose SSL options for outputs of type ES and remote ES outputs to use
in case of mTLS configuration.

These options correspond to `--elastic-agent-cert,
--elastic-agent-cert-key, --certificate-authorities` CLI options
explained [in the
docs](https://www.elastic.co/guide/en/fleet/current/tls-overview.html#mutual-tls-connection)
and they need to be defined only when the user intends to configure mTLS
in the system.

Secrets inputs are disabled for now in the UI until the needed changes
to fleet server are done.

### Testing
- Create an ES or remote ES output with all the SSL fields compiled
- Either make it default or assign it as a custom output for an agent
policy
- Check that the agent policy has the input data under `fleet` section:

<img width="741" alt="Screenshot 2025-02-20 at 11 56 05"
src="https://github.com/user-attachments/assets/bc891bbc-d669-4e31-951f-d045b64328e7"
/>


<details>
  <summary>Screenshots</summary>

<img width="803" alt="Screenshot 2025-02-20 at 16 07 38"
src="https://github.com/user-attachments/assets/e489cd51-6a40-4820-bf39-7670ad2ce69b"
/>
<img width="810" alt="Screenshot 2025-02-18 at 15 31 54"
src="https://github.com/user-attachments/assets/9f155b2b-1feb-424f-b02b-b232e1e4e63e"
/>


</details>


### Checklist
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
juliaElastic added a commit that referenced this pull request Mar 26, 2025
## Summary

Fix Remote ES output with secret token secret. The secrets were not
correctly saved to the full agent policy, bug introduced in this pr:
#208745

Also added a doc how to test sync integrations feature with 2 local
clusters.

To test:
- create remote ES output with service token as a secret
- create an agent policy and use the remote ES output as data and
monitoring output
- check that the full agent policy contains `outputs.secrets`
- enroll an agent and verify that it becomes healthy and the data is
ingested to the remote cluster

<img width="1704" alt="image"
src="https://github.com/user-attachments/assets/1744fb71-75b4-44f9-ade9-8a29dd44fd95"
/>
<img width="1619" alt="image"
src="https://github.com/user-attachments/assets/941936a3-ddc6-42c9-9c49-a941ae9007b6"
/>
<img width="816" alt="image"
src="https://github.com/user-attachments/assets/e9b703fb-b032-4879-8f0a-c69e67f1a4b2"
/>
<img width="1792" alt="image"
src="https://github.com/user-attachments/assets/077d6f1e-efc2-486b-a6d5-9a87ae012471"
/>
cqliu1 pushed a commit to cqliu1/kibana that referenced this pull request Mar 31, 2025
…c#216033)

## Summary

Fix Remote ES output with secret token secret. The secrets were not
correctly saved to the full agent policy, bug introduced in this pr:
elastic#208745

Also added a doc how to test sync integrations feature with 2 local
clusters.

To test:
- create remote ES output with service token as a secret
- create an agent policy and use the remote ES output as data and
monitoring output
- check that the full agent policy contains `outputs.secrets`
- enroll an agent and verify that it becomes healthy and the data is
ingested to the remote cluster

<img width="1704" alt="image"
src="https://github.com/user-attachments/assets/1744fb71-75b4-44f9-ade9-8a29dd44fd95"
/>
<img width="1619" alt="image"
src="https://github.com/user-attachments/assets/941936a3-ddc6-42c9-9c49-a941ae9007b6"
/>
<img width="816" alt="image"
src="https://github.com/user-attachments/assets/e9b703fb-b032-4879-8f0a-c69e67f1a4b2"
/>
<img width="1792" alt="image"
src="https://github.com/user-attachments/assets/077d6f1e-efc2-486b-a6d5-9a87ae012471"
/>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:skip This PR does not require backporting release_note:feature Makes this part of the condensed release notes Team:Fleet Team label for Observability Data Collection Fleet team v9.1.0

4 participants