Retry on TLS handshake errors in ES client

[ycombinator]

What is the problem this PR solves?

When the Elasticsearch output is configured with multiple hosts whose certificates chain to different CAs (or a temporarily untrusted host coexists with a trusted one), a TLS handshake failure against one host aborts the request outright — even though the underlying connection pool's dead-host failover would otherwise route the next attempt to a healthy host.

The pool already marks the failed host dead via OnFailure unconditionally on any transport error (elastic-transport-go/v8.8.0 elastictransport.go#L420), but Perform() only loops to the next host if RetryOnError returns true (elastic-transport-go/v8.8.0 elastictransport.go#L424). The existing fleet-server classifier only covered ECONNREFUSED/ECONNRESET (internal/pkg/es/client.go#L42), so cert verification errors leaked out and — for the startup data migrations in internal/pkg/dl/migration.go, which call client.UpdateByQuery once with no per-host retry of their own — crashed the process in a supervisor restart loop.

This was hit in the field on a Fleet Server 9.2.3 deployment whose policy carried two ES output hosts but whose trust store only contained the ECK-injected CA for one of them; the migration kept landing on the untrusted host via round-robin.

How does this PR solve the problem?

Adds a new WithRetryOnTLSHandshakeError ConfigOption, applied inside applyDefaultOptions, which extends the RetryOnError classifier to return true for *tls.CertificateVerificationError unwrapped via errors.As (covering wrapping by *url.Error, fmt.Errorf, etc.).

The option composes with any previously-installed RetryOnError predicate via OR semantics — WithRetryOnErrs(ECONNREFUSED, ECONNRESET) is still honored — so it only widens the set of retried errors and never clobbers an existing classifier.

With this in place, on a TLS handshake error against host A:

1. The pool marks A dead via OnFailure (existing behavior).
2. RetryOnError now returns true, so Perform() loops.
3. pool.Next() is called again; A is no longer in the live list, so the request goes to B.

This is a no-op when ssl.verification_mode: none is configured, since that path sets InsecureSkipVerify=true and crypto/tls never produces a *tls.CertificateVerificationError. The option does not bypass or override that setting.

It is also bounded by the existing MaxRetries setting (5), so a single-host pool with a permanently bad cert still terminates — at most one extra retry attempt before giving up.

How to test this PR locally

Unit tests cover both pieces:

go test ./internal/pkg/es/ -run 'TestIsTLSHandshakeError|TestWithRetryOnTLSHandshakeError' -v

• TestIsTLSHandshakeError — table-driven, exercises the unwrap chain: nil, unrelated error, syscall.ECONNREFUSED, direct *tls.CertificateVerificationError, wrapped via fmt.Errorf %w, wrapped via *url.Error (mirrors how net/http actually surfaces it), and doubly wrapped (fmt.Errorf around *url.Error).
• TestWithRetryOnTLSHandshakeError — verifies (a) the option works standalone, and (b) when layered on top of WithRetryOnErrs(syscall.ECONNREFUSED), both predicates fire (OR semantics), and neither matches ECONNRESET.

End-to-end manual reproduction: configure a Fleet Server policy with two ES output hosts where only one CA is in the trust store, and observe that startup migrations no longer crash the process when the untrusted host is picked first by round-robin.

Design Checklist

• I have ensured my design is stateless and will work when multiple fleet-server instances are behind a load balancer.
• I have or intend to scale test my changes, ensuring it will work reliably with 100K+ agents connected.
• I have included fail safe mechanisms to limit the load on fleet-server: rate limiting, circuit breakers, caching, load shedding, etc.

Checklist

• I have commented my code, particularly in hard-to-understand areas
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in ./changelog/fragments using the changelog tool

[mergify]

This pull request does not have a backport label. Could you fix it @ycombinator? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-./d./d is the label to automatically backport to the 8./d branch. /d is the digit
• backport-active-all is the label that automatically backports to all active branches.
• backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
• backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

[michel-laterman]

LGTM. We should consider replaceing isTLSHandshakeError with a more generic call in the future

[github-actions]

@Mergifyio backport 8.19 9.3 9.4

[mergify]

backport 8.19 9.3 9.4

✅ Backports have been created

Details

• #6780 [8.19](backport #6767) Retry on TLS handshake errors in ES client has been created for branch 8.19 but encountered conflicts

Cherry-pick of 52f9cbe has failed:

On branch mergify/bp/8.19/pr-6767
Your branch is up to date with 'origin/8.19'.

You are currently cherry-picking commit 52f9cbe.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	new file:   changelog/fragments/1775592233-retry-on-tls-handshake-errors-in-es-client.yaml
	modified:   internal/pkg/es/client_test.go

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   internal/pkg/es/client.go

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

• #6781 [9.3](backport #6767) Retry on TLS handshake errors in ES client has been created for branch 9.3
• #6782 [9.4](backport #6767) Retry on TLS handshake errors in ES client has been created for branch 9.4