Expose HTTP connection metrics to telemetry

[DaveCTurner]

Closes ES-12223

[elasticsearchmachine]

Pinging @elastic/es-distributed-coordination (Team:Distributed Coordination)

[elasticsearchmachine]

Hi @DaveCTurner, I've created a changelog YAML for you.

[mhl-b]

Do you know which thread runs channel initializer? Is it same stack that accepts socket?

I'm wondering for case when transport threads are saturated and we accept new socket but haven't setup channel pipeline yet, so these metrics wont reflect that. Should we measure those on parent channel that listen socket?

[mhl-b]

LGTM, minor question about place for metrics capturing.

[DaveCTurner]

Do you know which thread runs channel initializer? Is it same stack that accepts socket?

No, the channel is initialized on its own event loop, see io.netty.channel.AbstractChannel.AbstractUnsafe#register. I don't think that matters very much for these stats: if the event loop is saturated then we have much bigger problems than these stats being slightly stale.