Add epoch blob-cache metric

[henningandersen]

Add epoch (changes) as a metric to the blob-cache, giving visibility into the velocity of change in the cache and indications of cache churn

Relates ES-12595

[elasticsearchmachine]

Pinging @elastic/es-distributed-indexing (Team:Distributed Indexing)

[elasticsearchmachine]

Hi @henningandersen, I've created a changelog YAML for you.

[tlrx]

LGTM

[andreidan]

Thanks for working on this Henning.

The change itself LGTM but I'm not entirely sure what this metric will give us?

i.e. a steadily increasing epoch will point to our decaying task running as expected ?
I could perhaps see the metric being interesting if we are to correlate it with an entry's lastAccessedEpoch but I'm not exactly following why is this metric useful in isolation?

[henningandersen]

We discussed this and clarified that it is the rate of change that is interesting, in particular for comparing across different clusters, a high rate could seem to be an indication of cache undersize or problem relative to a low rate.

[henningandersen]

Thanks both!