[Elasticsearch] Add metric type to fields

[constanca-m]

What does this PR do?

Adds metric type to the fields of the following data streams:

• CCR:
  • Please check:
    • elasticsearch.ccr.bytes_read: counter or gauge?
    • max_seq_no is a gauge - Correct?
• Cluster stats:
  • Everything is a gauge. There are no cumulative values.
• Enrich:
  • Since there was no accurate way to test the metrics without the executing policy name defined, this will still need validation in the future.
• Index
• Index summary
• Index recovery
• Ingest pipeline
• ML Job
• Node
• Node stats

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.

Related issues

Relates to #6618

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2023-07-06T07:43:42.235+0000

• Duration: 31 min 7 sec

Test stats 🧪

Test	Results
Failed	0
Passed	60
Skipped	0
Total	60

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

[klacabane]

enrich: Since there was no accurate way to test the metrics without the executing policy name defined, this will still need validation in the future.

Can we still use the value provided there even though it's not the right one ? What are the implications making the current value a blocker ?

[constanca-m]

Can we still use the value provided there even though it's not the right one ?

Yes, of course. I just could not filter the value by the policy name, that is why I wrote that is still pending validation.

What are the implications making the current value a blocker ?

Even if we set the metric type right away, the data stream cannot be migrated to TSDB because the dimensions are not defined. And the dimensions can only be set once we know the value of the policy name. @klacabane

[elasticmachine]

🌐 Coverage report

Name	Metrics % (covered/total)	Diff
Packages	100.0% (5/5)	💚
Files	100.0% (9/9)	💚
Classes	100.0% (9/9)	💚
Methods	87.5% (98/112)	👎 -12.5
Lines	91.98% (562/611)	👍 2.85
Conditionals	100.0% (0/0)	💚

[klacabane]

Please check:
elasticsearch.ccr.bytes_read: counter or gauge?
max_seq_no is a gauge - Correct?

elasticsearch.ccr.bytes_read: gauge
max_seq_no: gauge

And the dimensions can only be set once we know the value of the policy name

That's because the current value can be changing for a given policy, and we need a fixed one ?

[constanca-m]

elasticsearch.ccr.bytes_read: gauge
max_seq_no: gauge

Thank you.

That's because the current value can be changing for a given policy, and we need a fixed one ?

Because it is not the correct value. A dimension needs to uniquely identify each document per timestamp. From the enrich dataset, we know that the name of the executing policy should be unique. However, the value is not the expected name, but a different sentence. If this sentence is unique, then it would work as well for dimension, but it is not the expected one.

[klacabane]

LGTM, thanks for the change

[ritalwar]

LGTM!

[elasticmachine]

Package elasticsearch - 1.8.1 containing this change is available at https://epr.elastic.co/search?package=elasticsearch