[vSphere][datastorecluster] Add new Datastream Datastore Cluster by kush-elastic · Pull Request #11089 · elastic/integrations

kush-elastic · 2024-09-11T05:45:26Z

Description

Here are the following metrics to be added as part of new datastream datastore cluster in the vSphere Integration.

Metrics Type	Metrics	API	Field Mappings
Datastore cluster capacity	StoragePod.Summary.Capacity	Summary	capacity.bytes
	StoragePod.Summary.FreeSpace	Summary	free_space.bytes
Datastore cluster	childEntity(type= datastore)	Extra API	datastore.names
	childEntity(type= datastore)	Extra API	datastore.count

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.

TSDB

Values being used:
        elasticsearch_host = https://localhost:9200
        elasticsearch_ca_path = /root/.elastic-package/profiles/default/certs/elasticsearch/ca-cert.pem
        elasticsearch_user = elastic
        elasticsearch_pwd = changeme
        data_stream = metrics-vsphere.datastorecluster-default
        docs_index = First index of the data stream
        settings_mappings_index = Last index of the data stream
        max_docs = All documents
        get_overlapping_files = True
        directory_overlapping_files = overwritten-docs-metrics-vsphere.datastorecluster-default
        display_docs = 10
        copy_docs_per_dimension = 2

You're testing with version 8.15.2-SNAPSHOT.

Testing data stream metrics-vsphere.datastorecluster-default.
Index being used for the documents is .ds-metrics-vsphere.datastorecluster-default-2024.09.22-000001.
Index being used for the settings and mappings is .ds-metrics-vsphere.datastorecluster-default-2024.09.22-000001.

The time series fields for the TSDB index are: 
        - dimension (10 fields):
                - agent.id
                - cloud.account.id
                - cloud.availability_zone
                - cloud.instance.id
                - cloud.provider
                - cloud.region
                - container.id
                - host.name
                - service.address
                - vsphere.datastorecluster.name
        - gauge (3 fields):
                - vsphere.datastorecluster.capacity.bytes
                - vsphere.datastorecluster.datastore.count
                - vsphere.datastorecluster.free_space.bytes
        - routing_path (10 fields):
                - agent.id
                - cloud.account.id
                - cloud.availability_zone
                - cloud.instance.id
                - cloud.provider
                - cloud.region
                - container.id
                - host.name
                - service.address
                - vsphere.datastorecluster.name

Index tsdb-index-enabled successfully created.

Copying documents from .ds-metrics-vsphere.datastorecluster-default-2024.09.22-000001 to tsdb-index-enabled...
All 6 documents taken from index .ds-metrics-vsphere.datastorecluster-default-2024.09.22-000001 were successfully placed to index tsdb-index-enabled.

Related issues

Closes https://github.com/elastic/obs-integration-team/issues/45

elasticmachine · 2024-09-11T06:08:20Z

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

lucian-ioan

Add a changelog entry with version "1.14.0-next" as per this issue.

…nto 45-implement-the-change-to-datastore_cluster-datastream-in-integrations-using-the-snapshot-agent

willemdh · 2024-09-19T17:17:02Z

Is there support for Vsphere 8?

…nto 45-implement-the-change-to-datastore_cluster-datastream-in-integrations-using-the-snapshot-agent

kush-elastic · 2024-09-22T05:36:49Z

Is there support for Vsphere 8?

We have tested current integration with version 7.0.3, i cannot say confidently but it might just work fine with 8.0.0 as well.

niraj-elastic

LGTM

lucian-ioan

LGTM

…-in-integrations-using-the-snapshot-agent' of https://github.com/kush-elastic/integrations into 45-implement-the-change-to-datastore_cluster-datastream-in-integrations-using-the-snapshot-agent

ishleenk17

Approving, please address 1 pending comment.

elasticmachine · 2024-09-22T12:33:35Z

💚 Build Succeeded

Buildkite Build
Commit: 2151e37

History

💚 Build #16259 succeeded 2d702eb
💚 Build #16258 succeeded 68bc844
💔 Build #16253 failed 20c51ac
💔 Build #16196 failed daf72a4
💔 Build #16195 failed 09ed488
💔 Build #16176 failed 3c53304

cc @kush-elastic

elastic-sonarqube · 2024-09-22T12:33:39Z

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
97.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

elastic-vault-github-plugin-prod · 2024-09-26T14:30:32Z

Package vsphere - 1.15.0 containing this change is available at https://epr.elastic.co/search?package=vsphere

…stic#11089) * initial commit for datastore cluster datastream * resolve review comments * update changelog entry * update README.md * udpate README * update changelog entry * reorder changelog * fix changelog * udpate docker-compose * fix conflicts * fix description of period * update and fix system test for datastorecluster * update processor URL * remove agent fields

initial commit for datastore cluster datastream

45f3247

kush-elastic requested a review from a team as a code owner September 11, 2024 05:45

kush-elastic requested review from harnish-crest-data, ishleenk17, lucian-ioan and niraj-elastic September 11, 2024 05:45

kush-elastic self-assigned this Sep 11, 2024

andrewkroh added Integration:vsphere VMware vSphere Team:Obs-InfraObs Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations] labels Sep 11, 2024

kush-elastic added enhancement New feature or request Metricbeat and removed Team:Obs-InfraObs Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations] labels Sep 11, 2024

andrewkroh added the Team:Obs-InfraObs Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations] label Sep 11, 2024

lucian-ioan reviewed Sep 11, 2024

View reviewed changes

Comment thread packages/vsphere/data_stream/datastorecluster/fields/fields.yml Outdated

lucian-ioan reviewed Sep 11, 2024

View reviewed changes

Comment thread packages/vsphere/data_stream/datastorecluster/manifest.yml

lucian-ioan suggested changes Sep 11, 2024

View reviewed changes

niraj-elastic suggested changes Sep 18, 2024

View reviewed changes

Comment thread packages/vsphere/data_stream/datastorecluster/fields/fields.yml Outdated

Comment thread packages/vsphere/data_stream/datastorecluster/fields/fields.yml

Comment thread packages/vsphere/data_stream/datastorecluster/fields/agent.yml Outdated

kush-elastic added 3 commits September 19, 2024 16:14

resolve review comments

f237db8

Merge branch 'main' of https://github.com/kush-elastic/integrations i…

9cf1462

…nto 45-implement-the-change-to-datastore_cluster-datastream-in-integrations-using-the-snapshot-agent

update changelog entry

cbc91c1

ishleenk17 reviewed Sep 19, 2024

View reviewed changes

Comment thread packages/vsphere/_dev/build/docs/README.md Outdated

Comment thread packages/vsphere/_dev/build/docs/README.md Outdated

kush-elastic added 8 commits September 19, 2024 16:32

update README.md

2af88ca

udpate README

341c14f

update changelog entry

712e084

Merge branch 'main' of https://github.com/kush-elastic/integrations i…

c1daf33

…nto 45-implement-the-change-to-datastore_cluster-datastream-in-integrations-using-the-snapshot-agent

reorder changelog

8b50421

fix changelog

6283078

udpate docker-compose

502aea3

Merge branch 'main' of https://github.com/kush-elastic/integrations i…

3c53304

…nto 45-implement-the-change-to-datastore_cluster-datastream-in-integrations-using-the-snapshot-agent

ishleenk17 reviewed Sep 19, 2024

View reviewed changes

Comment thread packages/vsphere/_dev/build/docs/README.md Outdated

Comment thread packages/vsphere/_dev/deploy/docker/Dockerfile

Comment thread packages/vsphere/data_stream/datastorecluster/manifest.yml Outdated

niraj-elastic suggested changes Sep 19, 2024

View reviewed changes

Comment thread packages/vsphere/_dev/build/docs/README.md Outdated

kush-elastic added 2 commits September 19, 2024 22:29

fix conflicts

09ed488

Merge branch 'main' of https://github.com/kush-elastic/integrations i…

daf72a4

…nto 45-implement-the-change-to-datastore_cluster-datastream-in-integrations-using-the-snapshot-agent

kush-elastic added 3 commits September 21, 2024 19:08

Merge branch 'main' of https://github.com/kush-elastic/integrations i…

752584a

…nto 45-implement-the-change-to-datastore_cluster-datastream-in-integrations-using-the-snapshot-agent

fix description of period

20c51ac

update and fix system test for datastorecluster

68bc844

kush-elastic requested review from ishleenk17, lucian-ioan and niraj-elastic September 22, 2024 05:39

niraj-elastic approved these changes Sep 22, 2024

View reviewed changes

lucian-ioan approved these changes Sep 22, 2024

View reviewed changes

ishleenk17 reviewed Sep 22, 2024

View reviewed changes

Comment thread packages/vsphere/_dev/deploy/docker/docker-compose.yml

Comment thread packages/vsphere/data_stream/datastorecluster/fields/agent.yml Outdated

Comment thread packages/vsphere/data_stream/datastorecluster/manifest.yml Outdated

kush-elastic added 2 commits September 22, 2024 15:49

update processor URL

fbb3d14

Merge branch '45-implement-the-change-to-datastore_cluster-datastream…

2d702eb

…-in-integrations-using-the-snapshot-agent' of https://github.com/kush-elastic/integrations into 45-implement-the-change-to-datastore_cluster-datastream-in-integrations-using-the-snapshot-agent

ishleenk17 approved these changes Sep 22, 2024

View reviewed changes

remove agent fields

2151e37

kush-elastic merged commit 8966518 into elastic:main Sep 22, 2024

Navigation Menu

Uh oh!

[vSphere][datastorecluster] Add new Datastream Datastore Cluster - #11089

[vSphere][datastorecluster] Add new Datastream Datastore Cluster#11089
kush-elastic merged 20 commits into
elastic:mainfrom
kush-elastic:45-implement-the-change-to-datastore_cluster-datastream-in-integrations-using-the-snapshot-agent

kush-elastic commented Sep 11, 2024 •

edited

Loading

elasticmachine commented Sep 11, 2024

Uh oh!

Uh oh!

lucian-ioan left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

willemdh commented Sep 19, 2024

kush-elastic commented Sep 22, 2024

niraj-elastic left a comment

lucian-ioan left a comment

Uh oh!

Uh oh!

Uh oh!

ishleenk17 left a comment

elasticmachine commented Sep 22, 2024

elastic-sonarqube Bot commented Sep 22, 2024

elastic-vault-github-plugin-prod Bot commented Sep 26, 2024

Labels

7 participants

Uh oh!

Conversation

kush-elastic commented Sep 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist

TSDB

Related issues

elasticmachine commented Sep 11, 2024

🚀 Benchmarks report

Uh oh!

Uh oh!

lucian-ioan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

willemdh commented Sep 19, 2024

kush-elastic commented Sep 22, 2024

niraj-elastic left a comment

Choose a reason for hiding this comment

lucian-ioan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ishleenk17 left a comment

Choose a reason for hiding this comment

elasticmachine commented Sep 22, 2024

💚 Build Succeeded

History

elastic-sonarqube Bot commented Sep 22, 2024

Quality Gate passed

elastic-vault-github-plugin-prod Bot commented Sep 26, 2024

Labels

7 participants

kush-elastic commented Sep 11, 2024 •

edited

Loading