Skip to content

[vSphere][virtualmachine] Add support for additional metrics #10942

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
lucian-ioan merged 23 commits into from
Sep 20, 2024
Merged

[vSphere][virtualmachine] Add support for additional metrics #10942

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
8e782c6
add additional fields
lucian-ioan Aug 30, 2024
3e86570
revert fields metric types
Sep 2, 2024
260ef06
redo system test
Sep 2, 2024
953b653
add changelog entry
Sep 2, 2024
e1af4af
add snapshot field
lucian-ioan Sep 18, 2024
56c61bd
use group for host
lucian-ioan Sep 18, 2024
28978e0
update info field
lucian-ioan Sep 18, 2024
6c72eb4
add triggered alarms
lucian-ioan Sep 18, 2024
9d3b1a0
update manifest file
lucian-ioan Sep 18, 2024
3f909da
Merge branch 'main' into update_vsphere_virtual_machine_integration
lucian-ioan Sep 18, 2024
a782210
add more metric types
lucian-ioan Sep 18, 2024
8e01de8
change snapshot info to group
lucian-ioan Sep 18, 2024
d6b9916
update readme
lucian-ioan Sep 18, 2024
eca6813
update type
lucian-ioan Sep 18, 2024
649c864
default period description
lucian-ioan Sep 19, 2024
d2f467e
update version to 1.15.0-next
lucian-ioan Sep 19, 2024
6b27dc4
update triggered alarms
lucian-ioan Sep 19, 2024
f546aa8
add system tests
lucian-ioan Sep 19, 2024
9c318ec
Merge remote-tracking branch 'origin' into update_vsphere_virtual_mac…
lucian-ioan Sep 19, 2024
75b3a08
update readme
lucian-ioan Sep 19, 2024
4aa0c34
add tag in stream.yml.hbs
lucian-ioan Sep 19, 2024
ba0636d
update IPs and domains
lucian-ioan Sep 20, 2024
a009454
Merge branch 'main' into update_vsphere_virtual_machine_integration
lucian-ioan Sep 20, 2024
File filter

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions packages/vsphere/changelog.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
# newer versions go on top
- version: "1.15.0-next"
changes:
- description: Add additional metrics for virtualmachine datastream
type: enhancement
link: https://github.com/elastic/integrations/pull/10942
- description: Add resourcepool datastream.
type: enhancement
link: https://github.com/elastic/integrations/pull/10996
Expand Down
143 changes: 143 additions & 0 deletions packages/vsphere/data_stream/virtualmachine/_dev/test/pipeline/test-virtualmachine.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
{
"events": [
{
"@timestamp": "2024-09-02T09:44:14.128Z",
"agent": {
"ephemeral_id": "2d60906e-b972-4981-a356-c57ccb79108b",
"id": "8ea73fb3-a0a0-4270-aad6-e88edb8b385a",
"name": "elastic-agent-55444",
"type": "metricbeat",
"version": "8.16.0"
},
"data_stream": {
"dataset": "vsphere.virtualmachine",
"namespace": "64133",
"type": "metrics"
},
"ecs": {
"version": "8.11.0"
},
"elastic_agent": {
"id": "8ea73fb3-a0a0-4270-aad6-e88edb8b385a",
"snapshot": true,
"version": "8.16.0"
},
"event": {
"agent_id_status": "verified",
"dataset": "vsphere.virtualmachine",
"duration": 52726937,
"ingested": "2024-09-02T09:44:17Z",
"module": "vsphere"
},
"host": {
"architecture": "x86_64",
"containerized": true,
"hostname": "elastic-agent-55444",
"id": "e744630f9d4f43dc818e497d221bd0b2",
"ip": [
"172.18.0.4",
"172.20.0.2"
],
"mac": [
"02-42-AC-12-00-04",
"02-42-AC-14-00-02"
],
"name": "elastic-agent-55444",
"os": {
"codename": "focal",
"family": "debian",
"kernel": "5.15.153.1-microsoft-standard-WSL2",
"name": "Ubuntu",
"platform": "ubuntu",
"type": "linux",
"version": "20.04.6 LTS (Focal Fossa)"
}
},
"metricset": {
"name": "virtualmachine",
"period": 10000
},
"service": {
"address": "https://svc-vsphere-metrics:8989/sdk",
"type": "vsphere"
},
"vsphere": {
"virtualmachine": {
"name": "xt0nmfpv9",
"uptime": 5348978,
"status": "green",
"host": {
"id": "host-32",
"hostname": "phx-w1c1-esxi04.com"
},
"cpu": {
"free": {
"mhz": 0
},
"used": {
"mhz": 161
},
"total": {
"mhz": 0
}
},
"network": {
"names": [
"PROD_VCF_VMS"
],
"count": 1
},
"memory": {
"used": {
"guest": {
"bytes": 686817280
},
"host": {
"bytes": 29027729408
}
},
"total": {
"guest": {
"bytes": 68719476736
}
},
"free": {
"guest": {
"bytes": 68032659456
}
}
},
"network_names": [
"PROD_VCF_VMS"
],
"datastore": {
"count": 1,
"names": [
"VxRailtoup-Virtual-Datastore-bc1d-5aa310fb"
]
},
"os": "CentOS 4/5/6/7 (64-bit)",
"snapshot": {
"info": [
{
"id": 1,
"name": "VM Snapshot 7%2f3%2f2024, 4:01:21 PM",
"description": "Created to demo",
"createtime": "2024-07-03T20:01:34.329Z",
"state": "poweredOn"
},
{
"createtime": "2024-07-05T23:35:40.859Z",
"state": "poweredOn",
"id": 2,
"name": "VM Snapshot 7%2f5%2f2024, 7:35:37 PM",
"description": "backup"
}
],
"count": 2
}
}
}
}
]
}
143 changes: 143 additions & 0 deletions ...here/data_stream/virtualmachine/_dev/test/pipeline/test-virtualmachine.json-expected.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
{
"expected": [
{
"@timestamp": "2024-09-02T09:44:14.128Z",
"agent": {
"ephemeral_id": "2d60906e-b972-4981-a356-c57ccb79108b",
"id": "8ea73fb3-a0a0-4270-aad6-e88edb8b385a",
"name": "elastic-agent-55444",
"type": "metricbeat",
"version": "8.16.0"
},
"data_stream": {
"dataset": "vsphere.virtualmachine",
"namespace": "64133",
"type": "metrics"
},
"ecs": {
"version": "8.11.0"
},
"elastic_agent": {
"id": "8ea73fb3-a0a0-4270-aad6-e88edb8b385a",
"snapshot": true,
"version": "8.16.0"
},
"event": {
"agent_id_status": "verified",
"dataset": "vsphere.virtualmachine",
"duration": 52726937,
"ingested": "2024-09-02T09:44:17Z",
"module": "vsphere"
},
"host": {
"architecture": "x86_64",
"containerized": true,
"hostname": "elastic-agent-55444",
"id": "e744630f9d4f43dc818e497d221bd0b2",
"ip": [
"172.18.0.4",
Comment thread
ishleenk17 marked this conversation as resolved.
"172.20.0.2"
],
"mac": [
"02-42-AC-12-00-04",
"02-42-AC-14-00-02"
],
"name": "elastic-agent-55444",
"os": {
"codename": "focal",
"family": "debian",
"kernel": "5.15.153.1-microsoft-standard-WSL2",
"name": "Ubuntu",
"platform": "ubuntu",
"type": "linux",
"version": "20.04.6 LTS (Focal Fossa)"
}
},
"metricset": {
"name": "virtualmachine",
"period": 10000
},
"service": {
"address": "https://svc-vsphere-metrics:8989/sdk",
"type": "vsphere"
},
"vsphere": {
"virtualmachine": {
"cpu": {
"free": {
"mhz": 0
},
"total": {
"mhz": 0
},
"used": {
"mhz": 161
}
},
"datastore": {
"count": 1,
"names": [
"VxRailtoup-Virtual-Datastore-bc1d-5aa310fb"
]
},
"host": {
"hostname": "phx-w1c1-esxi04.com",
"id": "host-32"
},
"memory": {
"free": {
"guest": {
"bytes": 68032659456
}
},
"total": {
"guest": {
"bytes": 68719476736
}
},
"used": {
"guest": {
"bytes": 686817280
},
"host": {
"bytes": 29027729408
}
}
},
"name": "xt0nmfpv9",
"network": {
"count": 1,
"names": [
"PROD_VCF_VMS"
]
},
"network_names": [
"PROD_VCF_VMS"
],
"os": "CentOS 4/5/6/7 (64-bit)",
"snapshot": {
"count": 2,
"info": [
{
"createtime": "2024-07-03T20:01:34.329Z",
"description": "Created to demo",
"id": 1,
"name": "VM Snapshot 7%2f3%2f2024, 4:01:21 PM",
"state": "poweredOn"
},
{
"createtime": "2024-07-05T23:35:40.859Z",
"description": "backup",
"id": 2,
"name": "VM Snapshot 7%2f5%2f2024, 7:35:37 PM",
"state": "poweredOn"
}
]
},
"status": "green",
"uptime": 5348978
}
}
}
]
}
8 changes: 7 additions & 1 deletion packages/vsphere/data_stream/virtualmachine/agent/stream/stream.yml.hbs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,15 @@ username: {{username}}
password: {{password}}
# If insecure is true, don't verify the server's certificate chain
insecure: {{insecure}}

tags:
{{#each tags as |tag|}}
- {{tag}}
{{/each}}

# Get custom fields when using virtualmachine metric set. Default false.
get_custom_fields: {{get_custom_fields}}
{{#if processors}}
processors:
{{processors}}
{{/if}}
{{/if}}
28 changes: 23 additions & 5 deletions packages/vsphere/data_stream/virtualmachine/fields/fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -134,8 +134,26 @@
type: long
metric_type: gauge
description: The number of snapshots of this virtualmachine.
- name: triggerd_alarms.*
type: object
object_type: keyword
description: >-
List of all the triggerd alarms.
- name: triggered_alarms
type: group
description: >
List of all the triggered alarms.
fields:
- name: status
type: keyword
description: Status of the alarm.
- name: triggered_time
type: date
description: Time when the alarm was triggered.
- name: description
type: keyword
description: Description of the alarm.
- name: entity_name
type: keyword
description: Name of the entity associated with the alarm.
- name: name
type: keyword
description: Name of the alarm.
- name: id
type: keyword
description: Unique identifier for the alarm.
13 changes: 13 additions & 0 deletions packages/vsphere/data_stream/virtualmachine/manifest.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,19 @@ streams:
type: bool
multi: false
default: false
- name: period
type: text
title: Period
description: The period for fetching metrics, e.g., every 1s, 1m, 1h.
default: 20s
- name: tags
type: text
title: Tags
multi: true
required: true
show_user: false
default:
- vsphere-virtualmachine
- name: processors
type: yaml
title: Processors
Expand Down
Loading