Skip to content

[vSphere] Filter alerts and warnings from triggered alarms. #11230

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
niraj-elastic merged 7 commits into from
Sep 25, 2024
Merged

[vSphere] Filter alerts and warnings from triggered alarms. #11230

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
e56c360
add alerts and warnings
niraj-elastic Sep 24, 2024
b9fdb69
update changelog
niraj-elastic Sep 24, 2024
ff6a115
remove null fields
niraj-elastic Sep 24, 2024
394e83e
update pipeline
niraj-elastic Sep 24, 2024
650c700
Merge branch 'main' of https://github.com/niraj-elastic/integrations …
niraj-crest Sep 24, 2024
0a81546
address review comments
niraj-elastic Sep 25, 2024
4ffe534
Merge branch 'vsphere_triggered_alarms' of https://github.com/niraj-e…
niraj-elastic Sep 25, 2024
File filter

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions packages/vsphere/changelog.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
# newer versions go on top
- version: "1.15.0-next"
changes:
- description: Filter alerts and warnings in all datastreams.
type: enhancement
link: https://github.com/elastic/integrations/pull/11230
- description: Fix issue with TSDB network.name.
type: bugfix
link: https://github.com/elastic/integrations/pull/11229
Expand Down
34 changes: 34 additions & 0 deletions packages/vsphere/data_stream/cluster/_dev/test/pipeline/test-cluster-metrics.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,40 @@
{
"vsphere": {
"cluster": {
"triggered_alarms": [
{
"status": "red",
"triggered_time": "2024-09-09T13:23:00.786Z",
"description": "Default alarm to monitor system boards. See the host's Hardware Status tab for more details.",
"entity_name": "121.0.0.0",
"name": "Host hardware system board status",
"id": "alarm-121.host-12"
},
{
"triggered_time": "2024-09-09T13:23:00.786Z",
"description": "Default alarm to monitor storage. See the host's Hardware Status tab for more details.",
"entity_name": "121.0.0.0",
"name": "Host storage status",
"id": "alarm-124.host-12",
"status": "red"
},
{
"entity_name": "121.0.0.0",
"name": "Host memory usage",
"id": "alarm-4.host-12",
"status": "yellow",
"triggered_time": "2024-08-28T10:31:26.621Z",
"description": "Default alarm to monitor host memory usage"
},
{
"name": "CPU Utilization",
"id": "alarm-703.host-12",
"status": "red",
"triggered_time": "2024-08-28T10:31:26.621Z",
"description": "",
"entity_name": "121.0.0.0"
}
],
"das_config": {
"admission": {
"control": {
Expand Down
45 changes: 45 additions & 0 deletions ...es/vsphere/data_stream/cluster/_dev/test/pipeline/test-cluster-metrics.json-expected.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,13 @@
},
"vsphere": {
"cluster": {
"alert": {
"names": [
"Host hardware system board status",
"Host storage status",
"CPU Utilization"
]
},
"das_config": {
"admission": {
"control": {
Expand Down Expand Up @@ -33,6 +40,44 @@
"names": [
"VM_Network_1"
]
},
"triggered_alarms": [
{
"description": "Default alarm to monitor system boards. See the host's Hardware Status tab for more details.",
"entity_name": "121.0.0.0",
"id": "alarm-121.host-12",
"name": "Host hardware system board status",
"status": "red",
"triggered_time": "2024-09-09T13:23:00.786Z"
},
{
"description": "Default alarm to monitor storage. See the host's Hardware Status tab for more details.",
"entity_name": "121.0.0.0",
"id": "alarm-124.host-12",
"name": "Host storage status",
"status": "red",
"triggered_time": "2024-09-09T13:23:00.786Z"
},
{
"description": "Default alarm to monitor host memory usage",
"entity_name": "121.0.0.0",
"id": "alarm-4.host-12",
"name": "Host memory usage",
"status": "yellow",
"triggered_time": "2024-08-28T10:31:26.621Z"
},
{
"entity_name": "121.0.0.0",
"id": "alarm-703.host-12",
"name": "CPU Utilization",
"status": "red",
"triggered_time": "2024-08-28T10:31:26.621Z"
}
],
"warning": {
"names": [
"Host memory usage"
]
}
}
}
Expand Down
41 changes: 41 additions & 0 deletions packages/vsphere/data_stream/cluster/elasticsearch/ingest_pipeline/default.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,47 @@ processors:
- set:
field: ecs.version
value: '8.11.0'
- script:
if: ctx.vsphere?.cluster?.triggered_alarms != null
lang: painless
source: >
def alerts = [];
def warnings = [];
for (alarm in ctx.vsphere.cluster.triggered_alarms) {
if (alarm.status == 'red') {
alerts.add(alarm.name);
}
if (alarm.status == 'yellow') {
warnings.add(alarm.name);
}
}
ctx.alerts = alerts;
ctx.warnings = warnings;
- rename:
field: alerts
target_field: vsphere.cluster.alert.names
ignore_missing: true
- rename:
field: warnings
target_field: vsphere.cluster.warning.names
ignore_missing: true
- script:
description: "Drops null/empty values recursively."
lang: painless
source: |
boolean drop(Object o) {
if (o == null || o == "") {
return true;
} else if (o instanceof Map) {
((Map) o).values().removeIf(v -> drop(v));
return (((Map) o).size() == 0);
} else if (o instanceof List) {
((List) o).removeIf(v -> drop(v));
return (((List) o).size() == 0);
}
return false;
}
drop(ctx);
on_failure:
- set:
field: error.message
Expand Down
10 changes: 9 additions & 1 deletion packages/vsphere/data_stream/cluster/fields/fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@
type: group
description: Cluster information.
fields:
- name: alert.names
type: keyword
description: >
List of all the alerts on this Cluster.
- name: das_config
type: group
fields:
Expand Down Expand Up @@ -77,4 +81,8 @@
description: Status of the alarm.
- name: triggered_time
type: date
description: Time when the alarm was triggered.
description: Time when the alarm was triggered.
- name: warning.names
type: keyword
description: >
List of all the warnings on this Cluster.
120 changes: 77 additions & 43 deletions packages/vsphere/data_stream/datastore/_dev/test/pipeline/test-datastore.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,55 +1,89 @@
{
"events": [
{
"vsphere": {
"datastore": {
"disk": {
"capacity": {
"usage": {
{
"vsphere": {
"datastore": {
"triggered_alarms": [
{
"status": "red",
"triggered_time": "2024-09-09T13:23:00.786Z",
"description": "Default alarm to monitor system boards. See the host's Hardware Status tab for more details.",
"entity_name": "121.0.0.0",
"name": "Host hardware system board status",
"id": "alarm-121.host-12"
},
{
"triggered_time": "2024-09-09T13:23:00.786Z",
"description": "Default alarm to monitor storage. See the host's Hardware Status tab for more details.",
"entity_name": "121.0.0.0",
"name": "Host storage status",
"id": "alarm-124.host-12",
"status": "red"
},
{
"entity_name": "121.0.0.0",
"name": "Host memory usage",
"id": "alarm-4.host-12",
"status": "yellow",
"triggered_time": "2024-08-28T10:31:26.621Z",
"description": "Default alarm to monitor host memory usage"
},
{
"name": "CPU Utilization",
"id": "alarm-703.host-12",
"status": "red",
"triggered_time": "2024-08-28T10:31:26.621Z",
"description": "",
"entity_name": "121.0.0.0"
}
],
"disk": {
"capacity": {
"usage": {
"bytes": 520505786368
},
"bytes": 1610344300544
},
"provisioned": {
"bytes": 520505786368
}
},
"capacity": {
"free": {
"bytes": 37120094208
},
"total": {
"bytes": 74686664704
},
"bytes": 1610344300544
"used": {
"bytes": 37566570496,
"pct": 0.502988996026061
}
},
"provisioned": {
"bytes": 520505786368
}
},
"capacity": {
"free": {
"bytes": 37120094208
"fstype": "OTHER",
"host": {
"count": 1,
"names": "DC0_H0"
},
"name": "LocalDS_0",
"read": {
"bytes": 1024
},
"total": {
"bytes": 74686664704
"status": "green",
"vm": {
"count": 4,
"names": [
"DC0_C0_RP0_VM0",
"DC0_C0_RP0_VM1",
"DC0_H0_VM0",
"DC0_H0_VM1"
]
},
"used": {
"bytes": 37566570496,
"pct": 0.502988996026061
"write": {
"bytes": 450560
}
},
"fstype": "OTHER",
"host": {
"count": 1,
"names": "DC0_H0"
},
"name": "LocalDS_0",
"read": {
"bytes": 1024
},
"status": "green",
"vm": {
"count": 4,
"names": [
"DC0_C0_RP0_VM0",
"DC0_C0_RP0_VM1",
"DC0_H0_VM0",
"DC0_H0_VM1"
]
},
"write": {
"bytes": 450560
}
}
}
}
]
]
}
45 changes: 45 additions & 0 deletions packages/vsphere/data_stream/datastore/_dev/test/pipeline/test-datastore.json-expected.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,13 @@
},
"vsphere": {
"datastore": {
"alert": {
"names": [
"Host hardware system board status",
"Host storage status",
"CPU Utilization"
]
},
"capacity": {
"free": {
"bytes": 37120094208
Expand Down Expand Up @@ -39,6 +46,39 @@
"bytes": 1024
},
"status": "green",
"triggered_alarms": [
{
"description": "Default alarm to monitor system boards. See the host's Hardware Status tab for more details.",
"entity_name": "121.0.0.0",
"id": "alarm-121.host-12",
"name": "Host hardware system board status",
"status": "red",
"triggered_time": "2024-09-09T13:23:00.786Z"
},
{
"description": "Default alarm to monitor storage. See the host's Hardware Status tab for more details.",
"entity_name": "121.0.0.0",
"id": "alarm-124.host-12",
"name": "Host storage status",
"status": "red",
"triggered_time": "2024-09-09T13:23:00.786Z"
},
{
"description": "Default alarm to monitor host memory usage",
"entity_name": "121.0.0.0",
"id": "alarm-4.host-12",
"name": "Host memory usage",
"status": "yellow",
"triggered_time": "2024-08-28T10:31:26.621Z"
},
{
"entity_name": "121.0.0.0",
"id": "alarm-703.host-12",
"name": "CPU Utilization",
"status": "red",
"triggered_time": "2024-08-28T10:31:26.621Z"
}
],
"vm": {
"count": 4,
"names": [
Expand All @@ -48,6 +88,11 @@
"DC0_H0_VM1"
]
},
"warning": {
"names": [
"Host memory usage"
]
},
"write": {
"bytes": 450560
}
Expand Down
Loading