Skip to content

[Kibana] Add condition configuration for Kibana logs and metrics - #5359

Merged
klacabane merged 4 commits into
elastic:mainfrom
BenB196:add-kibana-condition
Mar 7, 2023
Merged

[Kibana] Add condition configuration for Kibana logs and metrics#5359
klacabane merged 4 commits into
elastic:mainfrom
BenB196:add-kibana-condition

Conversation

@BenB196

@BenB196 BenB196 commented Feb 22, 2023 •
edited
Loading

Copy link
Copy Markdown
Contributor

Enhancement

What does this PR do?

  1. Add the ability to set the condition option to both Kibana Logs and Metrics inputs

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

  • condition is a top level of both inputs as I couldn't think of a reason where a user would want conditions to be different based on what is being collected within an input, and this greatly simplifies things for users.
  • The examples below have version numbers that don't directly match this PR, it was the only way I could get this integration to show up in Kibana for testing purposes.
Before Upgrade 
inputs:
  - id: logfile-kibana-8db829a3-4ae0-4bad-8851-4b652c5c0bf0
    name: kibana-1
    revision: 1
    type: logfile
    use_output: default
    meta:
      package:
        name: kibana
        version: 2.3.0
    data_stream:
      namespace: default
    package_policy_id: 8db829a3-4ae0-4bad-8851-4b652c5c0bf0
    streams:
      - id: logfile-kibana.audit-8db829a3-4ae0-4bad-8851-4b652c5c0bf0
        data_stream:
          dataset: kibana.audit
          type: logs
        paths:
          - /var/log/kibana/*_audit.json
        exclude_files:
          - .gz$
      - id: logfile-kibana.log-8db829a3-4ae0-4bad-8851-4b652c5c0bf0
        data_stream:
          dataset: kibana.log
          type: logs
        paths:
          - /var/log/kibana/kibana.stdout
        exclude_files:
          - .gz$
  - id: kibana/metrics-kibana-8db829a3-4ae0-4bad-8851-4b652c5c0bf0
    name: kibana-1
    revision: 1
    type: kibana/metrics
    use_output: default
    meta:
      package:
        name: kibana
        version: 2.3.0
    data_stream:
      namespace: default
    package_policy_id: 8db829a3-4ae0-4bad-8851-4b652c5c0bf0
    streams:
      - id: >-
          kibana/metrics-kibana.stack_monitoring.cluster_actions-8db829a3-4ae0-4bad-8851-4b652c5c0bf0
        data_stream:
          dataset: kibana.stack_monitoring.cluster_actions
          type: metrics
        metricsets:
          - cluster_actions
        hosts:
          - 'http://localhost:5601'
        period: 10s
        ssl: null
      - id: >-
          kibana/metrics-kibana.stack_monitoring.cluster_rules-8db829a3-4ae0-4bad-8851-4b652c5c0bf0
        data_stream:
          dataset: kibana.stack_monitoring.cluster_rules
          type: metrics
        metricsets:
          - cluster_rules
        hosts:
          - 'http://localhost:5601'
        period: 10s
        ssl: null
      - id: >-
          kibana/metrics-kibana.stack_monitoring.node_actions-8db829a3-4ae0-4bad-8851-4b652c5c0bf0
        data_stream:
          dataset: kibana.stack_monitoring.node_actions
          type: metrics
        metricsets:
          - node_actions
        hosts:
          - 'http://localhost:5601'
        period: 10s
        ssl: null
      - id: >-
          kibana/metrics-kibana.stack_monitoring.node_rules-8db829a3-4ae0-4bad-8851-4b652c5c0bf0
        data_stream:
          dataset: kibana.stack_monitoring.node_rules
          type: metrics
        metricsets:
          - node_rules
        hosts:
          - 'http://localhost:5601'
        period: 10s
        ssl: null
      - id: >-
          kibana/metrics-kibana.stack_monitoring.stats-8db829a3-4ae0-4bad-8851-4b652c5c0bf0
        data_stream:
          dataset: kibana.stack_monitoring.stats
          type: metrics
        metricsets:
          - stats
        hosts:
          - 'http://localhost:5601'
        period: 10s
        ssl: null
      - id: >-
          kibana/metrics-kibana.stack_monitoring.status-8db829a3-4ae0-4bad-8851-4b652c5c0bf0
        data_stream:
          dataset: kibana.stack_monitoring.status
          type: metrics
        metricsets:
          - status
        hosts:
          - 'http://localhost:5601'
        period: 10s
        ssl: null
After Upgrade (No changes) 
inputs:
  - id: logfile-kibana-8db829a3-4ae0-4bad-8851-4b652c5c0bf0
    name: kibana-1
    revision: 2
    type: logfile
    use_output: default
    meta:
      package:
        name: kibana
        version: 2.4.0
    data_stream:
      namespace: default
    package_policy_id: 8db829a3-4ae0-4bad-8851-4b652c5c0bf0
    streams:
      - id: logfile-kibana.audit-8db829a3-4ae0-4bad-8851-4b652c5c0bf0
        data_stream:
          dataset: kibana.audit
          type: logs
        exclude_files:
          - .gz$
        paths:
          - /var/log/kibana/*_audit.json
      - id: logfile-kibana.log-8db829a3-4ae0-4bad-8851-4b652c5c0bf0
        data_stream:
          dataset: kibana.log
          type: logs
        exclude_files:
          - .gz$
        paths:
          - /var/log/kibana/kibana.stdout
  - id: kibana/metrics-kibana-8db829a3-4ae0-4bad-8851-4b652c5c0bf0
    name: kibana-1
    revision: 2
    type: kibana/metrics
    use_output: default
    meta:
      package:
        name: kibana
        version: 2.4.0
    data_stream:
      namespace: default
    package_policy_id: 8db829a3-4ae0-4bad-8851-4b652c5c0bf0
    streams:
      - id: >-
          kibana/metrics-kibana.stack_monitoring.cluster_actions-8db829a3-4ae0-4bad-8851-4b652c5c0bf0
        data_stream:
          dataset: kibana.stack_monitoring.cluster_actions
          type: metrics
        period: 10s
        hosts:
          - 'http://localhost:5601'
        metricsets:
          - cluster_actions
        ssl: null
      - id: >-
          kibana/metrics-kibana.stack_monitoring.cluster_rules-8db829a3-4ae0-4bad-8851-4b652c5c0bf0
        data_stream:
          dataset: kibana.stack_monitoring.cluster_rules
          type: metrics
        period: 10s
        hosts:
          - 'http://localhost:5601'
        metricsets:
          - cluster_rules
        ssl: null
      - id: >-
          kibana/metrics-kibana.stack_monitoring.node_actions-8db829a3-4ae0-4bad-8851-4b652c5c0bf0
        data_stream:
          dataset: kibana.stack_monitoring.node_actions
          type: metrics
        period: 10s
        hosts:
          - 'http://localhost:5601'
        metricsets:
          - node_actions
        ssl: null
      - id: >-
          kibana/metrics-kibana.stack_monitoring.node_rules-8db829a3-4ae0-4bad-8851-4b652c5c0bf0
        data_stream:
          dataset: kibana.stack_monitoring.node_rules
          type: metrics
        period: 10s
        hosts:
          - 'http://localhost:5601'
        metricsets:
          - node_rules
        ssl: null
      - id: >-
          kibana/metrics-kibana.stack_monitoring.stats-8db829a3-4ae0-4bad-8851-4b652c5c0bf0
        data_stream:
          dataset: kibana.stack_monitoring.stats
          type: metrics
        period: 10s
        hosts:
          - 'http://localhost:5601'
        metricsets:
          - stats
        ssl: null
      - id: >-
          kibana/metrics-kibana.stack_monitoring.status-8db829a3-4ae0-4bad-8851-4b652c5c0bf0
        data_stream:
          dataset: kibana.stack_monitoring.status
          type: metrics
        period: 10s
        hosts:
          - 'http://localhost:5601'
        metricsets:
          - status
        ssl: null
After Upgrade (Conditions Added) 
inputs:
  - id: logfile-kibana-8db829a3-4ae0-4bad-8851-4b652c5c0bf0
    name: kibana-1
    revision: 3
    type: logfile
    use_output: default
    meta:
      package:
        name: kibana
        version: 2.4.0
    data_stream:
      namespace: default
    package_policy_id: 8db829a3-4ae0-4bad-8851-4b652c5c0bf0
    streams:
      - id: logfile-kibana.audit-8db829a3-4ae0-4bad-8851-4b652c5c0bf0
        data_stream:
          dataset: kibana.audit
          type: logs
        exclude_files:
          - .gz$
        condition: k8s.labels == 'abc'
        paths:
          - /var/log/kibana/*_audit.json
      - id: logfile-kibana.log-8db829a3-4ae0-4bad-8851-4b652c5c0bf0
        data_stream:
          dataset: kibana.log
          type: logs
        exclude_files:
          - .gz$
        condition: k8s.labels == 'abc'
        paths:
          - /var/log/kibana/kibana.stdout
  - id: kibana/metrics-kibana-8db829a3-4ae0-4bad-8851-4b652c5c0bf0
    name: kibana-1
    revision: 3
    type: kibana/metrics
    use_output: default
    meta:
      package:
        name: kibana
        version: 2.4.0
    data_stream:
      namespace: default
    package_policy_id: 8db829a3-4ae0-4bad-8851-4b652c5c0bf0
    streams:
      - id: >-
          kibana/metrics-kibana.stack_monitoring.cluster_actions-8db829a3-4ae0-4bad-8851-4b652c5c0bf0
        data_stream:
          dataset: kibana.stack_monitoring.cluster_actions
          type: metrics
        period: 10s
        condition: k8s.labels == 'abc'
        hosts:
          - 'http://localhost:5601'
        metricsets:
          - cluster_actions
        ssl: null
      - id: >-
          kibana/metrics-kibana.stack_monitoring.cluster_rules-8db829a3-4ae0-4bad-8851-4b652c5c0bf0
        data_stream:
          dataset: kibana.stack_monitoring.cluster_rules
          type: metrics
        period: 10s
        condition: k8s.labels == 'abc'
        hosts:
          - 'http://localhost:5601'
        metricsets:
          - cluster_rules
        ssl: null
      - id: >-
          kibana/metrics-kibana.stack_monitoring.node_actions-8db829a3-4ae0-4bad-8851-4b652c5c0bf0
        data_stream:
          dataset: kibana.stack_monitoring.node_actions
          type: metrics
        period: 10s
        condition: k8s.labels == 'abc'
        hosts:
          - 'http://localhost:5601'
        metricsets:
          - node_actions
        ssl: null
      - id: >-
          kibana/metrics-kibana.stack_monitoring.node_rules-8db829a3-4ae0-4bad-8851-4b652c5c0bf0
        data_stream:
          dataset: kibana.stack_monitoring.node_rules
          type: metrics
        period: 10s
        condition: k8s.labels == 'abc'
        hosts:
          - 'http://localhost:5601'
        metricsets:
          - node_rules
        ssl: null
      - id: >-
          kibana/metrics-kibana.stack_monitoring.stats-8db829a3-4ae0-4bad-8851-4b652c5c0bf0
        data_stream:
          dataset: kibana.stack_monitoring.stats
          type: metrics
        period: 10s
        condition: k8s.labels == 'abc'
        hosts:
          - 'http://localhost:5601'
        metricsets:
          - stats
        ssl: null
      - id: >-
          kibana/metrics-kibana.stack_monitoring.status-8db829a3-4ae0-4bad-8851-4b652c5c0bf0
        data_stream:
          dataset: kibana.stack_monitoring.status
          type: metrics
        period: 10s
        condition: k8s.labels == 'abc'
        hosts:
          - 'http://localhost:5601'
        metricsets:
          - status
        ssl: null

How to test this PR locally

  1. Install current integration version
  2. Upgrade integration to this PR
  3. Add/Remove condition values.

Related issues

Screenshots

image

image
@BenB196
BenB196 requested a review from a team as a code owner February 22, 2023 17:56
@elasticmachine

elasticmachine commented Feb 22, 2023 •
edited
Loading

Copy link
Copy Markdown

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2023-03-06T17:14:25.446+0000

  • Duration: 29 min 57 sec

Test stats 🧪

Test Results
Failed 0
Passed 28
Skipped 0
Total 28

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.
@BenB196 BenB196 changed the title Add condition configuration for logs and metrics Feb 22, 2023
@BenB196 BenB196 changed the title Add condition configuration for Kibana logs and metrics Feb 22, 2023
klacabane
klacabane reviewed Mar 6, 2023
View reviewed changes
Comment thread packages/kibana/manifest.yml Outdated
klacabane
klacabane reviewed Mar 6, 2023
View reviewed changes
Comment thread packages/kibana/manifest.yml Outdated
@klacabane

klacabane commented Mar 6, 2023

Copy link
Copy Markdown
Contributor

/test
@klacabane klacabane added Integration:kibana Kibana Team:Infra Monitoring UI - DEPRECATED Label for the Infrastructure Monitoring UI team. - DEPRECATED - Use Team:obs-ux-infra_services >enhancement labels Mar 6, 2023
@elasticmachine

elasticmachine commented Mar 6, 2023

Copy link
Copy Markdown

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (2/2) 💚
Files 100.0% (4/4) 💚
Classes 100.0% (4/4) 💚
Methods 94.872% (37/39) 👍 69.872
Lines 87.234% (82/94) 👎 -12.766
Conditionals 100.0% (0/0) 💚
BenB196 and others added 2 commits March 6, 2023 11:45
@BenB196 @klacabane
Update packages/kibana/manifest.yml
a303f1b 
Co-authored-by: Kevin Lacabane <klacabane@gmail.com>
@BenB196 @klacabane
Update packages/kibana/manifest.yml
0594e6d 
Co-authored-by: Kevin Lacabane <klacabane@gmail.com>
klacabane
klacabane approved these changes Mar 6, 2023
View reviewed changes

@klacabane klacabane left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you for the change
@klacabane

klacabane commented Mar 6, 2023

Copy link
Copy Markdown
Contributor

/test
@klacabane
klacabane merged commit 69aa2ac into elastic:main Mar 7, 2023
@elasticmachine

elasticmachine commented Mar 7, 2023

Copy link
Copy Markdown

Package kibana - 2.3.0-preview1 containing this change is available at https://epr.elastic.co/search?package=kibana
@BenB196 BenB196 mentioned this pull request Mar 16, 2023
Merged
4 tasks
agithomas pushed a commit to agithomas/integrations that referenced this pull request Mar 20, 2023
@BenB196 @klacabane @agithomas
[Kibana] Add condition configuration for Kibana logs and metrics (e…
4149833 
…lastic#5359)

* Add `condition` configuration for logs and metrics

* Update changelog.yml

* Update packages/kibana/manifest.yml

Co-authored-by: Kevin Lacabane <klacabane@gmail.com>

* Update packages/kibana/manifest.yml

Co-authored-by: Kevin Lacabane <klacabane@gmail.com>

---------

Co-authored-by: Kevin Lacabane <klacabane@gmail.com>
agithomas pushed a commit to agithomas/integrations that referenced this pull request Mar 21, 2023
@BenB196 @klacabane @agithomas
[Kibana] Add condition configuration for Kibana logs and metrics (e…
4bd11de 
…lastic#5359)

* Add `condition` configuration for logs and metrics

* Update changelog.yml

* Update packages/kibana/manifest.yml

Co-authored-by: Kevin Lacabane <klacabane@gmail.com>

* Update packages/kibana/manifest.yml

Co-authored-by: Kevin Lacabane <klacabane@gmail.com>

---------

Co-authored-by: Kevin Lacabane <klacabane@gmail.com>
@BenB196
BenB196 deleted the add-kibana-condition branch November 29, 2023 12:37
orestisfl pushed a commit to orestisfl/integrations that referenced this pull request May 15, 2026
@BenB196 @klacabane
[Kibana] Add condition configuration for Kibana logs and metrics (e…
162abb5 
…lastic#5359)

* Add `condition` configuration for logs and metrics

* Update changelog.yml

* Update packages/kibana/manifest.yml

Co-authored-by: Kevin Lacabane <klacabane@gmail.com>

* Update packages/kibana/manifest.yml

Co-authored-by: Kevin Lacabane <klacabane@gmail.com>

---------

Co-authored-by: Kevin Lacabane <klacabane@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

>enhancement Integration:kibana Kibana Team:Infra Monitoring UI - DEPRECATED Label for the Infrastructure Monitoring UI team. - DEPRECATED - Use Team:obs-ux-infra_services

3 participants

@BenB196 @elasticmachine @klacabane