Sql Input Package by ishleenk17 · Pull Request #5167 · elastic/integrations

ishleenk17 · 2023-02-02T06:11:55Z

Enhancement

What does this PR do?

This PR creates the SQL Input Package which can be used to retrieve metrics by querying any SQL database.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.

How to test this PR locally

Install the input package through Kibana. Bring up any SQL database. Specify the configurations as mentioned in the README. The metrics should be visible in Kibana

Related issues

Closes Build sql input package #5078

Screenshots

Configuration for Input Package

Metrics Retrieval for Mysql database using SQL input Package

This reverts commit 94487c3.

elasticmachine · 2023-02-02T06:17:10Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2023-02-15T07:13:15.795+0000
Duration: 13 min 42 sec

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

lalit-satapathy

An ideal scenario for generic sql input is driver is selected by a drop-down list and it provides a sample DSN notation for the driver pre-populated. May be our UI does not support such selections for now?

agithomas · 2023-02-08T05:03:39Z

Can you please verify if the below test case is passed? This is because, i do not find any test cases attached with this PR.

Queries having special characters such as quotes, double quotes, special characters. Please verify if the query is fetching results.

Can there be preconditions / troubleshooting section added such as

Ensure that query does not end with semi colon
Ensure correctness of the query before executing
Ensure the database user executing the database query has the required permissions
For oracle, you can consider adding a link to the Oracle integration documentation to find out the pre-requisite. This includes setting of certain environment variables, installation of client
Recommend to change the text Hosts in Manifest.yml to Host / Connection String

agithomas

Review comments added

agithomas · 2023-02-08T05:25:59Z

In the screenshot, the name sql_ishleen is found. Please change it to a more appropriate name suiting the documentation bestpractices

agithomas · 2023-02-08T05:29:24Z

Are there plans to include system testing and it's files & configs as part of this PR?

ishleenk17 · 2023-02-08T05:35:40Z

Can you please verify if the below test case is passed? This is because, i do not find any test cases attached with this PR.

Queries having special characters such as quotes, double quotes, special characters. Please verify if the query is fetching results.

Can there be preconditions / troubleshooting section added such as

Ensure that query does not end with semi colon

Ensure correctness of the query before executing

Ensure the database user executing the database query has the required permissions

For oracle, you can consider adding a link to the Oracle integration documentation to find out the pre-requisite. This includes setting of certain environment variables, installation of client

Recommend to change the text Hosts in Manifest.yml to Host / Connection String

This will basically use the SQL beats module. So if we are claiming support doe special characters for our DB's using SQL as input then that holds true for the input package. Whatever SQL module supports, SQL input package will support.

We can point out the specific DB's Integrations to provide the complete way of configuring and prerequisites. I don't think we should write details about all the individual DB's here. Linking to the specific DB readme would be better

ishleenk17 · 2023-02-08T05:36:07Z

Are there plans to include system testing and it's files & configs as part of this PR?

System Tests are in plan to be added for Input packages. They are not currently available.

agithomas

Review feedback shared.

lalit-satapathy · 2023-02-15T07:20:12Z

LGTM for the first release.

agithomas

Feedback shared.

agithomas · 2023-02-15T08:10:38Z

+Expects a two-column table that looks like a key/value result. The left column is considered a key and the right column the value. This mode generates a single event on each fetch operation.
+
+table:
+Expects any number of columns. This mode generates a single event for each row.


It does not generate when query does not return any rows. So, suggest to clearly state this.

agithomas · 2023-02-15T08:14:32Z

Suggestion: It may be best to replace or add additional screenshot that shows how to include multiple queries.

agithomas

Review comments updated

agithomas

Approved

elasticmachine · 2023-02-15T10:23:59Z

Package sql - 0.0.1 containing this change is available at https://epr.elastic.co/search?package=sql

* Change cpu and memory calculation in vsphere * SQL Input Package: Draft version * Revert "Change cpu and memory calculation in vsphere" This reverts commit 94487c3. * Update Codeowners * Update README * Update README.md * Update README.md * Update README.md * Update README.md

* Change cpu and memory calculation in vsphere * SQL Input Package: Draft version * Revert "Change cpu and memory calculation in vsphere" This reverts commit e4dd5a341b7f66066762bbd1aab265f64b849244. * Update Codeowners * Update README * Update README.md * Update README.md * Update README.md * Update README.md

ishleenk17 added 3 commits January 4, 2023 11:47

Change cpu and memory calculation in vsphere

94487c3

SQL Input Package: Draft version

18b4d1d

Revert "Change cpu and memory calculation in vsphere"

167a162

This reverts commit 94487c3.

ishleenk17 requested a review from a team February 2, 2023 06:11

ishleenk17 self-assigned this Feb 2, 2023

ishleenk17 and others added 2 commits February 2, 2023 11:56

Update Codeowners

e397801

Merge branch 'main' into sql_input_ishleen

8772ac9

lalit-satapathy reviewed Feb 6, 2023

View reviewed changes

Comment thread packages/sql_input/docs/README.md

Comment thread packages/sql_input/docs/README.md Outdated

Comment thread packages/sql_input/docs/README.md Outdated

ishleenk17 marked this pull request as ready for review February 6, 2023 10:52

Update README

62cc1cb

lalit-satapathy reviewed Feb 7, 2023

View reviewed changes

Comment thread packages/sql_input/docs/README.md

Comment thread packages/sql_input/docs/README.md

agithomas reviewed Feb 8, 2023

View reviewed changes

Comment thread packages/sql_input/manifest.yml

Comment thread packages/sql_input/docs/README.md Outdated

agithomas reviewed Feb 8, 2023

View reviewed changes

Comment thread packages/sql_input/docs/README.md

agithomas reviewed Feb 8, 2023

View reviewed changes

ishleenk17 added 2 commits February 9, 2023 13:09

Update README.md

8a274ec

Update README.md

dc527c1

agithomas reviewed Feb 13, 2023

View reviewed changes

Comment thread packages/sql_input/docs/README.md

Update README.md

c26f49c

elastic deleted a comment from lalit-satapathy Feb 14, 2023

lalit-satapathy reviewed Feb 14, 2023

View reviewed changes

Comment thread packages/sql_input/docs/README.md Outdated

Update README.md

c326c30

agithomas reviewed Feb 15, 2023

View reviewed changes

agithomas approved these changes Feb 15, 2023

View reviewed changes

ishleenk17 merged commit b723d42 into elastic:main Feb 15, 2023

andrewkroh added Integration:sql SQL Input New Integration Issue or pull request for creating a new integration package. labels Aug 13, 2024

ishleenk17 deleted the sql_input_ishleen branch February 7, 2025 03:44

Navigation Menu

Uh oh!

Sql Input Package - #5167

Sql Input Package#5167
ishleenk17 merged 10 commits into
elastic:mainfrom
ishleenk17:sql_input_ishleen

ishleenk17 commented Feb 2, 2023 •

edited

Loading

elasticmachine commented Feb 2, 2023 •

edited

Loading

Build stats

lalit-satapathy left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

agithomas commented Feb 8, 2023

Uh oh!

Uh oh!

Uh oh!

agithomas left a comment

agithomas commented Feb 8, 2023

agithomas commented Feb 8, 2023

ishleenk17 commented Feb 8, 2023

ishleenk17 commented Feb 8, 2023

agithomas left a comment

Uh oh!

Uh oh!

lalit-satapathy commented Feb 15, 2023

agithomas left a comment

agithomas Feb 15, 2023 •

edited

Loading

agithomas commented Feb 15, 2023

agithomas left a comment

agithomas left a comment

elasticmachine commented Feb 15, 2023

Labels

5 participants

Uh oh!

Conversation

ishleenk17 commented Feb 2, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Checklist

How to test this PR locally

Related issues

Screenshots

elasticmachine commented Feb 2, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Build stats

🤖 GitHub comments

lalit-satapathy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

agithomas commented Feb 8, 2023

Uh oh!

Uh oh!

Uh oh!

agithomas left a comment

Choose a reason for hiding this comment

agithomas commented Feb 8, 2023

agithomas commented Feb 8, 2023

ishleenk17 commented Feb 8, 2023

ishleenk17 commented Feb 8, 2023

agithomas left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lalit-satapathy commented Feb 15, 2023

agithomas left a comment

Choose a reason for hiding this comment

agithomas Feb 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

agithomas commented Feb 15, 2023

agithomas left a comment

Choose a reason for hiding this comment

agithomas left a comment

Choose a reason for hiding this comment

elasticmachine commented Feb 15, 2023

Labels

5 participants

ishleenk17 commented Feb 2, 2023 •

edited

Loading

elasticmachine commented Feb 2, 2023 •

edited

Loading

agithomas Feb 15, 2023 •

edited

Loading