[O11y][MSSQL] Add optional mssql.query field to `performance` data stream for debugging purposes by harnish-crest-data · Pull Request #12212 · elastic/integrations

harnish-crest-data · 2024-12-31T05:58:27Z

Enhancement

Proposed commit message

MSSQL perfromance data stream uses the merge_results: true parameter. From this parameter, I observed that all the sql_queries results are being merged in single event and we are not receiving the sql.query field in the response. Refer
Added preserve_sql_queries flag that can be used as preserve sql.query field, if it's enabled.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.
I have verified that any added dashboard complies with Kibana's Dashboard good practices

Related issues

Relates SQL server integration provide the sql.query for debug #12029

Screenshot

lalit-satapathy · 2024-12-31T11:13:11Z

Lets wait till the beats code change, This can be an array.

array won't be supported.

Error: building package failed: invalid content found in built zip package: found 1 validation error:

file "/github_repo/harnish_repo/integrations/build/packages/microsoft_sqlserver-2.9.5.zip/data_stream/performance/fields/fields.yml" is invalid: field 0.fields.1.type: 0.fields.1.type must be one of the following: "aggregate_metric_double", "alias", "histogram", "constant_keyword", "text", "match_only_text", "keyword", "long", "integer", "short", "byte", "double", "float", "half_float", "scaled_float", "date", "date_nanos", "boolean", "binary", "integer_range", "float_range", "long_range", "double_range", "date_range", "ip_range", "group", "geo_point", "object", "ip", "nested", "flattened", "wildcard", "version", "unsigned_long"

elastic-vault-github-plugin-prod · 2025-01-16T06:40:21Z

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

shmsr

same comments for the other PR too

shmsr · 2025-01-16T11:27:47Z

Suggested change

- name: preserve_sql_queries

required: true

show_user: false

title: Preserve SQL queries (debug)

description: Preserve the SQL queries used. Enable for debugging purposes only. This feature becomes available in Kibana version 8.18 onwards.

type: bool

multi: false

default: false

- name: preserve_sql_queries

required: true

show_user: false

title: Preserve SQL Queries

description: Preserves SQL queries for debugging purposes. This feature is available in Elastic stack version 8.18 and later.

type: bool

multi: false

default: false

shmsr · 2025-01-16T11:30:48Z

https://github.com/elastic/integrations/pull/12097/files

^^

Here we are not accepting tags from users as of now. But see this PR where because of indentation we faced issue when user input was also there.

Updated, thanks!

shmsr · 2025-01-16T11:32:49Z

Suggested change

description: Executed queries by this package.

description: The SQL queries executed.

elasticmachine · 2025-01-16T12:14:12Z

💚 Build Succeeded

Buildkite Build
Commit: d7c48308ab70118896fad049a6b6ea09d901fd05

History

💚 Build #20491 succeeded b4d72cea6324d955a0ec72fcb53cb92bb7eb03d4
💚 Build #20474 succeeded 08db39c7e43735284379b756acad3697dad9a14a
💔 Build #20472 failed a39665c2b966e801cd4b5ca433fb1c3557719f90
💚 Build #19937 succeeded 215e6b175de81f0b22bc7f728b8752dbda114038
💚 Build #19934 succeeded 4a748570dfd130a029499903749c3490018e778c
💚 Build #19932 succeeded 032e88286b746b91011b8500135ef54db2fb5d00

cc @harnish-elastic

elastic-sonarqube · 2025-01-16T12:14:13Z

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
93.5% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

elastic-vault-github-plugin-prod · 2025-01-17T05:53:10Z

Package microsoft_sqlserver - 2.10.0 containing this change is available at https://epr.elastic.co/package/microsoft_sqlserver/2.10.0/

…ream for debugging purposes (elastic#12212) * Remove unnecessary processor in performance data stream * add changelog entry * added debug flag that can preserve mssql.query field when enabled * Update packages/microsoft_sqlserver/data_stream/performance/manifest.yml Co-authored-by: Lalit Satapathy <69236064+lalit-satapathy@users.noreply.github.com> * update tag name * minor change in pipeline and system tests * minor change in tag description * address review comments --------- Co-authored-by: Lalit Satapathy <69236064+lalit-satapathy@users.noreply.github.com>

harnish-crest-data added the Integration:microsoft_sqlserver Microsoft SQL Server label Dec 31, 2024

harnish-crest-data self-assigned this Dec 31, 2024

harnish-crest-data added Team:Obs-InfraObs Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations] bugfix Pull request that fixes a bug issue labels Dec 31, 2024

harnish-crest-data marked this pull request as ready for review December 31, 2024 05:59

harnish-crest-data requested a review from a team as a code owner December 31, 2024 05:59

harnish-crest-data requested a review from lalit-satapathy December 31, 2024 05:59

harnish-crest-data marked this pull request as draft December 31, 2024 06:04

harnish-crest-data marked this pull request as ready for review December 31, 2024 10:13

harnish-crest-data changed the title ~~[O11y][MSSQL] Remove unnecessary processor in performance data stream~~ Dec 31, 2024

lalit-satapathy reviewed Dec 31, 2024

View reviewed changes

harnish-crest-data requested a review from lalit-satapathy January 1, 2025 06:51

shmsr mentioned this pull request Jan 9, 2025

x-pack/metricbeat/module/sql: Add debug queries when merge_results: true elastic/beats#42271

Merged

6 tasks

mergify Bot mentioned this pull request Jan 10, 2025

[8.x](backport #42271) x-pack/metricbeat/module/sql: Add debug queries when merge_results: true elastic/beats#42285

Merged

6 tasks

harnish-crest-data requested a review from shmsr January 16, 2025 06:20

harnish-crest-data changed the title ~~[Don't Merge][O11y][MSSQL] Remove unnecessary processor in performance data stream~~ Jan 16, 2025

harnish-crest-data mentioned this pull request Jan 16, 2025

SQL server integration provide the sql.query for debug #12029

Closed

harnish-crest-data changed the title ~~[Don't Merge][O11y][MSSQL] Add optional mssql.query field to performance data stream for debugging purposes~~ Jan 16, 2025

harnish-crest-data added enhancement New feature or request and removed bugfix Pull request that fixes a bug issue labels Jan 16, 2025

shmsr reviewed Jan 16, 2025

View reviewed changes

shmsr mentioned this pull request Jan 16, 2025

[O11y][MSSQL] Add mssql.query field to transaction_logs data stream for debug purposes #12216

Merged

5 tasks

harnish-crest-data requested a review from shmsr January 16, 2025 11:56

shmsr approved these changes Jan 17, 2025

View reviewed changes

harnish-crest-data merged commit 9cddd71 into elastic:main Jan 17, 2025

harnish-crest-data deleted the mssql-update-pipeline branch February 7, 2025 06:27

Navigation Menu

Uh oh!

[O11y][MSSQL] Add optional mssql.query field to `performance` data stream for debugging purposes - #12212

[O11y][MSSQL] Add optional mssql.query field to `performance` data stream for debugging purposes#12212
harnish-crest-data merged 0 commit into
elastic:mainfrom
chavdaharnish:mssql-update-pipeline

harnish-crest-data commented Dec 31, 2024 •

edited

Loading

Uh oh!

lalit-satapathy Dec 31, 2024

harnish-crest-data Dec 31, 2024

elastic-vault-github-plugin-prod Bot commented Jan 16, 2025

shmsr left a comment

shmsr Jan 16, 2025

shmsr Jan 16, 2025

harnish-crest-data Jan 16, 2025

shmsr Jan 16, 2025

elasticmachine commented Jan 16, 2025

elastic-sonarqube Bot commented Jan 16, 2025

elastic-vault-github-plugin-prod Bot commented Jan 17, 2025

Labels

4 participants

-      - name: preserve_sql_queries
-        required: true
-        show_user: false
-        title: Preserve SQL queries (debug)
-        description: Preserve the SQL queries used. Enable for debugging purposes only. This feature becomes available in Kibana version 8.18 onwards.
-        type: bool
-        multi: false
-        default: false
+      - name: preserve_sql_queries
+        required: true
+        show_user: false
+        title: Preserve SQL Queries
+        description: Preserves SQL queries for debugging purposes. This feature is available in Elastic stack version 8.18 and later.
+        type: bool
+        multi: false
+        default: false

	description: Executed queries by this package.
	description: The SQL queries executed.

Uh oh!

Conversation

harnish-crest-data commented Dec 31, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed commit message

Checklist

Related issues

Screenshot

Uh oh!

lalit-satapathy Dec 31, 2024

Choose a reason for hiding this comment

harnish-crest-data Dec 31, 2024

Choose a reason for hiding this comment

elastic-vault-github-plugin-prod Bot commented Jan 16, 2025

🚀 Benchmarks report

shmsr left a comment

Choose a reason for hiding this comment

shmsr Jan 16, 2025

Choose a reason for hiding this comment

shmsr Jan 16, 2025

Choose a reason for hiding this comment

harnish-crest-data Jan 16, 2025

Choose a reason for hiding this comment

shmsr Jan 16, 2025

Choose a reason for hiding this comment

elasticmachine commented Jan 16, 2025

💚 Build Succeeded

History

elastic-sonarqube Bot commented Jan 16, 2025

Quality Gate passed

elastic-vault-github-plugin-prod Bot commented Jan 17, 2025

Labels

4 participants

harnish-crest-data commented Dec 31, 2024 •

edited

Loading