Skip to content

[Web & messaging OTel] Regenerate OTel dashboards with adHocDataViews - #20378

Merged
giorgi-imerlishvili-elastic merged 2 commits into
elastic:mainfrom
giorgi-imerlishvili-elastic:regenerate-dashboards-web-messaging
Aug 6, 2026
Merged

[Web & messaging OTel] Regenerate OTel dashboards with adHocDataViews#20378
giorgi-imerlishvili-elastic merged 2 commits into
elastic:mainfrom
giorgi-imerlishvili-elastic:regenerate-dashboards-web-messaging

Conversation

@giorgi-imerlishvili-elastic

Copy link
Copy Markdown
Contributor

Proposed commit message

Regenerate the compiled Kibana dashboards for six OTel packages (iis_otel, traefik_otel, apache_tomcat_otel, haproxy_otel, activemq_otel, ibmmq_otel) by recompiling their YAML sources with the Elastic fork of the dashboard compiler (elastic/kb-yaml-to-lens, which adds adHocDataViews to compiled objects).

  • WHAT: every ES|QL Lens layer in the shipped dashboard JSON now carries a populated adHocDataViews, internalReferences, and per-layer index. No queries, titles, visualization types, dimensions, filters, controls, or layout change — the only intentional diff is those additions (plus an occasional benign legendPosition).
  • WHY: the previously pinned compiler (kb-dashboard-cli==0.4.1) emitted "adHocDataViews": {} and omitted the per-layer index/internalReferences. On Kibana 9.3+ this breaks the panel "Explore in Discover" action (elastic/obs-integration-team#1014). Repopulating those fields restores it.

iis_otel is JSON-only: its _dev YAML source is intentionally left unchanged so the pinned validate-dashboards CI compiler does not recompile empty adHocDataViews and fail CI; only its compiled JSON is updated.

Each package gets a patch version bump and a bugfix changelog entry.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

  • Confirm the only diff vs. the previously compiled dashboards is adHocDataViews, internalReferences, and per-layer index (plus occasional legendPosition) — no query/title/viz-type/layout drift.
  • Confirm iis_otel is intentionally JSON-only (its _dev/shared/kibana YAML is unchanged) so the pinned dashboard-compiler CI check stays green.
  • Spot-check "Explore in Discover" on a rendered ES|QL panel on a Kibana 9.3+ stack.

How to test this PR locally

Executed by the author on a Kibana 9.5 stack (issue #1014 reproduces on 9.3+):

  • Installed all six regenerated packages via Fleet and queried every dashboard saved object. All 23 dashboards return HTTP 200, panel counts are unchanged vs. main, and every ES|QL panel now has a populated adHocDataViews:
    • iis_otel — 2 dashboards / 34 ES|QL panels / 34 with adHocDataViews
    • traefik_otel — 4 / 37 / 37
    • apache_tomcat_otel — 6 / 104 / 104
    • haproxy_otel — 4 / 41 / 41
    • activemq_otel — 3 / 36 / 36
    • ibmmq_otel — 4 / 64 / 64
  • Before/after on traefik_otel (with sample data ingested so panels render): on the main build the panel menu exposed no "Explore in Discover" and the dashboard JSON had empty adHocDataViews; on this build "Explore in Discover" appears, opens Discover with the panel's ES|QL query / index pattern / columns / time range preserved, and no Cannot read properties of undefined (reading 'id') (Prepare requested resources visualizations for Kubernetes Observability #1014) console error.

Reviewer steps:

  • Install any of these packages on a Kibana >= 9.3 stack, open a dashboard with a rendering ES|QL panel, use the panel menu -> "Explore in Discover", and confirm it opens cleanly with the query preserved and no console error.

Related issues

  • Relates elastic/obs-integration-team#1148
  • Relates elastic/obs-integration-team#1014

Screenshots

This PR changes only compiled dashboard JSON (no UI/config forms). The before/after "Explore in Discover" behavior on traefik_otel was verified live on Kibana 9.5; a summary is posted on the tracking issue elastic/obs-integration-team#1148.

Recompile the committed Kibana dashboards for iis_otel, traefik_otel,
apache_tomcat_otel, haproxy_otel, activemq_otel and ibmmq_otel with the
Elastic fork of the dashboard compiler so every ES|QL Lens layer gains a
populated adHocDataViews, internalReferences and per-layer index. This fixes
the broken "Explore in Discover" action on Kibana 9.3+
(elastic/obs-integration-team#1014). The only intended JSON change is those
additions (plus an occasional benign legendPosition).

iis_otel is JSON-only (its _dev YAML is left untouched) so the pinned CI
dashboard compiler does not recompile empty adHocDataViews and fail CI.

Bumps each package's patch version and adds a bugfix changelog entry
(PR link placeholder to be replaced once the PR number is known).
@giorgi-imerlishvili-elastic
giorgi-imerlishvili-elastic requested a review from a team as a code owner July 28, 2026 10:23
@giorgi-imerlishvili-elastic giorgi-imerlishvili-elastic added the bug Something isn't working, use only for issues label Jul 28, 2026
@github-actions

Copy link
Copy Markdown
Contributor

✅ Elastic Docs Style Checker (Vale)

No issues found on modified lines!


The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

✅ All changelog entries have the correct PR link.

@infra-vault-gh-plugin-prod

Copy link
Copy Markdown

💚 Build Succeeded

cc @giorgi-imerlishvili-elastic

@giorgi-imerlishvili-elastic giorgi-imerlishvili-elastic added Integration:iis_otel IIS OpenTelemetry assets Integration:traefik_otel Traefik OpenTelemetry Assets Integration:apache_tomcat_otel Apache Tomcat OpenTelemetry Assets Integration:haproxy_otel Haproxy OpenTelemetry Assets Integration:activemq_otel ActiveMQ OpenTelemetry Assets Integration:ibmmq_otel IBM MQ OpenTelemetry Assets labels Jul 28, 2026
@andrewkroh andrewkroh added dashboard Relates to a Kibana dashboard bug, enhancement, or modification. Team:Obs-InfraObs Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations] labels Jul 28, 2026
@shmsr
shmsr requested a review from Copilot August 6, 2026 07:59

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Regenerates compiled Kibana dashboard saved-object JSON for several web & messaging OpenTelemetry asset packages so ES|QL Lens layers include populated adHocDataViews, internalReferences, and per-layer index (to restore “Explore in Discover” behavior on newer Kibana versions). Also bumps each affected package’s patch version and records the change in changelog.yml.

Changes:

  • Patch-version bumps in affected package manifest.yml files.
  • Regenerated dashboard JSON assets to include adHocDataViews / internalReferences / per-layer index metadata.
  • Added bugfix changelog entries referencing the regeneration.

Reviewed changes

Copilot reviewed 33 out of 35 changed files in this pull request and generated no comments.

Show a summary per file
File Description
packages/traefik_otel/manifest.yml Patch version bump to ship regenerated dashboards.
packages/traefik_otel/kibana/dashboard/traefik_otel-tls-config.json Regenerated compiled dashboard JSON with populated ES
packages/traefik_otel/kibana/dashboard/traefik_otel-services.json Regenerated compiled dashboard JSON with populated ES
packages/traefik_otel/kibana/dashboard/traefik_otel-process.json Regenerated compiled dashboard JSON with populated ES
packages/traefik_otel/kibana/dashboard/traefik_otel-overview.json Regenerated compiled dashboard JSON with populated ES
packages/traefik_otel/changelog.yml Adds bugfix changelog entry for dashboard regeneration.
packages/iis_otel/manifest.yml Patch version bump to ship regenerated dashboards.
packages/iis_otel/changelog.yml Adds bugfix changelog entry for dashboard regeneration.
packages/ibmmq_otel/manifest.yml Patch version bump to ship regenerated dashboards.
packages/ibmmq_otel/kibana/dashboard/ibmmq_otel-overview.json Regenerated compiled dashboard JSON with populated ES
packages/ibmmq_otel/kibana/dashboard/ibmmq_otel-error-analysis.json Regenerated compiled dashboard JSON with populated ES
packages/ibmmq_otel/changelog.yml Adds bugfix changelog entry for dashboard regeneration.
packages/haproxy_otel/manifest.yml Patch version bump to ship regenerated dashboards.
packages/haproxy_otel/kibana/dashboard/haproxy_otel-server.json Regenerated compiled dashboard JSON with populated ES
packages/haproxy_otel/kibana/dashboard/haproxy_otel-overview.json Regenerated compiled dashboard JSON with populated ES
packages/haproxy_otel/kibana/dashboard/haproxy_otel-frontend.json Regenerated compiled dashboard JSON with populated ES
packages/haproxy_otel/kibana/dashboard/haproxy_otel-backend.json Regenerated compiled dashboard JSON with populated ES
packages/haproxy_otel/changelog.yml Adds bugfix changelog entry for dashboard regeneration.
packages/apache_tomcat_otel/manifest.yml Patch version bump to ship regenerated dashboards.
packages/apache_tomcat_otel/kibana/dashboard/apache_tomcat_otel-thread-pool-connections.json Regenerated compiled dashboard JSON with populated ES
packages/apache_tomcat_otel/kibana/dashboard/apache_tomcat_otel-request-processing.json Regenerated compiled dashboard JSON with populated ES
packages/apache_tomcat_otel/changelog.yml Adds bugfix changelog entry for dashboard regeneration.
packages/activemq_otel/manifest.yml Patch version bump to ship regenerated dashboards.
packages/activemq_otel/kibana/dashboard/activemq_otel-overview.json Regenerated compiled dashboard JSON with populated ES
packages/activemq_otel/kibana/dashboard/activemq_otel-destinations.json Regenerated compiled dashboard JSON with populated ES
packages/activemq_otel/kibana/dashboard/activemq_otel-broker-health.json Regenerated compiled dashboard JSON with populated ES
packages/activemq_otel/changelog.yml Adds bugfix changelog entry for dashboard regeneration.
Suppressed comments (1)

packages/haproxy_otel/kibana/dashboard/haproxy_otel-overview.json:123

  • The PR description states the only intentional diffs in compiled dashboards are additions of adHocDataViews, internalReferences, and per-layer index (plus occasional legendPosition). In this dashboard, the compiled Lens state also changes layerId values (e.g. state.visualization.layerId here) and the dashboard attributes.description text (line 4), which goes beyond the described/checked scope. Please either update the PR description/checklist to acknowledge these additional diffs and confirm they’re expected/benign, or regenerate in a way that preserves stable layer IDs if that’s required.
                            "visualization": {
                                "layerId": "737d5f1c-e4b5-bced-df13-f856e5fa3136",
                                "layerType": "data",
                                "metricAccessor": "add1b5db-6cde-6603-e764-93623cd42282",
                                "showBar": false,
                                "applyColorTo": "background"

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@mergify

mergify Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

  • Queue this pull request
@giorgi-imerlishvili-elastic
giorgi-imerlishvili-elastic merged commit 7229fc0 into elastic:main Aug 6, 2026
10 checks passed
@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package activemq_otel - 0.3.1 containing this change is available at https://epr.elastic.co/package/activemq_otel/0.3.1/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package apache_tomcat_otel - 0.3.1 containing this change is available at https://epr.elastic.co/package/apache_tomcat_otel/0.3.1/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package haproxy_otel - 0.3.1 containing this change is available at https://epr.elastic.co/package/haproxy_otel/0.3.1/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package ibmmq_otel - 0.3.1 containing this change is available at https://epr.elastic.co/package/ibmmq_otel/0.3.1/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package iis_otel - 0.5.1 containing this change is available at https://epr.elastic.co/package/iis_otel/0.5.1/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package traefik_otel - 0.3.1 containing this change is available at https://epr.elastic.co/package/traefik_otel/0.3.1/

@giorgi-imerlishvili-elastic
giorgi-imerlishvili-elastic deleted the regenerate-dashboards-web-messaging branch August 25, 2026 15:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working, use only for issues dashboard Relates to a Kibana dashboard bug, enhancement, or modification. Integration:activemq_otel ActiveMQ OpenTelemetry Assets Integration:apache_tomcat_otel Apache Tomcat OpenTelemetry Assets Integration:haproxy_otel Haproxy OpenTelemetry Assets Integration:ibmmq_otel IBM MQ OpenTelemetry Assets Integration:iis_otel IIS OpenTelemetry assets Integration:traefik_otel Traefik OpenTelemetry Assets Team:Obs-InfraObs Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations]

4 participants