[aws_elb_otel] Add AWS ELB logs content pack for OpenTelemetry · Pull Request #15401 · elastic/integrations

ghost · 2025-09-18T19:25:18Z

Overview

Adds documentation and dashboard for collecting AWS ELB access logs with the EDOT Cloud Forwarder.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.
I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

[ ]

How to test this PR locally

Related issues

Closes elastic/obs-integration-team/issues/583

Screenshots

…_pack

use esql queries in dashboard

ishleenk17 · 2025-09-22T04:18:15Z

@gpop63 : Can you also share the screenshot of Elastic ELB Logs dashboard.

ishleenk17 · 2025-09-22T04:34:51Z

@gpop63 : Yet to review the dashboard. But shared other reviews.

The logo also, lets have one with ELB and OTEL image in the bottom. Refer to other OTEL Content Packs.

mykola-elastic · 2025-09-22T10:46:12Z

+
+For the full documentation on how to set up the EDOT Cloud Forwarder, follow this link: [EDOT Cloud Forwarder for AWS](https://www.elastic.co/docs/reference/opentelemetry/edot-cloud-forwarder/aws).
+
+## Requirements


The doc team uses new namings for sections, like "What do I need to use this integration?" instead of "Requirements". There are more source

Though this is not an integration. Not sure if the new one is good here

ishleenk17 · 2025-09-22T11:51:30Z

+* An Elastic **OTLP endpoint** and **API key** for authentication.
+
+## Setup
+


@gpop63 : We can mention like this for the config of ECF rather than mentioning the details in our CP.
Our CP aims to have dashboards. These config informations should be just FYI, high level

fix field names

ishleenk17 · 2025-09-22T13:52:36Z

NOTE: Once this PR is included in the ECOT Collector, we should change the dashboard filter to
scope.attributes.awslogs_encoding.format: elb_access_log

ghost · 2025-09-22T15:35:41Z

Can you also share the screenshot of Elastic ELB Logs dashboard.

@ishleenk17

This is Elastic ELB dashboard

OTEL dashboard

I have a widescreen thats why it seems more compact.

ishleenk17 · 2025-09-23T13:24:40Z

@gpop63 : Comments on the dashboard

Inbound and outbound traffic doens't showcase which one is inbound which is outbound. Also the y axis says response size. Is that correct ?
Y axis for status code should be Count.
Y axis for total requests should be Count.
Top IP addresses. Are we saying the IP addresses are in ocunt of 60K etc in the graph. Is there a better represeatation for this . Also, can we have maybe the top 10. Currently it looks cluttered.
Totle of dashboard doesn't have mention of OTEL in it.
In elastic dashboard I see different panels for ALB and ELB Inbound/outbound traffic. Are these different . Are the fields for these different ?

mykola-elastic · 2025-09-24T06:41:34Z

The first screenshot (integration page) in the PR description is low resolution

mykola-elastic · 2025-09-24T06:43:36Z

For the dashboard screenshot it would be good to have generate some data also in the 2xx and 5xx panels

tommyers-elastic · 2025-09-30T10:05:11Z

small nit - can we not say "content pack" in the text? whilst this dashboard is getting shipped in a content pack, the dashboard itself is still just a dashboard.

elastic-sonarqube · 2025-09-30T10:59:53Z

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
97.7% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

Co-authored-by: Mykola Kmet <mykola.kmet@elastic.co>

MichaelKatsoulis · 2025-10-20T07:09:19Z

+                        "meta": {
+                            "alias": null,
+                            "disabled": false,
+                            "field": "awslogs_encoding.format",


This field does not exist anymore

@gpop63 That is still not removed

MichaelKatsoulis · 2025-10-20T10:32:14Z

+                        "meta": {
+                            "alias": null,
+                            "disabled": false,
+                            "field": "awslogs_encoding.format",


@gpop63 That is still not removed

MichaelKatsoulis

LGTM. Just a nit readme update!

Co-authored-by: Michalis Katsoulis <michaelkatsoulis88@gmail.com>

ishleenk17 · 2025-10-22T12:28:21Z

@gpop63 : I see that we have used the dataset filter at panel level and not global level.
Any reason for that ?

@gpop63 : I was checking in the dashboard, there is no mention of the word "OpenTelemetry" in the title, description of the dashboard. Anyone looking at just the dashboard will not get to know it is having OTEL data.

Do we have this present somewhere ?

ghost · 2025-10-22T14:03:48Z

@gpop63 : I see that we have used the dataset filter at panel level and not global level.
Any reason for that ?

Initially there was no data stream filter we could use globally, so I added filters at panel level for safety and consistency but we could remove them without any issue right now.

@gpop63 : I was checking in the dashboard, there is no mention of the word "OpenTelemetry" in the title, description of the dashboard. Anyone looking at just the dashboard will not get to know it is having OTEL data.

OTEL keyword is part of the dashboard title.

ishleenk17 · 2025-10-22T14:08:47Z

Initially there was no data stream filter we could use globally, so I added filters at panel level for safety and consistency but we could remove them without any issue right now.

Yes lets do that.

elasticmachine · 2025-10-22T20:41:05Z

💚 Build Succeeded

Buildkite Build
Commit: d0741aa

History

💚 Build #32938 succeeded b854f24
💚 Build #32937 succeeded 6c8c075
💚 Build #32882 succeeded 1314da5
💚 Build #32809 succeeded f601148
💚 Build #32109 succeeded 0632b31
💚 Build #32024 succeeded 741f76b

cc @gpop63

ishleenk17

Looks good!

elastic-vault-github-plugin-prod · 2025-10-23T08:48:50Z

Package aws_elb_otel - 0.1.0 containing this change is available at https://epr.elastic.co/package/aws_elb_otel/0.1.0/

…ic#15401) * add content pack * fix pr id * add codeowners entry * fix codeowners * rename integration to aws_elb_otel use esql queries in dashboard * fix codeowners entry * update logo * update dashboard title fix field names * update docs * add dashboard datastream filter * fix dashboard filter * address comments * use bytes for y axis * Update packages/aws_elb_otel/changelog.yml Co-authored-by: Mykola Kmet <mykola.kmet@elastic.co> * update dashboard * update dashboard * remove content pack from title * Update packages/aws_elb_otel/manifest.yml Co-authored-by: Mykola Kmet <mykola.kmet@elastic.co> * update data stream filter * fix datastream filters * fix datastream filter * Update packages/aws_elb_otel/docs/README.md Co-authored-by: Michalis Katsoulis <michaelkatsoulis88@gmail.com> * remove datastream filter at lens lvl --------- Co-authored-by: Mykola Kmet <mykola.kmet@elastic.co> Co-authored-by: Michalis Katsoulis <michaelkatsoulis88@gmail.com>

add content pack

7badf9f

ghost self-requested a review as a code owner September 18, 2025 19:25

ghost self-assigned this Sep 18, 2025

ghost added New Integration Issue or pull request for creating a new integration package. dashboard Relates to a Kibana dashboard bug, enhancement, or modification. Team:Obs-InfraObs Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations] labels Sep 18, 2025

ghost requested a review from ishleenk17 September 18, 2025 19:26

mrg-elastic added 4 commits September 18, 2025 22:26

fix pr id

2cd67aa

add codeowners entry

4ffaba6

Merge remote-tracking branch 'upstream/main' into add_aws_elb_content…

74f6dac

…_pack

fix codeowners

09575f5

andrewkroh added the documentation Improvements or additions to documentation. Applied to PRs that modify *.md files. label Sep 18, 2025

rename integration to aws_elb_otel

97ebc00

use esql queries in dashboard

ghost changed the title ~~[elb_otel] Add AWS ELB logs content pack for OpenTelemetry~~ Sep 19, 2025

fix codeowners entry

2ee23f2

ishleenk17 reviewed Sep 22, 2025

View reviewed changes

Comment thread packages/aws_elb_otel/docs/README.md

Comment thread packages/aws_elb_otel/docs/README.md Outdated

Comment thread packages/aws_elb_otel/docs/README.md Outdated

Comment thread packages/aws_elb_otel/docs/README.md Outdated

mykola-elastic reviewed Sep 22, 2025

View reviewed changes

ishleenk17 reviewed Sep 22, 2025

View reviewed changes

mrg-elastic added 4 commits September 22, 2025 15:50

update logo

8568974

update dashboard title

f157310

fix field names

update docs

0e6892f

add dashboard datastream filter

f3d05bb

fix dashboard filter

3bce5d2

update dashboard

741f76b

daniela-elastic approved these changes Sep 30, 2025

View reviewed changes

remove content pack from title

0632b31

mykola-elastic reviewed Oct 16, 2025

View reviewed changes

Comment thread packages/aws_elb_otel/manifest.yml

Gabriel Pop and others added 2 commits October 17, 2025 10:26

Update packages/aws_elb_otel/manifest.yml

2b39476

Co-authored-by: Mykola Kmet <mykola.kmet@elastic.co>

update data stream filter

f601148

MichaelKatsoulis reviewed Oct 20, 2025

View reviewed changes

Comment thread packages/aws_elb_otel/kibana/dashboard/aws_elb_otel-918bae2f-9268-46da-acac-7f4029ca0bfa.json Outdated

MichaelKatsoulis reviewed Oct 20, 2025

View reviewed changes

fix datastream filters

1314da5

ghost requested a review from MichaelKatsoulis October 20, 2025 07:44

MichaelKatsoulis requested changes Oct 20, 2025

View reviewed changes

fix datastream filter

6c8c075

MichaelKatsoulis self-requested a review October 21, 2025 08:41

MichaelKatsoulis reviewed Oct 21, 2025

View reviewed changes

Comment thread packages/aws_elb_otel/docs/README.md Outdated

MichaelKatsoulis approved these changes Oct 21, 2025

View reviewed changes

Update packages/aws_elb_otel/docs/README.md

b854f24

Co-authored-by: Michalis Katsoulis <michaelkatsoulis88@gmail.com>

remove datastream filter at lens lvl

d0741aa

ishleenk17 approved these changes Oct 23, 2025

View reviewed changes

lalit-satapathy approved these changes Oct 23, 2025

View reviewed changes

ghost merged commit 6bd3578 into elastic:main Oct 23, 2025
7 checks passed

andrewkroh added the Integration:aws_elb_otel AWS ELB Logs OpenTelemetry Assets label Oct 23, 2025

This pull request was closed.

Navigation Menu

Uh oh!

[aws_elb_otel] Add AWS ELB logs content pack for OpenTelemetry - #15401

[aws_elb_otel] Add AWS ELB logs content pack for OpenTelemetry#15401
24 commits merged into
mainfrom
unknown repository

ghost commented Sep 18, 2025 •

edited by ghost

Loading

ishleenk17 commented Sep 22, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ishleenk17 commented Sep 22, 2025 •

edited

Loading

mykola-elastic Sep 22, 2025

mykola-elastic Sep 22, 2025

ishleenk17 Sep 22, 2025

ishleenk17 commented Sep 22, 2025

ghost commented Sep 22, 2025

ishleenk17 commented Sep 23, 2025

mykola-elastic commented Sep 24, 2025

mykola-elastic commented Sep 24, 2025

tommyers-elastic commented Sep 30, 2025

elastic-sonarqube Bot commented Sep 30, 2025

Uh oh!

Uh oh!

MichaelKatsoulis Oct 20, 2025

MichaelKatsoulis Oct 20, 2025

MichaelKatsoulis Oct 20, 2025

Uh oh!

MichaelKatsoulis left a comment

ishleenk17 commented Oct 22, 2025

ghost commented Oct 22, 2025

ishleenk17 commented Oct 22, 2025

elasticmachine commented Oct 22, 2025

ishleenk17 left a comment

Uh oh!

elastic-vault-github-plugin-prod Bot commented Oct 23, 2025

Labels

9 participants


		For the full documentation on how to set up the EDOT Cloud Forwarder, follow this link: [EDOT Cloud Forwarder for AWS](https://www.elastic.co/docs/reference/opentelemetry/edot-cloud-forwarder/aws).

		## Requirements

		* An Elastic OTLP endpoint and API key for authentication.

		## Setup

Uh oh!

Conversation

ghost commented Sep 18, 2025 • edited by ghost Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

Checklist

Author's Checklist

How to test this PR locally

Related issues

Screenshots

ishleenk17 commented Sep 22, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ishleenk17 commented Sep 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

mykola-elastic Sep 22, 2025

Choose a reason for hiding this comment

mykola-elastic Sep 22, 2025

Choose a reason for hiding this comment

ishleenk17 Sep 22, 2025

Choose a reason for hiding this comment

ishleenk17 commented Sep 22, 2025

ghost commented Sep 22, 2025

ishleenk17 commented Sep 23, 2025

mykola-elastic commented Sep 24, 2025

mykola-elastic commented Sep 24, 2025

tommyers-elastic commented Sep 30, 2025

elastic-sonarqube Bot commented Sep 30, 2025

Quality Gate passed

Uh oh!

Uh oh!

MichaelKatsoulis Oct 20, 2025

Choose a reason for hiding this comment

MichaelKatsoulis Oct 20, 2025

Choose a reason for hiding this comment

MichaelKatsoulis Oct 20, 2025

Choose a reason for hiding this comment

Uh oh!

MichaelKatsoulis left a comment

Choose a reason for hiding this comment

ishleenk17 commented Oct 22, 2025

ghost commented Oct 22, 2025

ishleenk17 commented Oct 22, 2025

elasticmachine commented Oct 22, 2025

💚 Build Succeeded

History

ishleenk17 left a comment

Choose a reason for hiding this comment

Uh oh!

elastic-vault-github-plugin-prod Bot commented Oct 23, 2025

Labels

9 participants

ghost commented Sep 18, 2025 •

edited by ghost

Loading

ishleenk17 commented Sep 22, 2025 •

edited

Loading