Add AWS Cloudwatch Input Package by muthu-mps · Pull Request #19468 · elastic/integrations

muthu-mps · 2026-06-10T03:03:43Z

Enhancement]

Proposed commit message

Summary

Initial release of the AWS CloudWatch OpenTelemetry Input Package (aws_cloudwatch_input_otel). The package wraps the OpenTelemetry Collector awscloudwatchmetrics receiver and exposes one Kibana integration per AWS service.

What's included

Six policy templates, one per service: aws_ec2, lambda, rds, sqs, elb, fargate.
Top-level vars: for region, AWS credentials (env-var-mapped), profile, role assumption, poll interval, and autodiscover knobs — defined once and inherited by every template.
Autodiscover-based metric collection per CloudWatch namespace, so new AWS metrics are ingested automatically.
Agent and agentless deployment modes enabled on every template.
Resource-detection processors (system + aws ec2/ecs) for host.* and cloud.* enrichment.
README.md.

What's intentionally out of scope for this PR

Curated named-metric lists.
Additional AWS services beyond the six above.
End-to-end agentless validation matrix runs.
These will be addressed in follow-up PRs.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.
I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

[ ]

How to test this PR locally

Related issues

Screenshots

github-actions · 2026-06-10T03:05:02Z

Elastic Docs Style Checker (Vale)

Summary: 1 suggestion found

💡 Suggestions (1): Optional style improvements. Apply when helpful.

File	Line	Rule	Message
packages/aws_cloudwatch_input_otel/changelog.yml	1	Elastic.Versions	Use 'later versions' instead of 'newer versions' when referring to versions.

The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.

github-actions · 2026-06-10T03:43:28Z

TL;DR

The Buildkite failure is a test failure in Check integrations aws_cloudwatch_input_otel: the PR head removes all package test fixtures under _dev/test, so the package test phase exits non-zero. Immediate fix is to restore policy/system test files (or equivalent replacements) for this package.

Remediation

Restore test fixtures removed in e08ff5d364:
- packages/aws_cloudwatch_input_otel/_dev/test/policy/test-ec2.yml
- packages/aws_cloudwatch_input_otel/_dev/test/policy/test-elb.yml
- packages/aws_cloudwatch_input_otel/_dev/test/policy/test-fargate.yml
- packages/aws_cloudwatch_input_otel/_dev/test/policy/test-lambda.yml
- packages/aws_cloudwatch_input_otel/_dev/test/policy/test-rds.yml
- packages/aws_cloudwatch_input_otel/_dev/test/policy/test-sqs.yml
- packages/aws_cloudwatch_input_otel/_dev/test/system/test-default-config.yml
Re-run the package check job after restoring/replacing these tests.

Investigation details

Root Cause

packages/aws_cloudwatch_input_otel/manifest.yml defines six policy templates (aws_ec2, lambda, rds, sqs, elb, fargate) at manifest.yml:137-227, but PR head commit e08ff5d364 deletes all corresponding policy test fixtures and the system test config. The Buildkite step then fails in package test execution and exits with status 1.

Evidence

Build: https://buildkite.com/elastic/integrations/builds/44348
Job/step: Check integrations aws_cloudwatch_input_otel
Key log excerpt (/tmp/gh-aw/buildkite-logs/integrations-check-integrations-aws_cloudwatch_input_otel.txt:81-86):

--- [aws_cloudwatch_input_otel] failed
🚨 Error: The command exited with status 1
user command error: exit status 1

Build command from failure summary:

.buildkite/scripts/test_one_package.sh packages/aws_cloudwatch_input_otel origin/main 3b114eb7da98f35f4d12ca64db57c890b56e7962

PR head commit deletion summary (e08ff5d364):

D packages/aws_cloudwatch_input_otel/_dev/test/policy/test-ec2.yml
D packages/aws_cloudwatch_input_otel/_dev/test/policy/test-elb.yml
D packages/aws_cloudwatch_input_otel/_dev/test/policy/test-fargate.yml
D packages/aws_cloudwatch_input_otel/_dev/test/policy/test-lambda.yml
D packages/aws_cloudwatch_input_otel/_dev/test/policy/test-rds.yml
D packages/aws_cloudwatch_input_otel/_dev/test/policy/test-sqs.yml
D packages/aws_cloudwatch_input_otel/_dev/test/system/test-default-config.yml

Verification

Local reproduction of the full Buildkite wrapper script was not possible in this environment because required CI env vars/tools (YQ_VERSION, elastic-package) are unavailable.
The failure attribution above is based on the Buildkite failure step + PR head diff evidence.

Follow-up

If you intentionally removed per-template tests, add replacement policy/system coverage in _dev/test/ so test_one_package.sh has valid package tests to execute.

Note

🔒 Integrity filter blocked 2 items

The following items were blocked because they don't meet the GitHub integrity level.

#19468 pull_request_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
Add AWS Cloudwatch Input Package #19468 pull_request_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".

To allow these resources, lower min-integrity in your GitHub frontmatter:

tools:
  github:
    min-integrity: approved  # merged | approved | unapproved | none

What is this? | From workflow: PR Buildkite Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

Introduces the AWS CloudWatch OpenTelemetry Input Package (aws_cloudwatch_input_otel), which wraps the awscloudwatchmetrics receiver for EC2, Lambda, RDS, SQS, ELB, and Fargate metrics, and declares the required provider_permissions so tooling can generate appropriately-scoped IAM policies for agentless deployments. - Package sourced from PR #19468 (add cloudwatch receiver). - Bumped format_version to 3.6.4 and version to 0.6.0 to enable provider_permissions. - Declared package-level AWS permissions: sts:GetCallerIdentity, cloudwatch:GetMetricData, cloudwatch:ListMetrics, tag:GetResources. All six policy templates (ec2, lambda, rds, sqs, elb, fargate) use the same receiver and API surface, so a single package-level block covers them all. - Addresses elastic/ingest-dev#8142. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

jsoriano

Approving from the integrations triaging side. Thanks!

jsoriano · 2026-06-11T11:22:05Z

+# to enumerate its metrics — so newly published metrics from AWS are picked up
+# automatically without package changes. New services can be added by introducing
+# a new `<svc>.yml.hbs` file plus a policy_template entry below.
+policy_templates:


I think this is the first input package we have with multiple policy templates, expect dragons. Please let us know if you find any issue.

Are multiple policy templates expected to be used in the same policy?

The UI pulls the dataset name config and gets duplicated in every policy template.

Do you know why its happening?

elastic-vault-github-plugin-prod · 2026-06-11T17:55:07Z

✅ All changelog entries have the correct PR link.

elasticmachine · 2026-06-11T18:10:59Z

💚 Build Succeeded

Buildkite Build
Commit: a3a9996

History

💚 Build #44407 succeeded 01112ac
💚 Build #44351 succeeded e08ff5d
💔 Build #44348 failed 3b114eb

elastic-vault-github-plugin-prod · 2026-06-11T18:29:53Z

Package aws_cloudwatch_input_otel - 0.1.0 containing this change is available at https://epr.elastic.co/package/aws_cloudwatch_input_otel/0.1.0/

nchaulet · 2026-06-16T18:05:24Z

+{{#if access_key_id}}
+access_key_id: {{access_key_id}}
+{{/if}}
+{{#if secret_access_key}}
+secret_access_key: {{secret_access_key}}
+{{/if}}
+{{#if session_token}}
+session_token: {{session_token}}
+{{/if}}
+{{#if credential_profile_name}}
+credential_profile_name: {{credential_profile_name}}
+{{/if}}
+{{#if shared_credential_file}}
+shared_credential_file: {{shared_credential_file}}
+{{/if}}
+{{#if role_arn}}
+role_arn: {{role_arn}}
+{{/if}}


Hi @muthu-mps while reviewing https://github.com/elastic/kibana/pull/273364/changes I tried those and it seems it not working as expected, I do not think we support root level variables for otel input

add cloudwatch receiver

a7d91b7

muthu-mps requested a review from a team as a code owner June 10, 2026 03:03

muthu-mps added 3 commits June 10, 2026 08:35

update code owner

3c256c2

update changelog entry

3b114eb

remove test files

e08ff5d

andrewkroh added New Integration Issue or pull request for creating a new integration package. documentation Improvements or additions to documentation. Applied to PRs that modify *.md files. labels Jun 10, 2026

jeniawhite mentioned this pull request Jun 11, 2026

[aws] Declare required permissions with provider_permissions #19405

Open

8 tasks

juliaElastic reviewed Jun 11, 2026

View reviewed changes

Comment thread packages/aws_cloudwatch_input_otel/manifest.yml

jsoriano requested a review from a team June 11, 2026 11:08

jsoriano approved these changes Jun 11, 2026

View reviewed changes

muthu-mps added 2 commits June 11, 2026 18:34

combine policy template and discover matrics using namepace filter

01112ac

update config

a3a9996

muthu-mps enabled auto-merge (squash) June 11, 2026 17:55

muthu-mps merged commit daf1f94 into elastic:main Jun 11, 2026
11 checks passed

andrewkroh added the Integration:aws_cloudwatch_input_otel AWS CloudWatch (OpenTelemetry) label Jun 12, 2026

jsoriano mentioned this pull request Jun 15, 2026

[Fleet] Fix multiple data stream selectors in input packages with multiple policy templates elastic/kibana#273364

Merged

8 tasks

nchaulet reviewed Jun 16, 2026

View reviewed changes

ishleenk17 changed the title ~~add cloudwatch receiver~~ Jun 17, 2026

ishleenk17 changed the title ~~add AWS Cloudwatch Input Package~~ Jun 17, 2026

Navigation Menu

Uh oh!

Add AWS Cloudwatch Input Package - #19468

Add AWS Cloudwatch Input Package#19468
muthu-mps merged 6 commits into
elastic:mainfrom
muthu-mps:add_aws_cloudwatch_otel_input

muthu-mps commented Jun 10, 2026

github-actions Bot commented Jun 10, 2026

github-actions Bot commented Jun 10, 2026

Root Cause

Evidence

Verification

Follow-up

Uh oh!

jsoriano left a comment

Uh oh!

jsoriano Jun 11, 2026

muthu-mps Jun 11, 2026

elastic-vault-github-plugin-prod Bot commented Jun 11, 2026

elasticmachine commented Jun 11, 2026

Uh oh!

elastic-vault-github-plugin-prod Bot commented Jun 11, 2026

nchaulet Jun 16, 2026

Labels

7 participants

Uh oh!

Conversation

muthu-mps commented Jun 10, 2026

Proposed commit message

Summary

What's included

What's intentionally out of scope for this PR

Checklist

Author's Checklist

How to test this PR locally

Related issues

Screenshots

github-actions Bot commented Jun 10, 2026

Elastic Docs Style Checker (Vale)

github-actions Bot commented Jun 10, 2026

TL;DR

Remediation

Root Cause

Evidence

Verification

Follow-up

Uh oh!

jsoriano left a comment

Choose a reason for hiding this comment

Uh oh!

jsoriano Jun 11, 2026

Choose a reason for hiding this comment

muthu-mps Jun 11, 2026

Choose a reason for hiding this comment

elastic-vault-github-plugin-prod Bot commented Jun 11, 2026

elasticmachine commented Jun 11, 2026

💚 Build Succeeded

History

Uh oh!

elastic-vault-github-plugin-prod Bot commented Jun 11, 2026

nchaulet Jun 16, 2026

Choose a reason for hiding this comment

Labels

7 participants