[AWS Lambda] Add lambda logs overview dashboard

[agithomas]

• Enhancement

Proposed commit message

Add AWS Lambda logs overview dashboard

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

• Package upgrade testing
• Integration test

How to test this PR locally

• elastic-package build && elastic-package stack up -v -d --services package-registry

Related issues

#14070

Screenshots

Metrics overview dashboard

Log overview dashboard

[elastic-vault-github-plugin-prod]

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

[daniela-elastic]

Can we add the dashboard to the existing AWS dashboard as a link panel so customers can have all Lambda signals in one place

[agithomas]

Can we add the dashboard to the existing AWS dashboard as a link panel so customers can have all Lambda signals in one place

@daniela-elastic , The above-mentioned suggested changes are applied. I have updated the screenshots. Please take a look.

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 3bd95ef

History

• 💚 Build #27012 succeeded e9c56c2

cc @agithomas

[muthu-mps]

1. Lambda Event Source Mapping Metrics - Can we make the description in the 'i' icon? If its not possible then I would recommend to have one line description.
2. The metrics panel title case doesn't match with the logs panel. Can we update it?
3. Average memory allocated and used. Shouldn't this panel have Total memory allocated and used instead of average? If we decide to keep average memory here, then the other chart Memory Allocated vs Memory used, Is this average or total?
4. Init duration vs duration vs Billed duration. What is the actual metrics for Duration? Is this include the complete duration or something different?

[muthu-mps]

To improve clarity, we may want to rename the dashboard titles. The metrics overview dashboard is missing a high-level summary, and the logs overview includes a summary that references both metrics and logs, which is confusing. We should consider renaming the panels to make them more generic

[agithomas]

Lambda Event Source Mapping Metrics - Can we make the description in the 'i' icon? If its not possible then I would recommend to have one line description.
The metrics panel title case doesn't match with the logs panel. Can we update it?

These are beyond the scope of this PR. I will create a separate ER issue mentioning your suggestions.

Average memory allocated and used. Shouldn't this panel have Total memory allocated and used instead of average? If we decide to keep average memory here, then the other chart Memory Allocated vs Memory used, Is this average or total?

Are you referring to using sum() aggregation instead of average()? Given a timeseries, average() memory utilisation may be more preferred

Init duration vs duration vs Billed duration. What is the actual metrics for Duration? Is this include the complete duration or something different?

As you could see in the screenshot, it uses timeseries in the horizontal axis. The value in a specific timeslice / datapoint is average-ed and displayed.

Is your suggestion replacing Memory allocated vs memory used as Average memory allocated vs memory used?

If yes, do you also consider substituting Memory allocated vs memory used also as Average memory allocated vs memory used ?

The metrics overview dashboard is missing a high-level summary

This is not part of the scope of this PR. I will add this point to a spearate ER

logs overview includes a summary that references both metrics and logs

This is true. Each invocation of a Lambda function publishes a “REPORT” log entry at the end of the invocation. The entry provides details about the actual duration, billed duration, and amount of memory used and these comes under the metrics category and are the ones that are displayed under the Logs overview dashboard.

[agithomas]

@muthu-mps, please find the ER issue created to track the dashboard improvement suggestions you shared.

[muthu-mps]

@muthu-mps, please find the ER issue created to track the dashboard improvement suggestions you shared.

Thanks @agithomas!

[muthu-mps]

Average memory allocated and used. Shouldn't this panel have Total memory allocated and used instead of average? If we decide to keep average memory here, then the other chart Memory Allocated vs Memory used, Is this average or total?

Are you referring to using sum() aggregation instead of average()? Given a timeseries, average() memory utilisation may be more preferred

No

Is your suggestion replacing Memory allocated vs memory used as Average memory allocated vs memory used?

Yes, WDYT?

Init duration vs duration vs Billed duration. What is the actual metrics for Duration? Is this include the complete duration or something different?

As you could see in the screenshot, it uses timeseries in the horizontal axis. The value in a specific timeslice / datapoint is average-ed and displayed.

I am looking into the metric name, As we have 3 different duration values, I hope the duration is not overall duration.

• Init Duration
• Duration (Is this relates to a specific operation or simple duration?)
• Billed Duration

[muthu-mps]

As we stick not to change the names for the memory panels. Change looks good!

[elastic-vault-github-plugin-prod]

Package aws - 3.7.0 containing this change is available at https://epr.elastic.co/package/aws/3.7.0/