[AWS] [API Gateway] Split metrics dashboard by API type and filter documents visualized

[lucian-ioan]

Urgency

• High

Activity Type

• Bugfix

What does this PR do?

1. Split the existing API Gateway metrics dashboard into 3 dashboards by API type.
2. Add additional metric counters and navigation panels.
3. Change visualization types to enhance clarity.
4. Filter data so only documents with API Identifier and Stage dimensions are considered. This is necessary due to additional documents being fetched based on API settings (ex. Detailed Metrics) which will result in inaccurate metrics (ex. Count).
5. Additional metric filters within the HTTP/WebSocket dashboards to prevent the redundant display of API IDs in both dashboards. (ApiId dimension is shared between HTTP and WebSocket)

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

• Remove [Metrics AWS] API Gateway Overview dashboard.
• Export all dashboards to 8.9
• Check all data is 1:1 with CloudWatch metric groupings (By Stage - REST and ApiId, Stage - HTTP/WebSocket).

How to test this PR locally

Related issues

• Closes Metrics dashboard approach for AWS API Gateway when Detailed Metrics is enabled #7837

Screenshots

REST:

HTTP:

WebSocket:

Example CloudWatch:

Example REST:

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2023-09-27T00:44:57.880+0000

• Duration: 54 min 14 sec

Test stats 🧪

Test	Results
Failed	0
Passed	211
Skipped	3
Total	214

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

[elasticmachine]

🌐 Coverage report

Name	Metrics % (covered/total)	Diff
Packages	100.0% (17/17)	💚
Files	94.444% (17/18)	👎 -5.556
Classes	94.444% (17/18)	👎 -5.556
Methods	85.953% (257/299)	👎 -6.639
Lines	86.024% (7509/8729)	👎 -1.654
Conditionals	100.0% (0/0)	💚

[kaiyan-sheng]

Overall it looks good to me. Not sure if users will like the separated dashboards for different kinds of API gateway. But we can always adjust later if we get feedback.
Can we add a screenshot (not sure if multiple screenshots is allowed) into documentation please? Thanks!

[zmoog]

LGTM.

I believe separating the single dashboard into multiple ones per type is fair. I don't have data to back this up, but in my experience as an application developer, I usually check one API at a time;.

[drewdaemon]

This better follows dashboard best practices. And, when the links panel arrives, this will be an excellent use case for it.

With the new links panel, navigation between the views is much more performant. The dashboard app itself doesn't have to reload when a link is clicked, minimizing the work it takes to show another API type.

With the links panel, filters and query are also preserved across navigation events.

[elasticmachine]

Package aws - 2.4.0 containing this change is available at https://epr.elastic.co/search?package=aws

[zmoog]

This better follows dashboard best practices.

@drewdaemon, what's your recommended reading list to know more about dashboard best practices?

[drewdaemon]

@zmoog unfortunately fragmentation is a problem right now. My team created a document. Some designers created a document. Someone from Security solution created a document. Looks like someone from your team tried to collect ideas into a markdown file.

I agree with everything I saw in the markdown file, so you could start there. I can send you our team's document as well. But, basically, we probably need to do some unification.

The specific "best practice" I was referring to is just to keep dashboards a bit smaller to improve comprehensibility and performance. Markdown panels for navigation work okay, but aren't perfect. I think the new links panel will make following this practice easier.