Skip to content

AWS Network Firewall integration - #2199

Merged
taylor-swanson merged 19 commits into
elastic:masterfrom
taylor-swanson:aws-firewall
Dec 13, 2021
Merged

AWS Network Firewall integration#2199
taylor-swanson merged 19 commits into
elastic:masterfrom
taylor-swanson:aws-firewall

Conversation

@taylor-swanson

@taylor-swanson taylor-swanson commented Nov 19, 2021 •
edited
Loading

Copy link
Copy Markdown
Contributor

What does this PR do?

This PR adds a new integration for AWS Network Firewall.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • If I'm introducing a new feature, I have modified the Kibana version constraint in my package's manifest.yml file to point to the latest Elastic stack release (e.g. ^7.13.0).

How to test this PR locally

cd packages/aws
elastic-package test pipeline

Related issues

Relates #1204

Screenshots

aws-firewall

filebeat-aws-firewall-overview
filebeat-aws-firewall-alerts
filebeat-aws-firewall-flows
metricbeat-aws-firewall-overview
@taylor-swanson
AWS Network Firewall integration
3ddbd1e 
- Add integration for AWS Network Firewall
@elasticmachine

elasticmachine commented Nov 19, 2021 •
edited
Loading

Copy link
Copy Markdown

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2021-12-13T19:38:19.718+0000

  • Duration: 27 min 1 sec

  • Commit: 6d7ba48

Test stats 🧪

Test Results
Failed 0
Passed 315
Skipped 0
Total 315

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.
legoguy1000
legoguy1000 reviewed Nov 23, 2021
View reviewed changes
Comment thread packages/aws/data_stream/firewall_logs/elasticsearch/ingest_pipeline/default.yml Outdated
Comment thread packages/aws/data_stream/firewall_logs/elasticsearch/ingest_pipeline/default.yml Outdated
Comment thread packages/aws/data_stream/firewall_logs/elasticsearch/ingest_pipeline/default.yml
legoguy1000
legoguy1000 reviewed Nov 29, 2021
View reviewed changes
Comment thread packages/aws/data_stream/firewall_logs/elasticsearch/ingest_pipeline/default.yml Outdated
@taylor-swanson
taylor-swanson marked this pull request as ready for review November 29, 2021 16:09
@elasticmachine

elasticmachine commented Nov 29, 2021

Copy link
Copy Markdown

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)
andrewkroh
andrewkroh reviewed Nov 30, 2021
View reviewed changes

@andrewkroh andrewkroh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice dashboards
Comment thread packages/aws/data_stream/firewall_logs/elasticsearch/ingest_pipeline/default.yml
@andrewkroh
andrewkroh requested a review from a team November 30, 2021 00:57
@andrewkroh

andrewkroh commented Nov 30, 2021

Copy link
Copy Markdown
Contributor

We should get the integrations team to review as well since they own the package.
andrewkroh
andrewkroh reviewed Nov 30, 2021
View reviewed changes
Comment thread packages/aws/data_stream/firewall_logs/elasticsearch/ingest_pipeline/default.yml Outdated
Comment thread packages/aws/data_stream/firewall_logs/elasticsearch/ingest_pipeline/default.yml
Comment thread packages/aws/data_stream/firewall_logs/elasticsearch/ingest_pipeline/default.yml
@jamiehynds jamiehynds added New Integration Issue or pull request for creating a new integration package. Integration:aws AWS labels Dec 8, 2021
@jamiehynds jamiehynds mentioned this pull request Dec 8, 2021
Closed
8 tasks
@kaiyan-sheng

kaiyan-sheng commented Dec 8, 2021

Copy link
Copy Markdown

Hi @taylor-swanson , this looks great!! One small thing: on the dashboard, could you edit the visualization title so [Logs AWS] and [Metrics AWS] don't show? [Logs AWS] and [Metrics AWS] should still be a part of the visualization title but just not displayed on the dashboard. Thanks!
@taylor-swanson

taylor-swanson commented Dec 8, 2021

Copy link
Copy Markdown
Contributor Author

Hi @taylor-swanson , this looks great!! One small thing: on the dashboard, could you edit the visualization title so [Logs AWS] and [Metrics AWS] don't show? [Logs AWS] and [Metrics AWS] should still be a part of the visualization title but just not displayed on the dashboard. Thanks!

I can certainly do that!
@taylor-swanson

taylor-swanson commented Dec 9, 2021

Copy link
Copy Markdown
Contributor Author

The visualizations have been updated to not include [Logs AWS] or [Metrics AWS] (on the dashboard itself). I also split the custom action packets from the rest of the packet metrics. If a customer didn't set up custom actions, this actually broke the graph (one of the vertical axes was missing).

Screen Shot 2021-12-09 at 10 02 20 AM

I can also collect new/updated screenshots, but I may hold off on that until any other issues are addressed. It's a bit cumbersome to get data to show up on the dashboards as I have to push live traffic through AWS.
kaiyan-sheng
kaiyan-sheng approved these changes Dec 9, 2021
View reviewed changes

@kaiyan-sheng kaiyan-sheng left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!
@taylor-swanson
taylor-swanson merged commit 9cba81e into elastic:master Dec 13, 2021
@taylor-swanson
taylor-swanson deleted the aws-firewall branch December 14, 2021 14:23
@jamiehynds

jamiehynds commented Jan 10, 2022

Copy link
Copy Markdown

@kaiyan-sheng anything stopping us from pushing this integration to production, or is there a need to wait until the next stack release?
@kaiyan-sheng

kaiyan-sheng commented Jan 10, 2022

Copy link
Copy Markdown

@jamiehynds Nothing is stopping us from pushing this to production 🙂 We just haven't done a package promotion in a while for AWS. Let me run the promote command right now.
@jamiehynds

jamiehynds commented Jan 11, 2022

Copy link
Copy Markdown

Thanks @kaiyan-sheng! On a related note, I created this issue to discuss automation of package promotion if you want to chime in: elastic/elastic-package#642

Once AWS Network Firewall is promoted, I'll get it added to the integrations page on the website and on the docs site too.
orestisfl pushed a commit to orestisfl/integrations that referenced this pull request May 15, 2026
@taylor-swanson
AWS Network Firewall integration (elastic#2199)
2056394 
- Added integration for AWS Network Firewall
- Added 2 data streams to collect logs and metrics
- Added ingest pipeline and tests for logs
- Added dashboards and visualizations for logs and metrics
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request Integration:aws AWS New Integration Issue or pull request for creating a new integration package.

6 participants

@taylor-swanson @elasticmachine @andrewkroh @kaiyan-sheng @jamiehynds @legoguy1000