Skip to content

Azure Firewall integration - #3578

Merged
ebeahan merged 37 commits into
elastic:mainfrom
ebeahan:package-azure-firewall
Jul 8, 2022
Merged

Azure Firewall integration#3578
ebeahan merged 37 commits into
elastic:mainfrom
ebeahan:package-azure-firewall

Conversation

@ebeahan

@ebeahan ebeahan commented Jun 23, 2022

Copy link
Copy Markdown
Member

What does this PR do?

This PR adds a new integration for Azure Firewall.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

How to test this PR locally

cd packages/azure
elastic-package test pipeline

Related issues

Relates to #1205

Screenshots

Screen Shot 2022-06-23 at 3 13 20 PM

Screen Shot 2022-06-23 at 3 15 21 PM

filebeat-azure-firewall-overview

filebeat-azure-firewall-dns-proxy

@ebeahan ebeahan added enhancement New feature or request Team:Security-External Integrations Integration:azure Azure Logs New Integration Issue or pull request for creating a new integration package. labels Jun 23, 2022
@ebeahan
ebeahan requested a review from a team as a code owner June 23, 2022 20:17
@elasticmachine

Copy link
Copy Markdown

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@elasticmachine

elasticmachine commented Jun 23, 2022

Copy link
Copy Markdown

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-07-08T14:17:03.279+0000

  • Duration: 15 min 58 sec

Test stats 🧪

Test Results
Failed 0
Passed 105
Skipped 0
Total 105

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.
@elasticmachine

elasticmachine commented Jun 23, 2022

Copy link
Copy Markdown

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (7/7) 💚
Files 81.25% (13/16) 👎 -15.676
Classes 81.25% (13/16) 👎 -15.676
Methods 80.451% (107/133) 👎 -8.894
Lines 81.81% (1853/2265) 👎 -8.403
Conditionals 100.0% (0/0) 💚
@andrewkroh
andrewkroh requested a review from a team June 23, 2022 21:37
Comment thread packages/azure/_dev/build/docs/firewall_logs.md Outdated
Comment on lines 41 to 44

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

network_traffic has this as a long for the icmp datastream, but kubernetes has it as a keyword for an HTTP request code. I can't see it defined, so is there a preference?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless a numeric field needs to support range queries, ES recommends keyword for the better term query performance.

For ICMP messages, I think users are more likely query for a specific request code than across a range.

Comment thread packages/azure/data_stream/firewall_logs/agent/stream/azure-eventhub.yml.hbs Outdated
Comment thread packages/azure/data_stream/firewall_logs/sample_event.json Outdated
@ebeahan
ebeahan requested a review from a team July 6, 2022 20:16
@ebeahan

ebeahan commented Jul 6, 2022

Copy link
Copy Markdown
Member Author

Last round of comments have been addressed - ready for another review round.

Comment thread packages/azure/data_stream/firewall_logs/agent/stream/log.yml.hbs Outdated
Comment thread packages/azure/_dev/build/docs/firewall_logs.md Outdated
Comment thread packages/azure/_dev/build/docs/firewall_logs.md Outdated
Comment thread packages/azure/_dev/build/docs/firewall_logs.md Outdated
@ebeahan

ebeahan commented Jul 7, 2022

Copy link
Copy Markdown
Member Author

Thanks for the feedback, @efd6. I've incorporated your suggestions.

@efd6 efd6 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ebeahan
ebeahan force-pushed the package-azure-firewall branch from a4721e7 to 7cdd3df Compare July 8, 2022 14:16
@ebeahan

ebeahan commented Jul 8, 2022

Copy link
Copy Markdown
Member Author

@elastic/obs-cloud-monitoring would you also please review as package owners?

@kaiyan-sheng kaiyan-sheng left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@ebeahan
ebeahan merged commit baa50ca into elastic:main Jul 8, 2022
@ebeahan
ebeahan deleted the package-azure-firewall branch July 8, 2022 16:32
@ebeahan ebeahan mentioned this pull request Jul 8, 2022
15 tasks
orestisfl pushed a commit to orestisfl/integrations that referenced this pull request May 15, 2026
* adding foundation of firewall data stream

* switch to grok processor for network pipeline

* improve network pipeline for additional log cases

* migrate to using grok for app rules

* migrate dns proxy to also use grok processor

* remove unneeded processors

* improve dns test cases

* consolidate into one firewall pipeline

* unneeded removes

* add docs and sample

* include firewall in top-level readme

* add firewall logo svg

* rename firewall -> firewall_logs

* set cloud.account.id

* use correct constants

* use unique name for storage account container

* handle additional app rule case

* add dashboards

* add additional app log case

* clean up field defs

* update firewall doc

* correct case

* correct observer.name

* add firewall screenshots

* follow new conventions

* formatting

* bump version and update changelog

* s/filebeat azure module/Azure Logs package/

* sort sample and expected events

* index not needed

* simplify grok with case insensitivity

* regenerate firewall_logs with updated sample event

* bump ECS version

* clean up formatting

* doc improvements based on review feedback

* use v8.2.0

* use 8.3.0 in pipeline not field defs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request Integration:azure Azure Logs New Integration Issue or pull request for creating a new integration package.

4 participants