Skip to content

[Azure Blob Storage] Adding integration for Custom Azure Blob Storage Input - #4693

Merged
P1llus merged 8 commits into
elastic:mainfrom
P1llus:package_abs
Feb 7, 2023
Merged

[Azure Blob Storage] Adding integration for Custom Azure Blob Storage Input#4693
P1llus merged 8 commits into
elastic:mainfrom
P1llus:package_abs

Conversation

@P1llus

@P1llus P1llus commented Nov 22, 2022 •
edited
Loading

Copy link
Copy Markdown
Member

What does this PR do?

This adds support for using the custom Azure Blob Storage Input that was added to filebeat in 8.5.0.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

Related issues
@elasticmachine

elasticmachine commented Nov 22, 2022

Copy link
Copy Markdown

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)
@P1llus

P1llus commented Nov 22, 2022

Copy link
Copy Markdown
Member Author

Similar to #4692

There is currently a few questions up for discussion here:

  1. Will we add system tests, and how we should do it.
  2. Which ECS fields should we use, continuing a discussion around if we should include at least fields related to common metadata processors.
  3. Needs a new icon.
@P1llus
P1llus requested a review from a team November 22, 2022 11:54
@elasticmachine

elasticmachine commented Nov 22, 2022 •
edited
Loading

Copy link
Copy Markdown

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2023-02-07T08:53:37.581+0000

  • Duration: 14 min 4 sec

Test stats 🧪

Test Results
Failed 0
Passed 3
Skipped 0
Total 3

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.
@P1llus

P1llus commented Nov 22, 2022

Copy link
Copy Markdown
Member Author

Integration has not been fully tested yet, but will be soon (before merging)
@elasticmachine

elasticmachine commented Nov 22, 2022 •
edited
Loading

Copy link
Copy Markdown

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (0/0) 💚
Files 100.0% (0/0) 💚
Classes 100.0% (0/0) 💚
Methods 100.0% (3/3) 💚
Lines 100.0% (0/0) 💚
Conditionals 100.0% (0/0) 💚
kcreddy
kcreddy approved these changes Dec 1, 2022
View reviewed changes

@kcreddy kcreddy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍🏼
Once we have a way to run system tests, we could change to GA.
@botelastic

botelastic Bot commented Dec 31, 2022

Copy link
Copy Markdown

Hi! We just realized that we haven't looked into this PR in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!
@botelastic botelastic Bot added the Stalled label Dec 31, 2022
@narph

narph commented Jan 9, 2023

Copy link
Copy Markdown
Contributor

@P1llus , can we get this in?
@botelastic botelastic Bot removed the Stalled label Jan 9, 2023
@P1llus

P1llus commented Jan 16, 2023

Copy link
Copy Markdown
Member Author

Was waiting for elastic/stream#47 to finalize and be merged, and wanted to wait for the dynamic ECS template functionality that was very recently merged here: elastic/elastic-package#1073 so that we could include this functionality from the initial version of the package.
@P1llus

P1llus commented Jan 19, 2023

Copy link
Copy Markdown
Member Author

All blocking features have now been resolved, will just have to update the PR with the system test and add dynamic ECS template before merging: #5055
@P1llus

P1llus commented Jan 23, 2023

Copy link
Copy Markdown
Member Author

This new input needs to be added to the filebeat spec file for elastic-agent first: https://github.com/elastic/elastic-agent/blob/main/specs/filebeat.spec.yml
@P1llus P1llus mentioned this pull request Jan 23, 2023
Merged
6 tasks
@P1llus

P1llus commented Jan 23, 2023

Copy link
Copy Markdown
Member Author

Spec file updated as per: elastic/elastic-agent#2149
Awaiting new snapshot image to be generated.
ShourieG
ShourieG reviewed Feb 7, 2023
View reviewed changes
Comment thread packages/azure_blob_storage/_dev/build/docs/README.md Outdated
ShourieG
ShourieG reviewed Feb 7, 2023
View reviewed changes
Comment thread packages/azure_blob_storage/docs/README.md Outdated
ShourieG
ShourieG approved these changes Feb 7, 2023
View reviewed changes

@ShourieG ShourieG left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor doc changes otherwise LGTM
P1llus and others added 2 commits February 7, 2023 09:52
@P1llus @ShourieG
Update packages/azure_blob_storage/_dev/build/docs/README.md
4846bec 
Co-authored-by: ShourieG <105607378+ShourieG@users.noreply.github.com>
@P1llus
Update docs
13dc143
@P1llus
P1llus merged commit 6a6d42b into elastic:main Feb 7, 2023
@elasticmachine

elasticmachine commented Feb 7, 2023

Copy link
Copy Markdown

Package azure_blob_storage - 0.1.0 containing this change is available at https://epr.elastic.co/search?package=azure_blob_storage
@andrewkroh andrewkroh added the New Integration Issue or pull request for creating a new integration package. label Aug 13, 2024
orestisfl pushed a commit to orestisfl/integrations that referenced this pull request May 15, 2026
@P1llus @ShourieG
[Azure Blob Storage] Adding integration for Custom Azure Blob Storage…
e7ad354 
… Input (elastic#4693)

* Initial commit

* add changelog and CODEOWNERS

* stashing changes

* First MVP version

* adding tags

* Update packages/azure_blob_storage/_dev/build/docs/README.md

Co-authored-by: ShourieG <105607378+ShourieG@users.noreply.github.com>

* Update docs

---------

Co-authored-by: ShourieG <105607378+ShourieG@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Integration:azure_blob_storage Custom Azure Blob Storage Input New Integration Issue or pull request for creating a new integration package.

6 participants

@P1llus @elasticmachine @narph @kcreddy @ShourieG @andrewkroh