[azure-eventhub] Add support for managed identity

[zmoog]

Proposed commit message

Add support for managed identity authentication to the azure-eventhub input.

The input support both system and user-assigned identities.

System-assigned managed identity:

If you want to use system-assigned identity, set auth_type: managed_identity and the usual event hub and storage account related settings. Not need to specify authentication settings other than auth_type.

filebeat.inputs:
- type: azure-eventhub
  auth_type: "managed_identity"
  eventhub: "insights-operational-logs"
  eventhub_namespace: "your-namespace.servicebus.windows.net"
  consumer_group: "$Default"
  storage_account: "your-storage-account"

User-assigned managed identity:

If you wan to use a user-assigned identity, you need to set managed_identity_client_id.

filebeat.inputs:
- type: azure-eventhub
  auth_type: "managed_identity"
  managed_identity_client_id: "your-user-assigned-identity-client-id"
  eventhub: "insights-operational-logs"
  eventhub_namespace: "your-namespace.servicebus.windows.net"
  consumer_group: "$Default"
  storage_account: "your-storage-account"

Managed identity allows users to deploy the Elastic Agent with including credentials.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works. Where relevant, I have used the stresstest.sh script to run them under stress conditions and race detector to verify their stability.
• I have added an entry in ./changelog/fragments using the changelog tool.

How to test this PR locally

You can follow the detailed step-by-step guide at zmoog/public-notes#118

Related issues

• Relates [azure-eventhub] Add support for Managed Identity #48680

[github-actions]

🤖 GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @zmoog? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit
• backport-active-all is the label that automatically backports to all active branches.
• backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
• backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

[github-actions]

🔍 Preview links for changed docs

• docs/reference/filebeat/filebeat-input-azure-eventhub.md

[kaiyan-sheng]

LGTM!

[theletterf]

Made some suggestions!

[github-actions]

Vale Linting Results

Summary: 1 suggestion found

💡 Suggestions (1)

File	Line	Rule	Message
docs/reference/filebeat/filebeat-input-azure-eventhub.md	77	Elastic.WordChoice	Consider using 'refer to (if it's a document), view (if it's a UI element)' instead of 'See', unless the term is in the UI.

---

The Vale linter checks documentation changes against the Elastic Docs style guide.

To use Vale locally or report issues, refer to Elastic style guide for Vale.

[github-actions]

@Mergifyio backport 8.19 9.2 9.3

[mergify]

backport 8.19 9.2 9.3

✅ Backports have been created

Details

• #48843 [8.19](backport #48655) [azure-eventhub] Add support for managed identity has been created for branch 8.19 but encountered conflicts

Cherry-pick of 4f42aad has failed:

On branch mergify/bp/8.19/pr-48655
Your branch is up to date with 'origin/8.19'.

You are currently cherry-picking commit 4f42aad31.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	new file:   changelog/fragments/1770154527-azure-eventhub-managed-identity.yaml
	modified:   x-pack/filebeat/input/azureeventhub/auth.go
	modified:   x-pack/filebeat/input/azureeventhub/config.go
	modified:   x-pack/filebeat/input/azureeventhub/config_test.go
	new file:   x-pack/filebeat/input/azureeventhub/managed_identity.go
	new file:   x-pack/filebeat/input/azureeventhub/managed_identity_test.go

Unmerged paths:
  (use "git add/rm <file>..." as appropriate to mark resolution)
	deleted by us:   docs/reference/filebeat/filebeat-input-azure-eventhub.md

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

• #48844 [9.2](backport #48655) [azure-eventhub] Add support for managed identity has been created for branch 9.2
• #48845 [9.3](backport #48655) [azure-eventhub] Add support for managed identity has been created for branch 9.3