[filebeat] [s3 input] Use configured region when non_aws_bucket_name is defined

[tetianakravchenko]

Proposed commit message

• WHAT: use user-configured region when non_aws_bucket_name is defined
• WHY: To avoid calling GetBucketLocation API to determine the bucket region, when non_aws_bucket_name is used for S3-compatible services (e.g., Alibaba Cloud OSS, MinIO, DigitalOcean Spaces)

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works. Where relevant, I have used the stresstest.sh script to run them under stress conditions and race detector to verify their stability.
• I have added an entry in ./changelog/fragments using the changelog tool.

Disruptive User Impact

Now region configuration parameter is required when non_aws_bucket_name is set

Author's Checklist

• [ ]

How to test this PR locally

Related issues

• Closes S3 Input Calls GetBucketLocation for Non-AWS Buckets #47847

Use cases

Screenshots

Logs

[github-actions]

🤖 GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @tetianakravchenko? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit
• backport-active-all is the label that automatically backports to all active branches.
• backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
• backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

[github-actions]

🔍 Preview links for changed docs

• docs/reference/filebeat/filebeat-input-aws-s3.md

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[elasticmachine]

Pinging @elastic/obs-ds-hosted-services (Team:obs-ds-hosted-services)

[mauri870]

I skimmed through the changes and they look fine to me, but I'm not very familiar with the S3 input to give detailed feedback.

[Kavindu-Dodan]

LGTM 🙏

I left one nit - https://github.com/elastic/beats/pull/48534/changes#r2783833944 feel free to check if it makes sense and merge when ready

[github-actions]

✅ Vale Linting Results

No issues found on modified lines!

---

The Vale linter checks documentation changes against the Elastic Docs style guide.

To use Vale locally or report issues, refer to Elastic style guide for Vale.

[github-actions]

@Mergifyio backport 8.19 9.2 9.3

[mergify]

backport 8.19 9.2 9.3

✅ Backports have been created

Details

• #48780 [8.19](backport #48534) [filebeat] [s3 input] Use configured region when non_aws_bucket_name is defined has been created for branch 8.19 but encountered conflicts

Cherry-pick of b34256a has failed:

On branch mergify/bp/8.19/pr-48534
Your branch is up to date with 'origin/8.19'.

You are currently cherry-picking commit b34256a57.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	new file:   changelog/fragments/1769424344-filebeat-awss3-input-fix-region-detection.yaml
	modified:   x-pack/filebeat/_meta/config/filebeat.inputs.reference.xpack.yml.tmpl
	modified:   x-pack/filebeat/filebeat.reference.yml
	modified:   x-pack/filebeat/input/awss3/config.go
	modified:   x-pack/filebeat/input/awss3/config_test.go
	modified:   x-pack/filebeat/input/awss3/s3.go
	modified:   x-pack/filebeat/input/awss3/s3_test.go

Unmerged paths:
  (use "git add/rm <file>..." as appropriate to mark resolution)
	deleted by us:   docs/reference/filebeat/filebeat-input-aws-s3.md

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

• #48781 [9.2](backport #48534) [filebeat] [s3 input] Use configured region when non_aws_bucket_name is defined has been created for branch 9.2
• #48782 [9.3](backport #48534) [filebeat] [s3 input] Use configured region when non_aws_bucket_name is defined has been created for branch 9.3